Data processing circuit, method, storage medium, chip and electronic device

Abstract

The present disclosure provides a data processing circuit, method, storage medium, chip and electronic device. The circuit comprises a preprocessing module and an operation module. The preprocessing module is configured to cache input feature data based on a set number of elements, and when a number of to-be-determined elements in first converted feature data corresponding to the input feature data is determined to be equal to the number of elements, determine element values of the number of to-be-determined elements based on a plurality of target local elements on the determined input feature data, and input the element values of the to-be-determined elements to the operation module. The number of elements is the number of elements processed in parallel by the circuit. The plurality of target local elements include each local element required when the number of to-be-determined elements is equal to the number of elements. The operation module is configured to generate output feature data corresponding to the input feature data based on the received element values of the to-be-determined elements.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a data processing circuit, method, storage medium, chip, and electronic device. Background Technology

[0002] With the research and development of artificial intelligence, it has been widely applied to various aspects of people's lives, work, and entertainment. Artificial intelligence can include multiple technologies such as machine learning and computer vision; therefore, research on neural networks can promote or accelerate the development of artificial intelligence.

[0003] Generally, a neural network can include at least one feature processing layer, such as a convolutional layer or a pooling layer. The convolution operation, corresponding to the convolutional layer, is one of the main operations in a neural network, and its efficiency plays a crucial role in the overall computational efficiency of the network. Therefore, designing a circuit capable of performing convolution operations efficiently is particularly important. Summary of the Invention

[0004] In view of the above, this disclosure provides at least one data processing circuit, method, storage medium, chip, and electronic device.

[0005] In a first aspect, this disclosure provides a data processing circuit, including: a preprocessing module and a calculation module;

[0006] The preprocessing module is used to cache the input feature data based on a set number of elements, and when it is determined that there are a number of elements to be determined in the first transformed feature data corresponding to the input feature data, it determines the element values ​​of the number of elements to be determined based on multiple target local elements on the determined input feature data; and inputs the element values ​​of the elements to be determined to the calculation module.

[0007] Wherein, the number of elements is the number of elements that the circuit processes in parallel; the plurality of target local elements includes each local element required to determine the number of elements to be determined;

[0008] The calculation module is used to generate output feature data corresponding to the input feature data based on the received element value of the element to be determined.

[0009] Here, the circuit includes a preprocessing module and a computation module. The preprocessing module caches the input feature data based on a set number of elements, resulting in fast caching. Simultaneously, the preprocessing module also determines the element values ​​of a certain number of undetermined elements in the first transformed feature data corresponding to the input feature data, based on multiple target local elements in the determined input feature data. By determining the element values ​​of these undetermined elements in parallel, the efficiency of generating the first transformed feature data is improved. Furthermore, the computation module, based on the received element values ​​of the undetermined elements, can efficiently generate the output feature data corresponding to the input feature data, thus realizing the convolution operation process.

[0010] In one possible implementation, the preprocessing module includes a buffer and an accumulator; the accumulator includes a first accumulator with a number of elements.

[0011] The buffer is used to cache multiple local elements within the current window obtained from the input feature data sequentially according to a set number of elements; and

[0012] When determining the number of elements to be determined in the first transformed feature data corresponding to the input feature data, in response to the buffer including multiple target local elements, the multiple target local elements are output to the accumulation module;

[0013] Each first accumulator in the accumulation module is used to obtain the local elements required by the first accumulator to determine the element to be determined from the plurality of target local elements; and to accumulate the obtained local elements to obtain the element value corresponding to the element to be determined.

[0014] In practice, the cache is used to cache multiple local elements within the current window obtained from the input feature data in sequence according to the set number of elements, thereby improving the efficiency of the cache in caching multiple local elements within the current window.

[0015] Meanwhile, when the buffer determines the number of elements to be determined in the first transformed feature data, in response to the buffer including multiple target local elements, the multiple target local elements are output to the accumulation module; each first accumulator in the accumulation module is used to obtain the local elements required by the first accumulator to determine the element to be determined from the multiple target local elements; and the obtained local elements are accumulated to obtain the element value corresponding to the element to be determined, thus realizing the determination of the first transformed feature data, and determining the element value of the number of elements to be determined in parallel, thereby improving the efficiency of generating the first transformed feature data.

[0016] In one possible implementation, the number of storage spaces included in the buffer is determined based on the number of elements and the conversion relationship information between the input feature data and the first converted feature data; and / or, the storage capacity corresponding to each storage space in the buffer is determined based on the number of channels of the input feature data; wherein, the storage capacity is positively correlated with the number of channels.

[0017] Here, the amount of storage space included in the buffer can be determined according to the set number of elements and the transformation relationship between the input feature data and the first transformed feature data. This avoids wasting storage resources when the storage space is set too large, and also avoids the inability to calculate subsequent elements when the storage space is set too small, thus improving the utilization rate of the buffer. And / or, based on the number of channels of the input feature data, the storage capacity corresponding to each storage space on the buffer can be determined more accurately, so that the storage space of the buffer can cache local elements on each channel, realize parallel computing across multiple input channels, and improve computational efficiency.

[0018] In one possible implementation, the computation module includes a dot product processing module and a post-processing module; the dot product processing module is connected to the accumulation module and the post-processing module, respectively.

[0019] The dot product processing module is used to receive the element value of the element to be determined transmitted by the first accumulator; and based on the element value of the element to be determined and the obtained weight feature number, determine the second transformed feature data corresponding to the first transformed feature data; and input the second transformed feature data to the post-processing module.

[0020] The post-processing module is used to generate output feature data corresponding to the input feature data based on the element values ​​of each intermediate element included in the second transformed feature data.

[0021] Here, the dot product processing module can determine the second transformed feature data corresponding to the first transformed feature data based on the element value of the element to be determined and the number of weighted features obtained; the post-processing module generates the output feature data corresponding to the input feature data based on the element value of each intermediate element included in the second transformed feature data, thus realizing a fast convolution operation process.

[0022] In one possible implementation, the computation module includes a dot product processing module and a post-processing module; the dot product processing module is connected to the accumulation module and the post-processing module, respectively.

[0023] The dot product processing module includes: the number of dot product units; each dot product unit is connected to a first accumulator;

[0024] The dot product is used to receive the element value of the element to be determined output by the first accumulator; and multiply the element value of the element to be determined by the weight value corresponding to the element to be determined in the weight feature data to obtain the element value of the intermediate element that matches the element to be determined in the second transformed feature data corresponding to the first transformed feature data.

[0025] The post-processing module is used to generate output feature data corresponding to the input feature data based on the element values ​​of each intermediate element included in the received second transformed feature data.

[0026] Here, the dot product multiplies the received element value of the element to be determined with the weight value corresponding to the element in the weighted feature data, obtaining the element value of the intermediate element matching the element to be determined in the second transformed feature data corresponding to the first transformed feature data. The number of dot product units is equal to the number of elements, indicating that multiple dot product units can obtain the element values ​​of multiple intermediate elements in parallel, improving computational efficiency. Furthermore, the post-processing module can generate output feature data based on the element values ​​of each intermediate element included in the received second transformed feature data, improving the determination efficiency of the output feature data.

[0027] In one possible implementation, the dot product processing module further includes: at least one adder; the adder is connected to a plurality of matched dot product units;

[0028] The adder is used to receive the element values ​​of the intermediate elements output by the multiple connected dot productrs, merge the element values ​​of the multiple intermediate elements to obtain an intermediate merged value, and input the intermediate merged value to the post-processing module.

[0029] The post-processing module is used to generate output feature data corresponding to the input feature data based on the element value of the target element; wherein the element value of the target element includes the intermediate fusion value and / or the element value of the intermediate element.

[0030] Here, the adder receives the element values ​​of intermediate elements from multiple connected dot productors, and fuses these intermediate element values ​​to obtain an intermediate fused value. This intermediate fused value is then input to the post-processing module, where it serves as the intermediate value required for the post-processing module to determine the output feature data. Consequently, the post-processing module can quickly generate the output feature data based on the element values ​​of the target element, thus improving computational efficiency.

[0031] In one possible implementation, the post-processing module includes: a plurality of second accumulators; the number of the second accumulators matches the number of output elements included in the output feature data to be determined; each second accumulator is matched with one of the output elements;

[0032] The second accumulator is used to select a desired element value from the element values ​​of the input target element; and to determine the output element based on the selected desired element value.

[0033] Each of the output elements generates the output feature data.

[0034] Here, the second accumulator selects the required element value from the element values ​​of the input target element; based on the selected required element value, it determines the output element; wherein, each output element generates output feature data; thus, multiple output elements are obtained in parallel, improving computational efficiency.

[0035] In one possible implementation, the post-processing module further includes: a plurality of data selectors; the number of data selectors matches the number of the second accumulators; each data selector is connected to a matched second accumulator;

[0036] The data selector is used to determine the desired element value corresponding to the second accumulator from the element values ​​of the input target element; and send the determined desired element value to the second accumulator.

[0037] Here, the data selector determines the required element value corresponding to the second accumulator from the element values ​​of the input target element; and sends the determined required element value to the second accumulator. The second accumulator accumulates the received data to obtain the output element, that is, the output feature data, thus completing the fast convolution operation and improving the data processing efficiency.

[0038] The effects of the following devices and electronic equipment are described in the circuit description above, and will not be repeated here.

[0039] In a second aspect, this disclosure provides a chip, including: a memory and a data processing circuit as described in the first aspect or any embodiment; wherein the data processing circuit is configured to acquire input feature data from the memory, generate output feature data corresponding to the input feature data, and store the output feature data in the memory.

[0040] In this embodiment of the disclosure, the data processing circuit described in the above embodiments is used to process the input feature data, which can efficiently obtain the output feature data, thereby improving the processing efficiency of the AI ​​chip.

[0041] Thirdly, this disclosure provides an electronic device, including: the data processing circuit described in the first aspect or any embodiment; and / or, the chip described in the second aspect.

[0042] Fourthly, this disclosure provides a data processing method, including:

[0043] Obtain input feature data;

[0044] The input feature data is cached based on the set number of elements; and when a number of elements to be determined are found in the first transformed feature data corresponding to the input feature data, the element values ​​of the number of elements to be determined are determined based on multiple target local elements on the determined input feature data; and the element values ​​of the elements to be determined are input to the calculation module; so that the calculation module generates the output feature data corresponding to the input feature data based on the received element values ​​of the elements to be determined.

[0045] Wherein, the number of elements is the number of elements that the circuit processes in parallel; the plurality of target local elements includes each local element required to determine the number of elements to be determined.

[0046] Fifthly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the data processing method described in the fourth aspect above.

[0047] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a fast convolution operation process provided in an embodiment of this disclosure is shown.

[0050] Figure 2 A schematic diagram of the architecture of a data processing circuit provided in an embodiment of this disclosure is shown;

[0051] Figure 3 A schematic diagram of the architecture of another data processing circuit provided in an embodiment of this disclosure is shown;

[0052] Figure 4This diagram illustrates an example of input feature data provided in an embodiment of the present disclosure.

[0053] Figure 5 This diagram illustrates a buffer in a data processing circuit provided by an embodiment of the present disclosure;

[0054] Figure 6a A schematic diagram of multiple local elements within a current window provided by an embodiment of this disclosure is shown;

[0055] Figure 6b This illustration shows a schematic diagram of the transformation relationship information between local feature data composed of multiple local elements within the current window in the input feature data and the first transformed feature data, provided by an embodiment of this disclosure.

[0056] Figure 7 The diagram illustrates a data processing circuit provided in this disclosure, in which a buffer caches multiple local elements when determining an element to be determined;

[0057] Figure 8 A schematic diagram of the architecture of another data processing circuit provided in an embodiment of this disclosure is shown;

[0058] Figure 9 A schematic diagram of the architecture of another data processing circuit provided in an embodiment of this disclosure is shown;

[0059] Figure 10 A schematic diagram of the architecture of another data processing circuit provided in an embodiment of this disclosure is shown;

[0060] Figure 11 The diagram shows an architecture schematic of a dot product processing module in a data processing circuit provided in an embodiment of this disclosure.

[0061] Figure 12 This illustration shows a schematic diagram of the architecture of a post-processing module in a data processing circuit provided in an embodiment of the present disclosure;

[0062] Figure 13 A schematic diagram of the architecture of a chip provided in an embodiment of this disclosure is shown;

[0063] Figure 14 A schematic flowchart of a data processing method provided in an embodiment of this disclosure is shown. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0065] Neural networks can include at least one feature processing layer, such as convolutional layers and pooling layers. The convolution operation, corresponding to the convolutional layer, is one of the main operations in a neural network, and its efficiency plays a crucial role in the overall computational efficiency of the neural network.

[0066] Generally, in traditional convolution calculations, the speed and performance of convolution computation can be improved by performing parallel operations across multiple input channels. In practical applications, since the number of input channels in different convolutional feature processing layers may vary, it is necessary to pre-define a channel parallelism level and process the convolution computations of the convolutional feature processing layers in parallel according to the set channel parallelism level.

[0067] To ensure that different convolutional feature processing layers have a high utilization rate of the hardware structure, a suitable channel parallelism needs to be set. This channel parallelism cannot be too large or too small, that is, the channel parallelism cannot be increased indefinitely. Therefore, the number of input channels of the convolutional feature processing layer may not be an integer multiple of the set channel parallelism.

[0068] When the number of input channels of a convolutional feature processing layer is not an integer multiple of the channel parallelism, processing the convolution operation of the convolutional feature processing layer according to the channel parallelism will result in a loss of utilization of the hardware structure running the convolution operation and reduce the efficiency of the convolution operation corresponding to the convolutional feature processing layer.

[0069] To alleviate the above problems, embodiments of this disclosure provide a data processing circuit, method, storage medium, chip, and electronic device.

[0070] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0071] Convolution operations can include Winograd fast convolution operations, see [link to Winograd fast convolution operation](link to Winograd fast convolution operation). Figure 1As shown, winograd's fast convolution operation can include S101-S103, specifically:

[0072] S101: Using the first transformation matrix that matches the input feature data, the input feature data is transformed to obtain the first transformed feature data.

[0073] For example, if the input feature data is: [d0, d1, d2, d3; d4, d5, d6, d7; d8, d9, d10, d11; d12, d13, d14, d15], then after transforming the input feature data using the first transformation matrix, the first transformed feature data is obtained. The first transformed feature data can be represented as follows: [x0, x1, x2, x3; x4, x5, x6, x7; x8, x9, x10, x11; x12, x13, x14, x15].

[0074] S102: Perform a dot product operation between the weighted feature data and the first transformed feature data to obtain the second transformed feature data.

[0075] For example, the feature data after the second transformation can be represented as follows: [m0, m1, m2, m3; m4, m5, m6, m7; m8, m9, m10, m11; m12, m13, m14, m15].

[0076] The weight feature data can be the feature data obtained after transforming the initial weight data. For example, the initial weight data can be transformed using a second transformation matrix that matches the initial weight data to obtain the weight feature data.

[0077] S103: Using the third transformation matrix that matches the second transformed feature data, transform the second transformed feature data to obtain the output feature data.

[0078] For example, the output feature data can be represented as follows: [r0, r1, r2, r3], and the relationship between the output elements in the output feature data and the elements in the second transformed feature data is as follows:

[0079] r0=m0+m1+m2+m4+m5+m6+m8+m9+m10;

[0080] r1=m1-m2-m3+m5-m6-m7+m9-m10-m11;

[0081] r2=m4+m5+m6-m8-m9-m10-m12-m13-m14;

[0082] r3=m5-m6-m7-m9+m10+m11-m13+m14+m15.

[0083] Through the above process, the input feature data can be subjected to a fast convolution operation using Winograd to obtain the output feature data corresponding to the input feature data.

[0084] Based on this, embodiments of this disclosure propose a data processing circuit for processing the input feature data of a convolutional feature processing layer. Specifically, the convolution operation corresponding to the convolutional feature processing layer may include Winograd fast convolution operation, that is, the data processing circuit can perform Winograd fast convolution operation on the input feature data of the convolutional feature processing layer to obtain the output feature data corresponding to the input feature data.

[0085] See Figure 2 The diagram shown is a schematic of the architecture of a data processing circuit provided in an embodiment of this disclosure. The circuit includes a preprocessing module 21 and a calculation module 22.

[0086] The preprocessing module 21 is used to cache the input feature data based on a set number of elements, and when a certain number of elements to be determined are found in the first transformed feature data corresponding to the input feature data, it determines the element values ​​of the number of elements to be determined based on multiple target local elements on the determined input feature data; and inputs the element values ​​of the elements to be determined to the calculation module. Here, the number of elements is the number of elements processed in parallel by the circuit; the multiple target local elements include each local element required when determining the number of elements to be determined.

[0087] The calculation module 22 is used to generate output feature data corresponding to the input feature data based on the element value of the received element to be determined.

[0088] Here, the circuit includes a preprocessing module and a computation module. The preprocessing module caches the input feature data based on a set number of elements, resulting in fast caching. Simultaneously, the preprocessing module also determines the element values ​​of a certain number of undetermined elements in the first transformed feature data corresponding to the input feature data, based on multiple target local elements in the determined input feature data. By determining the element values ​​of these undetermined elements in parallel, the efficiency of generating the first transformed feature data is improved. Furthermore, the computation module, based on the received element values ​​of the undetermined elements, can efficiently generate the output feature data corresponding to the input feature data, thus realizing the convolution operation process.

[0089] For example, the preprocessing module can implement the preprocessing process of Winograd fast convolution operation, that is, it can execute the specific process of step S101. The operation module can implement the dot product processing and postprocessing process of Winograd fast convolution operation, that is, it can execute the specific processes of steps S102 and S103.

[0090] The number of elements can be set as needed, for example, it can be 2, 4, etc. Here, the number of elements can be defined as pixel parallelism, which represents the number of elements that the data processing circuit processes in parallel. For example, when the pixel parallelism is 2 (i.e., the number of elements is 2), the data processing circuit can determine two elements in the first transformed feature data corresponding to the input feature data in parallel, and the buffer in the data processing circuit can buffer 2 elements at a time, etc.

[0091] In implementation, the preprocessing module can cache the input feature data based on a set number of elements. For example, when the number of elements is 2, the preprocessing module can store 2 local elements from the input feature data each time until it has stored all the local elements in the input feature data. Furthermore, the preprocessing module is also used to determine the element values ​​of the number of elements to be determined in the first transformed feature data corresponding to the input feature data, based on multiple target local elements in the determined input feature data, when there are a number of elements to be determined.

[0092] For example, if the preprocessing module determines an element x0 to be determined in the first transformed feature data, and determining the value of x0 requires the use of target local elements d0, d2, d8, and d10 in the input feature data, the element value of the element to be determined can be determined based on the target local elements d0, d2, d8, and d10. For instance, d0 can be subtracted from d2, the difference can be added to d8, and the sum can be added to d10 to obtain the element value of the element x0 to be determined.

[0093] The preprocessing module inputs the element values ​​of the elements to be determined into the calculation module. The calculation module generates the output feature data corresponding to the input feature data based on the received element values ​​of the elements to be determined.

[0094] In one possible implementation, see Figure 3 As shown, the preprocessing module includes a buffer 201 and an accumulator 202; the accumulator includes a first accumulator 2021 with a number of elements; the number of elements is the number of elements that the circuit processes in parallel.

[0095] Buffer 201 is used to cache multiple local elements within the current window obtained from the input feature data in sequence according to the set number of elements.

[0096] The buffer 201 is further configured to, when determining the number of elements to be determined in the first transformed feature data corresponding to the input feature data, output multiple target local elements to the accumulation module in response to the buffer including multiple target local elements; wherein the multiple target local elements include each local element required when determining the number of elements to be determined.

[0097] Each first accumulator 2021 in the accumulation module is used to obtain the local elements required by the first accumulator to determine the element to be determined from multiple target local elements; and to accumulate the obtained local elements to obtain the element value corresponding to the element to be determined.

[0098] In practice, the cache is used to cache multiple local elements within the current window obtained from the input feature data in sequence according to the set number of elements, thereby improving the efficiency of the cache in caching multiple local elements within the current window.

[0099] Meanwhile, when the buffer determines the number of elements to be determined in the first transformed feature data, in response to the buffer including multiple target local elements, the multiple target local elements are output to the accumulation module; each first accumulator in the accumulation module is used to obtain the local elements required by the first accumulator to determine the element to be determined from the multiple target local elements; and the obtained local elements are accumulated to obtain the element value corresponding to the element to be determined, thus realizing the determination of the first transformed feature data, and determining the element value of the number of elements to be determined in parallel, thereby improving the efficiency of generating the first transformed feature data.

[0100] This data processing circuit can be used to process the input feature data of the convolutional feature processing layer, where the size of the input feature data can be arbitrary. For example, the size of the input feature data can be 128 width × 128 height × 1 channel, 256 width × 256 height × 8 channels, etc.

[0101] In implementation, the buffer 201 can obtain multiple local elements within the current window from the input feature data and cache these local elements sequentially according to the set number of elements. The window size can be set as needed; for example, the window size can be 4×4, 8×8, etc. When the window size is 4×4, the window contains 16 local elements, meaning that 16 local elements can be obtained from the input feature data each time.

[0102] See Figure 4The input feature data shown has a size of n×n. When the window size is 4×4, the current window can initially be window 1. Multiple local elements within the current window are retrieved from the input feature data and cached sequentially according to the set number of elements. After caching the local elements, the current window is slid, becoming window 2. Multiple local elements within the current window are then retrieved from the input feature data and cached sequentially according to the set number of elements, until all local elements in the input feature data are cached.

[0103] See Figure 5 The cache shown in Figure 'a' is an example of a cache that includes 24 storage spaces. Each storage space in the cache stores local elements, specifically local elements c6 to c15 in window c and local elements d0 to d13 in window d. Here, c6 is a local element in window c, and d2 is a local element in window d. The current window is window d, and window c is a previous history window.

[0104] Next, if the number of elements is 2, then according to the set number of elements, the local elements d14 and d15 in the current window can be periodically cached into the cache, while c6 and c7 can be deleted from the cache, resulting in an updated cache. See [link to updated cache] for details. Figure 5 As shown in b. In one possible implementation, the number of shifts in and out is the same as the number of elements; if two local elements are shifted in, then two local elements are shifted out of the buffer.

[0105] In one possible implementation, the number of storage spaces included in the buffer is determined based on the number of elements and the conversion relationship information between the input feature data and the first converted feature data; and / or, the storage capacity corresponding to each storage space in the buffer is determined based on the number of channels of the input feature data; wherein, the storage capacity is positively correlated with the number of channels.

[0106] During implementation, the amount of storage space included in the buffer is determined based on the set number of elements and the transformation relationship information between the input feature data and the first transformed feature data. The transformation relationship information between the input feature data and the first transformed feature data can be determined according to the operational principle of winograd's fast convolution operation. That is, according to the determined transformation relationship information, multiple local elements within the current window of the input feature data are transformed to generate the first transformed feature data.

[0107] For example, see Figure 6a The multiple local elements within the current window shown, and see also... Figure 6b The transformation relationship information between the local feature data d, composed of multiple local elements within the current window in the input feature data shown, and the first transformed feature data x. Among them, Figure 6a Multiple local elements are partial elements from the input feature data. Figure 6b The first transformed feature data x shown includes 16 elements to be determined, namely x0 to x15, which are the 16 elements to be determined corresponding to the first transformed feature data. It can be seen that each element to be determined is related to a local element in the local feature data d. For example, x0 is determined based on d0, d2, d8, and d10 in the input feature data.

[0108] For example, see Figure 6b As shown, assuming the index of d is j and the index of x is i, by using the transformation relationship information between the local feature data d corresponding to the input feature data and the feature data x after the first transformation, it can be found that when determining each element to be determined, there is the following relationship between the i corresponding to the element to be determined and the j corresponding to the required input feature data: i-10≤j≤i+10.

[0109] For example, when determining x0 (i.e., i is 0), according to x0 = d0 - d2 - d8 + d10, we need to use d0, d2, d8, and d10, and the maximum value of j is 10, which satisfies the relationship j ≤ i + 10; when determining x15 (i.e., i is 15), according to x15 = d15 + d5 - d7 - d13, we need to use d5, d7, d13, and d15, and the minimum value of j is 5, which satisfies i - 10 ≤ j.

[0110] Based on the relationship between i and j, the buffer needs at least 21 first storage spaces to cache multiple local elements in the input feature data. Furthermore, since the buffer caching process is periodic and continuous, one second storage space is also needed to cache subsequent local elements. The determined storage spaces can be further adjusted based on the number of elements to determine the total number of storage spaces included in the buffer.

[0111] For example, when the pixel parallelism (i.e. the number of elements) is 2, the amount of storage space in the buffer needs to be a multiple of 2. Therefore, the number of the first storage space can be adjusted from 21 to 22, and the number of the second storage space can be adjusted from 1 to 2, so that the amount of storage space included in the buffer is 24.

[0112] See Figure 7 The cache shown in Figure a has 22 first storage spaces for caching local elements c6 to d11, and two second storage spaces for caching d12 and d13. Alternatively, see [link to relevant documentation]. Figure 7 The cache shown in b has 22 first storage spaces for caching local elements d4 to d15, and two second storage spaces for caching e10 and e11. Figure 7 The buffer shown in b is Figure 7 The buffer shown in Figure a is obtained after multiple data caching processes. In one implementation, the shift-in and shift-out values ​​are consistent with the second storage space; therefore, the number of second storage spaces can be consistent with the number of elements when setting them.

[0113] For example, when the pixel parallelism is 4, the amount of storage space in the buffer needs to be a multiple of 4. Therefore, the number of the first storage space can be adjusted from 21 to 24, and the number of the second storage space can be adjusted from 1 to 4, so that the amount of storage space included in the buffer is 28.

[0114] See Figure 7 The buffer shown in Figure c has 24 first storage spaces for caching local elements c4 to d11, and 4 second storage spaces for caching d12, d13, d14, and d15. Alternatively, see [link to relevant documentation]. Figure 7 The buffer shown in Figure d has 24 first storage spaces for caching local elements d2 to d15, and 4 second storage spaces for caching e10, e11, e12, and e13. Figure 7 The buffer shown in d is Figure 7 The buffer shown in c is obtained after multiple data caching processes.

[0115] In implementation, the storage capacity of each storage space in the buffer can be determined based on the number of channels in the input feature data. The storage capacity is positively correlated with the number of channels. For example, if the number of channels is 1, the storage capacity can be 128 bits; if the number of channels is 2, the storage capacity can be 2 × 128 = 256 bits.

[0116] When the input feature data is 128×128×4, see [link / reference]. Figure 5 The cache space shown in Figure 'c' stores local elements from four channels. For example, the storage space corresponding to d0 caches local elements d01 (first channel), d02 (second channel), d03 (third channel), and d04 (fourth channel). The storage space corresponding to d1 caches local elements d11 (first channel), d12 (second channel), d13 (third channel), and d14 (fourth channel).

[0117] Here, the amount of storage space included in the buffer can be determined according to the set number of elements and the transformation relationship between the input feature data and the first transformed feature data. This avoids wasting storage resources when the storage space is set too large, and also avoids the inability to calculate subsequent elements when the storage space is set too small, thus improving the utilization rate of the buffer. And / or, based on the number of channels of the input feature data, the storage capacity corresponding to each storage space on the buffer can be determined more accurately, so that the storage space of the buffer can cache local elements on each channel, realize parallel computing across multiple input channels, and improve computational efficiency.

[0118] When determining the number of elements to be determined in the first transformed feature data corresponding to the input feature data, if the buffer includes all the local elements required to determine the number of elements to be determined, then in response to the buffer including multiple target local elements, the multiple target local elements are output to the accumulation module. The number of target local elements is related to the number of elements.

[0119] For example, when the number of elements is 2, according to Figure 6b It can be seen that if x0 and x1 (x0 and x1 are two elements to be determined) are determined, the required target local elements include d0, d1, d2, d8, d9, and d10, and the number of target local elements is 6. If x14 and x15 are determined, the required target local elements include d5, d6, d7, d13, d14, and d15, and the number of target local elements is also 6. It can be seen that if the number of elements is 2, the buffer will output 6 target local elements to the accumulation module.

[0120] For example, when the number of elements is 4, according to Figure 6b It can be seen that if x0, x1, x2, and x3 are determined, the required target local elements include d0, d1, d2, d3, d8, d9, d10, and d11, totaling 8 target local elements. If x12, x13, x14, and x15 are determined, the required target local elements include d4, d5, d6, d7, d12, d13, d14, and d15, also totaling 8 target local elements. Therefore, if the number of elements is 4, the buffer will output 8 target local elements to the accumulation module.

[0121] The number of first accumulators in the accumulator module is equal to the number of elements, meaning each first accumulator can determine one element to be determined. For example, if the number of elements is 2, the number of first accumulators in the accumulator module is 2, and the element values ​​of 2 elements to be determined can be determined each time; if x0 and x1 are to be determined, first accumulator 1 can be used to determine the element value of x0, and first accumulator 2 can be used to determine the element value of x1.

[0122] For example, if the number of elements is 4, the number of first accumulators in the accumulator module is 4. Each time, the element values ​​of 4 elements to be determined can be determined. If x0, x1, x2, and x3 are to be determined, the element value of x0 can be determined by first accumulator 1, the element value of x1 can be determined by first accumulator 2, the element value of x2 can be determined by first accumulator 3, and the element value of x3 can be determined by first accumulator 4.

[0123] In implementation, each first accumulator in the accumulation module can obtain the local elements needed to determine the element to be determined from multiple target local elements output by the buffer; and accumulate the obtained local elements to obtain the element value corresponding to the element to be determined; then input the element value of the element to be determined to the operation module. For example, when the first accumulator 1 determines the element value of x0, it can obtain d0, d2, d8, and d10 needed to determine x0 from multiple target local elements d0, d1, d2, d3, d8, d9, d10, and d11 output by the buffer, accumulate d0, d2, d8, and d10 to obtain the element value corresponding to the element to be determined x0. The accumulation module can obtain the element values ​​of each element to be determined in the first transformed feature data.

[0124] The calculation module 22 can generate output feature data corresponding to the input feature data based on the element values ​​of each element to be determined received.

[0125] Taking a data processing circuit with 2 elements as an example, the processing procedure will be explained.

[0126] First, the buffer can obtain multiple local elements d0 to d15 within the current window from the input feature data. The multiple local elements d0 to d15 are cached sequentially according to the set number of elements (i.e., 2 elements), that is, in the order of d0, d1, d2, d3, d4, d5, d6, d7, d8, d9, d10, d11, d12, d13, d14, d15.

[0127] Secondly, based on Figure 6b The transformation relationship between the local feature data d, composed of multiple local elements within the current window, and the first transformed feature data x, shown in the input feature data, indicates that when determining the elements x0 and x1 to be determined, six target local elements, d0, d1, d2, d8, d9, and d10, are required. Therefore, when these six target local elements are cached in the buffer, in response to the buffer containing multiple target local elements, d0, d1, d2, d8, d9, and d10 are output to the accumulation module. See also... Figure 7 As shown in Figure a, when the buffer includes d0 to d10, the d0, d1, d2, d8, d9, and d10 required to determine the elements x0 and x1 to be determined are output to the accumulation module.

[0128] See Figure 7 As shown in Figure b, when determining the elements x14 and x15 to be determined, six target local elements d5, d6, d7, d13, d14, and d15 are required. When the buffer includes d5 to d15, in response to the buffer including multiple target local elements, the d5, d6, d7, d13, d14, and d15 required to determine the elements x14 and x15 to be determined are output to the accumulation module.

[0129] The accumulation module includes two first accumulators (number of elements). One first accumulator is used to determine the first element to be determined out of two elements, and the other first accumulator is used to determine the second element to be determined out of two elements to be determined. For example, when the elements to be determined include x0 and x1, first accumulator 1 can be used to determine the value of element x0, and first accumulator 2 can be used to determine the value of element x1. Therefore, first accumulator 1 can obtain d0, d2, d8, and d10 from d0, d1, d2, d8, d9, and d10, accumulate d0, d2, d8, and d10 to obtain the value of element x0; then, the value of element x0 is input to the calculation module. The first accumulator 2 can obtain d1, d2, d9, and d10 from d0, d1, d2, d8, d9, and d10, and accumulate d1, d2, d9, and d10 to obtain the element value corresponding to the element x1 to be determined; then input the element value of the element x1 to be determined to the operation module.

[0130] The calculation module generates output feature data corresponding to the input feature data based on the element values ​​of each element to be determined received.

[0131] Taking a data processing circuit with 4 elements as an example, the processing procedure will be explained.

[0132] First, the cache can obtain multiple local elements d0 to d15 within the current window from the input feature data. The multiple local elements d0 to d15 are cached sequentially according to the set number of elements (i.e., 4 elements), that is, in the order of d0, d1, d2, d3, d4, d5, d6, d7, d8, d9, d10, d11, d12, d13, d14, d15.

[0133] Secondly, based on Figure 6bThe transformation relationship between the local feature data d, composed of multiple local elements within the current window, and the first transformed feature data x, shown in the input feature data, indicates that when determining the elements x0, x1, x2, and x3 to be determined, eight target local elements d0, d1, d2, d3, d8, d9, d10, and d11 are required. Therefore, when these eight target local elements are cached in the buffer, in response to the buffer containing multiple target local elements, d0, d1, d2, d3, d8, d9, d10, and d11 are output to the accumulation module. See also... Figure 7 As shown in c, when the buffer includes d0 to d11, the d0, d1, d2, d3, d8, d9, d10, and d11 required to determine the elements x0, x1, x2, and x3 to be determined are output to the accumulation module.

[0134] See Figure 7 As shown in d, when determining the elements x12, x13, x14, and x15, eight target local elements d4, d5, d6, d7, d12, d13, d14, and d15 are required. When the buffer includes d4 to d15, in response to the buffer including multiple target local elements, the d4, d5, d6, d7, d12, d13, d14, and d15 required to determine the elements x12, x13, x14, and x15 are output to the accumulation module.

[0135] The accumulation module includes four first accumulators (number of elements in total). First accumulator 1 determines the first element to be determined out of the four elements; first accumulator 2 determines the second element; first accumulator 3 determines the third element; and first accumulator 4 determines the fourth element. For example, when the elements to be determined include x0, x1, x2, and x3, first accumulator 1 can determine the value of element x0, first accumulator 2 can determine the value of element x1, first accumulator 3 can determine the value of element x2, and first accumulator 4 can determine the value of element x3.

[0136] Specifically, the first accumulator 1 retrieves d0, d2, d8, and d10 from d0, d1, d2, d3, d8, d9, d10, and d11, and accumulates these values ​​to obtain the element value corresponding to the element x0 to be determined; then, the element value of the element x0 to be determined is input to the arithmetic module. The first accumulator 2 retrieves d1, d2, d9, and d10 from d0, d1, d2, d3, d8, d9, d10, and d11, and accumulates these values ​​to obtain the element value corresponding to the element x1 to be determined; then, the element value of the element x1 to be determined is input to the arithmetic module. The first accumulator 3 retrieves d2, d1, d9, and d10 from d0, d1, d2, d3, d8, d9, d10, and d11, and accumulates these values ​​to obtain the element value corresponding to the element x2 to be determined; then, it inputs the element value of the element x2 to the arithmetic module. The first accumulator 4 retrieves d3, d1, d9, and d11 from d0, d1, d2, d3, d8, d9, d10, and d11, and accumulates these values ​​to obtain the element value corresponding to the element x3 to be determined; then, it inputs the element value of the element x3 to the arithmetic module.

[0137] The calculation module generates output feature data corresponding to the input feature data based on the element values ​​of each element to be determined received.

[0138] The following provides a detailed explanation of the arithmetic module.

[0139] The computation module 22 may include a dot product processing module and a post-processing module. The dot product processing module is used to perform a dot product operation between the element to be determined in the first transformed feature data and the weight feature data to obtain the second transformed feature data; that is, the dot product processing module can execute the S102 process in the Winograd fast convolution operation. The post-processing module is used to transform the second transformed feature data to obtain the output feature data; that is, the post-processing module can execute the S103 process in the Winograd fast convolution operation.

[0140] In one possible implementation, see Figure 8 As shown, the operation module 22 includes a dot product processing module 221 and a post-processing module 222; the dot product processing module 221 is connected to the accumulation module 202 and the post-processing module 222 respectively.

[0141] The dot product processing module 221 is used to receive the element value of the element to be determined transmitted by the first accumulator; and based on the element value of the element to be determined and the obtained weight feature number, determine the second transformed feature data corresponding to the first transformed feature data; and input the second transformed feature data to the post-processing module.

[0142] The post-processing module 222 is used to generate output feature data corresponding to the input feature data based on the element values ​​of each intermediate element included in the second transformed feature data.

[0143] The dot product processing module 221 receives the element values ​​of the elements to be determined transmitted by each first accumulator in the accumulation module; and obtains the weight feature data, which is the feature data obtained after transformation using a transformation matrix; for example, the initial weight data can be transformed using a transformation matrix that matches the size of the initial weight data to obtain the weight feature data.

[0144] The dot product processing module 221 multiplies the element value of the element to be determined with the feature value in the weighted feature data that matches the target position of the element to be determined, to obtain the element value of the intermediate element that matches the target position. Then, based on the element values ​​of each intermediate element, the second transformed feature data can be obtained. The obtained second transformed feature data is input to the post-processing module 222.

[0145] The post-processing module 222 generates output feature data corresponding to the input feature data based on the element values ​​of each intermediate element included in the second transformed feature data. For example, the post-processing module can use a transformation matrix that matches the second transformed feature data to transform the second transformed feature data and generate output feature data.

[0146] For example, if the input feature data is: [1, 1, 1, 1; 1, 1, 1, 1; 1, 1, 1, 1; 1, 1, 1, 1], then according to Figure 6b As shown, the first transformed feature data obtained through the preprocessing module can be: [0, 0, 0, 0; 0, 4, 0, 0; 0, 0, 0, 0; 0, 0, 0, 0]. If the obtained weight feature data is: [16, 12, 14, 16; 12, 9, 3, 12; 4, 3, 1, 4; 16, 12, 4, 16], the first transformed feature data and the weight feature data are processed by the dot product processing module to obtain the second transformed feature data: [0, 0, 0, 0; 0, 36, 0, 0; 0, 0, 0, 0; 0, 0, 0, 0]. The postprocessing module then generates the output feature data [36, 36, 36, 36] corresponding to the input feature data based on the element values ​​of each intermediate element included in the second transformed feature data.

[0147] Here, the dot product processing module can determine the second transformed feature data corresponding to the first transformed feature data based on the element value of the element to be determined and the number of weighted features obtained; the post-processing module generates the output feature data corresponding to the input feature data based on the element value of each intermediate element included in the second transformed feature data, thus realizing a fast convolution operation process.

[0148] In another possible implementation, see Figure 9 As shown, the operation module 22 includes a dot product processing module 221 and a post-processing module 222; the dot product processing module 221 is connected to the accumulation module 202 and the post-processing module 222 respectively.

[0149] The dot product processing module 221 includes: an element number of dot product units 2211; each dot product unit 2211 is connected to a first accumulator 2021.

[0150] The dot productor 2211 is used to receive the element value of the element to be determined output by the first accumulator; and multiply the element value of the element to be determined by the weight value in the weight feature data corresponding to the element to be determined, to obtain the element value of the intermediate element in the second transformed feature data corresponding to the first transformed feature data that matches the element to be determined.

[0151] The post-processing module 222 is used to generate output feature data corresponding to the input feature data based on the element values ​​of each intermediate element included in the received second transformed feature data.

[0152] The dot product processing module 221 includes a number of dot product units 2211, each connected to a first accumulator. Each dot product unit receives the element value of the element to be determined from the connected first accumulator, and acquires weight feature data. It multiplies the element value of the element to be determined with the weight value corresponding to the element in the weight feature data to obtain the element value of the intermediate element matching the element to be determined in the second transformed feature data corresponding to the first transformed feature data. The element values ​​of each intermediate element included in the second transformed feature data are then input to the post-processing module 222. The post-processing module 222 generates output feature data corresponding to the input feature data based on the received element values ​​of each intermediate element included in the second transformed feature data.

[0153] For example, when the pixel parallelism (i.e. the number of elements) is 2, if the input feature data is d = [1, 1, 1, 1; 1, 1, 1, 1; 1, 1, 1, 1; 1, 1, 1, 1], the preprocessing module can determine the element values ​​of two elements to be determined in the first transformed feature data x at one time, such as determining x0 and x1 (0, 0), x2 and x3 (0, 0), etc.

[0154] The dot product processing module 221 includes two dot product units, namely dot product unit 0 and dot product unit 1. If the weighted feature data is w = [16, 12, 14, 16; 12, 9, 3, 12; 4, 3, 1, 4; 16, 12, 4, 16], dot product unit 0 can multiply x0 = 0 with w0 = 16 to obtain m0 = 0 in the second transformed feature data m; dot product unit 1 can multiply x1 = 0 with w1 = 12 to obtain m1 = 0 in the second transformed feature data m. The dot product processing module 221 can determine the element values ​​of two intermediate elements included in the second transformed feature data m at each time. For example, m0 and m1 are (0, 0), m2 and m3 are (0, 0), m4 and m5 are (0, 36), etc. The post-processing module generates output feature data based on the element values ​​of each intermediate element included in the second transformed feature data.

[0155] With a pixel parallelism of 4, if the input feature data is d = [1, 1, 1, 1; 1, 1, 1, 1; 1, 1, 1, 1; 1, 1, 1, 1], the preprocessing module can determine the element values ​​of the four elements to be determined in the first transformed feature data x at one time. For example, it can determine x0, x1, x2 and x3 as (0, 0, 0, 0), x4, x5, x6 and x7 as (0, 4, 0, 0), etc.

[0156] The dot product processing module 221 includes four dot product units: dot product unit 0, dot product unit 1, dot product unit 2, and dot product unit 3. If the weighted feature data is w = [16, 12, 14, 16; 12, 9, 3, 12; 4, 3, 1, 4; 16, 12, 4, 16], dot product unit 0 can multiply x0 = 0 with w0 = 16 to obtain m0 = 0 in the second transformed feature data m; dot product unit 1 can multiply x1 = 0 with w1 = 12 to obtain m1 = 0 in the second transformed feature data m; dot product unit 2 can multiply x2 = 0 with w2 = 14 to obtain m2 = 0 in the second transformed feature data m; and dot product unit 3 can multiply x3 = 0 with w3 = 16 to obtain m3 = 0 in the second transformed feature data m. As can be seen, the dot product processing module can determine the element values ​​of the four intermediate elements in the second transformed feature data m each time, such as (0, 0, 0, 0), (0, 36, 0, 0), (0, 0, 0, 0), (0, 0, 0, 0). The post-processing module then generates the output feature data based on the element values ​​of each intermediate element included in the second transformed feature data.

[0157] Here, the dot product multiplies the received element value of the element to be determined with the weight value corresponding to the element in the weighted feature data, obtaining the element value of the intermediate element matching the element to be determined in the second transformed feature data corresponding to the first transformed feature data. The number of dot product units is equal to the number of elements, indicating that multiple dot product units can obtain the element values ​​of multiple intermediate elements in parallel, improving computational efficiency. Furthermore, the post-processing module can generate output feature data based on the element values ​​of each intermediate element included in the received second transformed feature data, improving the determination efficiency of the output feature data.

[0158] In one possible implementation, see Figure 10 As shown, the dot product processing module 221 further includes: at least one adder 2212; the adder 2212 is connected to a plurality of matched dot product units 2211.

[0159] Adder 2212 is used to receive the element values ​​of intermediate elements output by multiple connected dot productors, merge the element values ​​of multiple intermediate elements to obtain an intermediate merged value, and input the intermediate merged value to the post-processing module.

[0160] The post-processing module 222 is used to generate output feature data corresponding to the input feature data based on the element values ​​of the target element. The element values ​​of the target element include intermediate fusion values ​​and / or the element values ​​of intermediate elements.

[0161] The dot product processing module may further include at least one adder connected to multiple matched dot product units. The connection relationship between the adder and the dot product units can be determined based on the transformation relationship between the output feature data and the second transformation matrix. The transformation relationship between the output feature data r and the second transformation matrix m is as follows:

[0162] r0=m0+m1+m2+m4+m5+m6+m8+m9+m10;

[0163] r1=m1-m2-m3+m5-m6-m7+m9-m10-m11;

[0164] r2=m4+m5+m6-m8-m9-m10-m12-m13-m14;

[0165] r3=m5-m6-m7-m9+m10+m11-m13+m14+m15.

[0166] The output feature data r includes four output elements: r0, r1, r2, and r3.

[0167] Adder 2212 receives the element values ​​of intermediate elements from multiple connected dot productors, fuses these intermediate element values ​​to obtain an intermediate fused value, and inputs this fused value to post-processing module 222. Post-processing module 222 then generates output feature data corresponding to the input feature data based on the element values ​​of the target element. The element values ​​of the target element include the intermediate fused value, or the element values ​​of intermediate elements, or both the intermediate fused value and the element values ​​of intermediate elements.

[0168] See Figure 11 The diagram shows the structure of the dot product processing module. With a pixel parallelism of 2, the circuit of the corresponding dot product processing module is as follows: Figure 11 As shown in 'a', with a pixel parallelism of 4, the circuit of the corresponding dot product processing module is as follows: Figure 11 As shown in b in the figure.

[0169] Combination Figure 11 Example 'a' illustrates the process of the dot product processing module. In 'a', there are two dot product units and one adder. Each time, the two dot product units determine the element values ​​of two intermediate elements included in the second transformed feature data 'm', such as m0 and m1, m2 and m3, etc. The adder receives the element value of intermediate element m0 ​​output by dot product unit 0 and the element value of intermediate element m1 output by dot product unit 1, and fuses m0 and m1 to obtain the intermediate fused value m0+m1. The intermediate fused value obtained by the adder each time can include: m0+m1, m2+m3, m4+m5, m6+m7, m8+m9, m10+m11, m12+m13, m14+m15. Then, the post-processing module generates the output feature data corresponding to the input feature data based on the element value of the target element. The element value of the target element includes the intermediate fused value and / or the element value of the intermediate element.

[0170] With a pixel parallelism of 2, the element values ​​of the target element can include: the element values ​​of the intermediate elements m0 to m15, and / or the intermediate fusion values ​​m0+m1, m2+m3, m4+m5, m6+m7, m8+m9, m10+m11, m12+m13, m14+m15.

[0171] Combination Figure 11 Example b illustrates the process of the dot product processing module. In b, there are 4 dot product units and 2 adders, with the adders being three-input adders. Each of the 4 dot product units determines the element values ​​of the 4 intermediate elements included in the second transformed feature data m, such as m0, m1, m2, and m3, m4, m5, m6, and m7, etc.

[0172] Adder 0 receives the element values ​​of intermediate elements m0 (output from dot product 0), m1 (output from dot product 1), and m2 (output from dot product 2), and merges them to obtain the intermediate merged value m0 + m1 + m2; this intermediate merged value is then input to the post-processing module. Adder 1 receives the element values ​​of intermediate elements m1 (output from dot product 1), m2 (output from dot product 2), and m3 (output from dot product 3), and merges them to obtain the intermediate merged value m1 - m2 - m3; this intermediate merged value is then input to the post-processing module. Each of the two adders yields two intermediate fused values, for example: (m0+m1+m2, m1-m2-m3), (m4+m5+m6, m5-m6-m7), (m8+m9+m10, m9-m10-m11), (m12+m13+m14, m13-m14-m15). The post-processing module then generates output feature data based on the element values ​​of the target element.

[0173] With a pixel parallelism of 4, the element values ​​of the target element can include: intermediate blending values ​​(m0+m1+m2, m1-m2-m3), (m4+m5+m6, m5-m6-m7), (m8+m9+m10, m9-m10-m11), (m12+m13+m14, m13-m14-m15).

[0174] Here, the adder receives the element values ​​of intermediate elements from multiple connected dot productors, and fuses these intermediate element values ​​to obtain an intermediate fused value. This intermediate fused value is then input to the post-processing module, where it serves as the intermediate value required for the post-processing module to determine the output feature data. Consequently, the post-processing module can quickly generate the output feature data based on the element values ​​of the target element, thus improving computational efficiency.

[0175] In one possible implementation, see Figure 10 As shown, the post-processing module 222 includes: a plurality of second accumulators 2221; the number of second accumulators 2221 matches the number of output elements included in the output feature data to be determined; each second accumulator 2221 is matched with one output element.

[0176] The second accumulator 2221 is used to select the desired element value from the element values ​​of the input target element; and determine the output element based on the selected desired element value; wherein each output element generates output feature data.

[0177] For example, if the number of output elements in the output feature data to be determined is 4, then the number of second accumulators is 4. With a pixel parallelism of 2, based on the transformation relationship between the output feature data r and the second transformation matrix m, the second accumulator 0 needs to accumulate m0, m1, m2, m4, m5, m6, m8, m9, and m10 to obtain r0 when determining r0. Therefore, after the dot product processing module outputs m0, m1, m2, m4, m5, m6, m8, m9, and m10, the second accumulator 0 can obtain m0, m1, m2, m4, m5, m6, m8, m9, and m6, and accumulate the obtained element values ​​to obtain r0. Alternatively, when the dot product processing module outputs intermediate fused values ​​m0+m1, m4+m5, and m8+m9, the second accumulator 0 can select the intermediate fused values ​​m0+m1, m4+m5, m8+m9, and intermediate elements m2, m6, and m10. The selected intermediate fused values ​​m0+m1, m4+m5, and m8+m9 are then added to the intermediate elements m2, m6, and m10 to obtain r0. The processes by which the second accumulator 1 obtains r1, the second accumulator 2 obtains r2, and the third accumulator 3 obtains r3 can be referred to the above description and will not be detailed here. After obtaining r0, r1, r2, and r3, the output feature data is obtained.

[0178] With a pixel parallelism of 4, based on the transformation relationship between the output feature data r and the second transformation matrix m, the second accumulator 0 needs to accumulate m0, m1, m2, m4, m5, m6, m8, m9, and m10 to obtain r0 when determining r0. Therefore, the second accumulator 0 can select intermediate fusion values ​​m0+m1+m2, m4+m5+m6, and m8+m9+m10 from the target element, and accumulate these selected intermediate fusion values ​​to obtain the output element r0. Similarly, when determining the output element r1, the second accumulator 1 can select intermediate fusion values ​​m1-m2-m3, m5-m6-m7, and m9-m10-m11, and accumulate these selected intermediate fusion values ​​to obtain the output element r1. Similarly, r2 and r3 can be obtained, thus yielding the output feature data.

[0179] Here, the second accumulator selects the required element value from the element values ​​of the input target element; based on the selected required element value, it determines the output element; wherein, each output element generates output feature data; thus, multiple output elements are obtained in parallel, improving computational efficiency.

[0180] In one possible implementation, the post-processing module 222 further includes: a plurality of data selectors 2222; the number of data selectors 2222 matches the number of second accumulators 2221; each data selector 2222 is connected to a matched second accumulator 2221.

[0181] The data selector 2222 is used to determine the required element value corresponding to the second accumulator from the element values ​​of the input target element; and send the determined required element value to the second accumulator.

[0182] In implementation, a corresponding data selector can be determined for each second accumulator. For example, when there are multiple inputs to the post-processing module, the data selector can select at least one of the multiple inputs as the input of the second accumulator connected to the data selector, so that the second accumulator can accumulate the input data to obtain the output element.

[0183] See Figure 12 The diagram shows the structure of the post-processing module. With a pixel parallelism of 2, the circuit of the corresponding post-processing module is as follows: Figure 12 As shown in 'a'; with a pixel parallelism of 4, the corresponding post-processing module circuit is as follows: Figure 12 As shown in b in the figure.

[0184] Combination Figure 12 Example 'a' illustrates the process of the post-processing module. In example 'a', the input to the post-processing module is 3, meaning that each input to the post-processing module includes the element values ​​of intermediate elements and intermediate fusion values, such as m0, m1, m0+m1, m2, m3, m2+m4, etc. Data selector 0 is connected to second accumulator 0. Second accumulator 0 is used to determine the output element r0. Since r0 = m0+m1+m2+m4+m5+m6+m8+m9+m10, data selector 0 can sequentially select m0+m1, m2, m4+m5, m6, m8+m9, and m10 to input into second accumulator 0. Second accumulator 0 accumulates the received data to obtain the output element r0.

[0185] Furthermore, data selector 1 can sequentially input m1, m2+m3, m5, m6+m7, m9, and m10+m11 into the second accumulator 1. The second accumulator 1 will accumulate the received data to obtain the output element r1. For example, m2+m3 can be inverted and accumulated with m1, then m5 can be accumulated, m6+m7 can be inverted and accumulated, and so on, until r1 is obtained.

[0186] Similarly, the second accumulator 2 can obtain the output element r2, and the second accumulator 3 can obtain the output element r3. Thus, the output feature data is obtained.

[0187] Combination Figure 12 Example b illustrates the process of the post-processing module. In example b, the input to the post-processing module is 2, meaning that each input to the post-processing module includes two intermediate fused values, such as m0+m1+m2 and m1-m2-m3; m4+m5+m6 and m5-m6-m7, etc. Data selector 0 is connected to second accumulator 0. Second accumulator 0 is used to determine the output element r0. Since r0 = m0+m1+m2+m4+m5+m6+m8+m9+m10, data selector 0 can sequentially select m0+m1+m2, m4+m5+m6, and m8+m9+m10 to input into second accumulator 0. Second accumulator 0 accumulates the received data to obtain the output element r0.

[0188] Furthermore, data selector 1 can sequentially input m1-m2-m3, m5-m6-m7, and m9-m10-m11 into the second accumulator 1. The second accumulator 1 accumulates the received data to obtain output element r1. Similarly, the second accumulator 2 can obtain output element r2, and the second accumulator 3 can obtain output element r3. Thus, the output feature data is obtained.

[0189] Here, the data selector determines the required element value corresponding to the second accumulator from the element values ​​of the input target element; and sends the determined required element value to the second accumulator. The second accumulator accumulates the received data to obtain the output element, that is, the output feature data, thus completing the fast convolution operation and improving the data processing efficiency.

[0190] The following is an illustrative description of the process by which the circuit processes multi-channel input feature data.

[0191] If the input feature data d is 128×128×3, and the current window size is 4×4, take multiple local elements (d01, d02, d03) to (d151, d152, d153) of the current window in the input feature data as an example. Here, (d01, d02, d03) are the feature values ​​on the three channels of one feature position, and (d151, d152, d153) are the feature values ​​on the three channels of another feature position. For example, d151 is the feature value on the first channel, d152 is the feature value on the second channel, and d153 is the feature value on the third channel.

[0192] With a pixel parallelism of 2, the buffer caches (d01, d02, d03) to (d151, d152, d153) sequentially according to the number of elements, that is, in the order of (d01, d02, d03, d11, d12, d13), ..., (d141, d142, d143, d151, d152, d153). When the buffer includes (d01, d02, d03) to (d101, d102, d103), (d01, d02, d03), (d11, d12, d13), (d21, d22, d23), (d81, d82, d83), (d91, d92, d93), and (d101, d102, d103) are output to the accumulation module.

[0193] The first accumulator 0 in the accumulation module is used to determine the element x0 to be determined in the first transformed feature data. Therefore, the first accumulator 0 obtains the local elements (d01, d02, d03), (d21, d22, d23), (d81, d82, d83), (d91, d92, d93), and (d101, d102, d103) from (d01, d02, d03), (d11, d12, d13), (d21, d22, d23), (d81, d82, d83), (d91, d92, d93), and (d101, d102, d103) from (d01, d02, d03), (d21, d22, d23), and (d81, d92, d93), (d101, d102, d103). The process involves summing (d01, d02, d03), (d21, d22, d23), (d81, d82, d83), and (d101, d102, d103) to determine the element value corresponding to the element x0 to be determined. This element value is then input into the dot product 0. For example, summing d01, d21, d81, and d101 yields x01 for channel 0; summing d02, d22, d82, and d102 yields x02 for channel 1; summing d03, d23, d83, and d103 yields x03 for channel 2; and finally summing x01, x02, and x03 yields the element x0 to be determined. Similarly, we can obtain each element to be determined in the feature data after the first transformation.

[0194] Furthermore, the computation module can generate output feature data corresponding to the input feature data based on the received element values ​​of the element to be determined.

[0195] Based on the same concept, this disclosure also provides a chip, see [link to relevant documentation]. Figure 13 The diagram shown is an architectural schematic of an AI chip provided in an embodiment of this disclosure. The AI ​​chip includes a memory 1301 and a data processing circuit 1302 as described in the above embodiments.

[0196] The memory stores the input feature data and weight feature data required by the data processing circuit, as well as the output feature data generated by the data processing circuit. The data processing circuit is used to retrieve the input feature data from the memory, generate the corresponding output feature data, and store the output feature data into the memory.

[0197] In this embodiment of the disclosure, the data processing circuit described in the above embodiments is used to process the input feature data, which can efficiently obtain the output feature data, thereby improving the processing efficiency of the AI ​​chip.

[0198] Based on the same concept, this disclosure also provides an electronic device, including: the data processing circuit described in the above embodiments; and / or, the chip described in the above embodiments.

[0199] Based on the same concept, embodiments of this disclosure also provide a data processing method, the execution entity of which is generally a data processing circuit. In some possible implementations, the data processing method can be implemented by a processor calling computer-readable instructions stored in memory.

[0200] See Figure 14 The diagram shows a data processing method, S1401-S1402, wherein:

[0201] S1401, Obtain input feature data.

[0202] S1402, the input feature data is cached based on the set number of elements; and when a number of elements to be determined are found in the first transformed feature data corresponding to the input feature data, the element values ​​of the number of elements to be determined are determined based on multiple target local elements on the determined input feature data; the element values ​​of the elements to be determined are input to the calculation module; so that the calculation module generates the output feature data corresponding to the input feature data based on the received element values ​​of the elements to be determined.

[0203] The number of elements refers to the number of elements that the circuit processes in parallel; multiple target local elements include each local element required when the number of elements to be determined is determined.

[0204] In one possible implementation, the method further includes:

[0205] Multiple local elements within the current window, obtained from the input feature data, are cached sequentially according to a set number of elements; and

[0206] When determining the number of elements to be determined in the first transformed feature data corresponding to the input feature data, in response to the buffer including multiple target local elements, the multiple target local elements are output to the accumulation module;

[0207] From the plurality of target local elements, the local elements required by the first accumulator to determine the element to be determined are obtained; and the obtained local elements are accumulated to obtain the element value corresponding to the element to be determined.

[0208] In one possible implementation, the method further includes: the number of storage spaces included in the buffer is determined based on the number of elements and the conversion relationship information between the input feature data and the first converted feature data; and / or, the storage capacity corresponding to each storage space in the buffer is determined based on the number of channels of the input feature data; wherein the storage capacity is positively correlated with the number of channels.

[0209] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0210] Furthermore, this disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the data processing method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0211] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the data processing method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0212] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0213] This disclosure relates to the field of augmented reality (AR). It involves acquiring image information of target objects in a real-world environment and then using various visual algorithms to detect or identify the relevant features, states, and attributes of these objects, thereby achieving an AR effect that combines virtual and real elements to suit specific applications. For example, target objects may include human features such as faces, limbs, gestures, and movements; objects such as signs and markers; or venues such as sand tables, display areas, or displayed items. Visual algorithms may include visual localization, SLAM, 3D reconstruction, image registration, background segmentation, keypoint extraction and tracking of objects, and pose or depth detection. Specific applications can include interactive scenarios related to real-world scenes or objects, such as guided tours, navigation, explanations, reconstruction, and virtual effect overlay displays, as well as human-related special effects processing, such as makeup enhancement, limb enhancement, special effects displays, and virtual model displays. Convolutional neural networks (CNNs) can be used to detect or identify the relevant features, states, and attributes of target objects. The aforementioned CNNs are network models trained using deep learning frameworks.

[0214] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0215] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0217] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0218] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A data processing circuit, characterized by, The circuit comprises a preprocessing module and an operation module; The preprocessing module is configured to cache input feature data based on a set number of elements, and when a number of to-be-determined elements in first converted feature data corresponding to the input feature data is determined, determine element values of the number of to-be-determined elements based on a plurality of target local elements on the input feature data determined; and input the element values of the to-be-determined elements to the operation module; The number of elements is the number of elements processed in parallel by the circuit; the plurality of target local elements include each local element required when the number of to-be-determined elements is determined; The operation module is configured to generate output feature data corresponding to the input feature data based on the received element values of the to-be-determined elements; The preprocessing module comprises a buffer and an accumulation module; the accumulation module comprises a number of first accumulators; The buffer is configured to cache a plurality of local elements in a current window obtained from the input feature data in sequence according to a set number of elements; and When the number of to-be-determined elements in the first converted feature data corresponding to the input feature data is determined, in response to the buffer comprising a plurality of target local elements, the plurality of target local elements are output to the accumulation module; Each first accumulator in the accumulation module is configured to obtain a local element required when the to-be-determined element is determined from the plurality of target local elements; and accumulate the obtained local element to obtain an element value corresponding to the to-be-determined element; The operation module comprises a dot product processing module and a post-processing module; the dot product processing module is connected to the accumulation module and the post-processing module respectively; The dot product processing module is configured to receive the element values of the to-be-determined elements transmitted by the first accumulators; and determine second converted feature data corresponding to the first converted feature data based on the element values of the to-be-determined elements and obtained weight feature data; and input the second converted feature data to the post-processing module; The post-processing module is configured to generate output feature data corresponding to the input feature data based on element values of each intermediate element included in the second converted feature data.

2. The circuit of claim 1, wherein, The number of storage spaces in the buffer is determined based on the number of elements, and conversion relationship information between the input feature data and the first converted feature data; and / or the storage capacity of each storage space in the buffer is determined based on the number of channels of the input feature data; wherein the storage capacity is positively correlated with the number of channels.

3. The circuit of claim 1 or 2, characterized in that The operation module comprises a dot product processing module and a post-processing module; The dot product processing module is connected to the accumulation module and the post-processing module respectively; The dot product processing module comprises a number of dot products corresponding to the number of elements; each dot product is connected to a first accumulator; The dot product device is configured to receive the element value of the to-be-determined element output by the first accumulator, and multiply the element value of the to-be-determined element by a weight value corresponding to the to-be-determined element in weight feature data, to obtain an element value of a middle element matching the to-be-determined element in second converted feature data corresponding to the first converted feature data. The post-processing module is configured to generate output feature data corresponding to the input feature data based on the element values of the respective middle elements included in the received second converted feature data.

4. The circuit of claim 3, wherein, The dot product processing module further includes at least one adder, and the adder is connected to the matched plurality of dot product devices. The adder is configured to receive the element values of the middle elements output by the connected plurality of dot product devices, fuse the element values of the plurality of middle elements to obtain a middle fusion value, and input the middle fusion value to the post-processing module. The post-processing module is configured to generate output feature data corresponding to the input feature data based on the element value of the target element, wherein the element value of the target element includes the middle fusion value and / or the element value of the middle element.

5. The circuit of claim 4, wherein, The post-processing module includes a plurality of second accumulators, the number of the second accumulators matches the number of output elements included in the output feature data to be determined, and each second accumulator matches an output element. The second accumulator is configured to select a required element value from the input element value of the target element, and determine the output element based on the selected required element value. The respective output elements generate the output feature data.

6. The circuit of claim 5, wherein, The post-processing module further includes a plurality of data selectors, the number of the data selectors matches the number of the second accumulators, and each data selector is connected to a matched second accumulator. The data selector is configured to determine a required element value corresponding to the second accumulator from the input element value of the target element, and send the determined required element value to the second accumulator.

7. A chip, characterized by The chip includes a memory and the data processing circuit according to any one of claims 1-6, wherein the data processing circuit is configured to acquire input feature data from the memory, generate output feature data corresponding to the input feature data, and store the output feature data to the memory.

8. An electronic device, comprising: The chip includes a memory and the data processing circuit according to any one of claims 1-6, wherein the data processing circuit is configured to acquire input feature data from the memory, generate output feature data corresponding to the input feature data, and store the output feature data to the memory. The method is applied to the data processing circuit according to any one of claims 1-6, and the method includes:

9. A data processing method, characterized by, acquiring input feature data; ​ The input feature data is cached based on the set number of elements; and when the number of elements of to-be-determined elements is determined in the first converted feature data corresponding to the input feature data, the element values of the number of elements of to-be-determined elements are determined based on the determined plurality of target local elements on the input feature data; and the element values of the to-be-determined elements are input to the operation module; so that the operation module generates output feature data corresponding to the input feature data based on the received element values of the to-be-determined elements. The number of elements is the number of elements processed in parallel by the circuit; and the plurality of target local elements include each local element required when the number of elements of to-be-determined elements is determined.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is run by the processor to execute the steps of the data processing method in claim 9.

Images