Processing method and device based on processing array, electronic device, and storage medium

Abstract

The present disclosure provides a processing method and device based on a processing array, an electronic device and a storage medium, and belongs to the technical field of computers. The method comprises: obtaining each to-be-processed feature and splitting it into at least one first-type sub-feature and at least one second-type sub-feature; inputting each first-type sub-feature into the processing array for calculation separately to obtain a first calculation result of each to-be-processed feature; inputting the multiple second-type sub-features into the processing array for calculation after being combined to obtain a second calculation result of each to-be-processed feature; and determining a target calculation result of each to-be-processed feature according to the first calculation result and the second calculation result of each to-be-processed feature. According to the embodiment of the present disclosure, the utilization rate of the processing array can be improved, and the calculation time can be shortened, and the calculation efficiency can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a processing method and apparatus based on a processing array, an electronic device, and a storage medium. Background Technology

[0002] Processing arrays can execute various computational tasks in parallel, resulting in high processing efficiency. However, due to hardware cost limitations, processing arrays are typically limited in size. In related technologies, because computational tasks in neural networks are large-scale, it is usually necessary to break down a large computational task into several smaller computational tasks, which are then executed sequentially by the processing array to complete the entire computational task. However, if the task breakdown is not optimized, it can lead to low utilization of the processing array, resulting in longer task processing times and impacting processing efficiency. Summary of the Invention

[0003] This disclosure provides a processing method and apparatus based on a processing array, an electronic device, and a storage medium.

[0004] In a first aspect, this disclosure provides a processing method based on a processing array, the method comprising: acquiring each feature to be processed and splitting it into at least one first type of sub-feature and at least one second type of sub-feature; inputting each first type of sub-feature individually into the processing array for calculation to obtain a first calculation result for each feature to be processed; combining multiple second type of sub-features and inputting them into the processing array for calculation to obtain a second calculation result for each feature to be processed; and determining a target calculation result for each feature to be processed based on the first and second calculation results for each feature to be processed.

[0005] Secondly, this disclosure provides a processing apparatus, comprising: a splitting module for splitting each feature to be processed into at least one first type of sub-feature and at least one second type of sub-feature; a first processing module for individually inputting each first type of sub-feature into the processing array for calculation to obtain a first calculation result for each feature to be processed; a second processing module for combining multiple second type of sub-features and then inputting them into the processing array for calculation to obtain a second calculation result for each feature to be processed; and a determining module for determining a target calculation result for each feature to be processed based on the first and second calculation results for each feature to be processed.

[0006] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described processing array-based processing method.

[0007] Fourthly, this disclosure provides an electronic device comprising: a plurality of processing cores; and an on-chip network configured to interact with data between the plurality of processing cores and external data; wherein one or more of the processing cores store one or more instructions, and the one or more instructions are executed by the one or more processing cores to enable the one or more processing cores to perform the above-described processing array-based processing method.

[0008] Fifthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor / processing core, implements the above-described processing array-based processing method.

[0009] The embodiments provided in this disclosure decompose each feature to be processed into at least one first-type sub-feature and at least one second-type sub-feature. The sum of the computational costs of any two first-type sub-features is greater than a preset computational threshold of the processing array, and the sum of the computational costs of at least two second-type sub-features is less than or equal to the computational threshold of the processing array. Each first-type sub-feature is individually input into the processing array for computation to obtain a first computational result for each feature to be processed. Multiple second-type sub-features are combined and then input into the processing array for computation to obtain a second computational result for each feature to be processed. Based on the first and second computational results for each feature to be processed, a target computational result for each feature to be processed is determined. First, the reasonable decomposition of the features to be processed provides a foundation for subsequent feature combination. Second, by combining multiple second-type sub-features before inputting them into the processing array for computation, it is equivalent to performing the computation of multiple second-type sub-features in one processing step. Compared to directly calculating each second-type sub-feature separately, this effectively improves the utilization rate of the processing array and shortens the computation time, thus improving computational efficiency.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0012] Figure 1 A flowchart illustrating a processing method based on a processing array, provided as an embodiment of this disclosure;

[0013] Figure 2 A schematic diagram of a processing array provided in an embodiment of this disclosure;

[0014] Figure 3 A schematic diagram illustrating a processing method provided in an embodiment of this disclosure;

[0015] Figure 4 A schematic diagram illustrating a processing method provided in an embodiment of this disclosure;

[0016] Figure 5 A schematic diagram illustrating a processing method provided in an embodiment of this disclosure;

[0017] Figure 6 A schematic diagram illustrating a processing method provided in an embodiment of this disclosure;

[0018] Figure 7 A flowchart illustrating a processing method based on a processing array, provided as an embodiment of this disclosure;

[0019] Figure 8 A schematic diagram of feature splitting provided in an embodiment of this disclosure;

[0020] Figure 9 A schematic diagram of feature splitting provided in an embodiment of this disclosure;

[0021] Figure 10 A schematic diagram of feature splitting provided in an embodiment of this disclosure;

[0022] Figure 11 A block diagram of a processing apparatus provided in an embodiment of this disclosure;

[0023] Figure 12 A block diagram of an electronic device provided in an embodiment of this disclosure;

[0024] Figure 13 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0027] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0029] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0030] Convolution is one of the important computational methods in fields such as neural networks. In related technologies, when the feature size is large and the processing array size is small, the large feature is usually split into several smaller features, and then the processing array performs the computation. However, this approach can lead to long processing times, thus affecting the processing efficiency of the task.

[0031] According to the processing array-based processing method of this disclosure, each feature to be processed is divided into at least one first-type sub-feature and at least one second-type sub-feature; wherein the sum of the computational amounts of any two first-type sub-features is greater than a preset computational threshold of the processing array, and the sum of the computational amounts of at least two second-type sub-features is less than or equal to the computational threshold of the processing array; each first-type sub-feature is individually input into the processing array for calculation to obtain a first calculation result for each feature to be processed; multiple second-type sub-features are combined and then input into the processing array for calculation to obtain a second calculation result for each feature to be processed; based on the first and second calculation results for each feature to be processed, a target calculation result for each feature to be processed is determined. First, by reasonably dividing the features to be processed, a foundation is provided for subsequent feature combination; second, by combining multiple second-type sub-features before inputting them into the processing array for calculation, it is equivalent to performing the calculation of multiple second-type sub-features in one processing step, which can effectively improve the utilization rate of the processing array and shorten the calculation time compared to directly calculating each second-type sub-feature separately, thereby improving the calculation efficiency.

[0032] The processing method based on a processing array according to embodiments of this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0033] Figure 1 A flowchart illustrating a processing method based on a processing array, provided as an embodiment of this disclosure. (Refer to...) Figure 1 The processing method includes:

[0034] In step S11, each feature to be processed is obtained and split into at least one first type of sub-feature and at least one second type of sub-feature.

[0035] In step S12, each first-class sub-feature is individually input into the processing array for calculation to obtain the first calculation result of each feature to be processed.

[0036] In step S13, multiple second-type sub-features are combined and then input into the processing array for calculation to obtain the second calculation result of each feature to be processed.

[0037] In step S14, the target calculation result of each feature to be processed is determined based on the first calculation result and the second calculation result of each feature to be processed.

[0038] In some alternative implementations, the features to be processed include features obtained from data such as images, text, and audio, which can be represented in tensor form.

[0039] In some alternative implementations, the processing array is an array of multiple processing elements (PEs).

[0040] In this embodiment, a processing array is mainly used to perform convolution operations on features. Since convolution operations include a large number of addition and multiplication operations, and features are usually represented in tensor form, the processing array supports matrix multiplication, matrix addition, and other functions.

[0041] For example, a processing unit may consist of a multiply-accumulator (MAC) unit and memory (including registers or scratchpads, etc.). A PE array composed of multiple processing units can be used to process matrix multiplication and addition in parallel to implement convolution operations.

[0042] Figure 2 This is a schematic diagram of a processing array provided in an embodiment of this disclosure. (Refer to...) Figure 2 The processing array consists of 36 processing units (PEs) arranged in a 6x6 array.

[0043] For the feature to be processed (In, Out), if the input channel size In = 6 and the output channel size Out = 2, and the row direction of the processing array is used as the input channel direction and the column direction of the processing array is used as the output channel direction, then only the first two columns of the processing array (any other two columns are also acceptable) need to be used for calculation to obtain the convolution result of the feature to be processed. The weight information of the convolution kernel required during the calculation process can be obtained from the preset storage space (including on-chip space (e.g., the memory of the processing unit, on-chip shared memory, etc.) and / or off-chip space).

[0044] Furthermore, if In = 7 and Out = 2, then in related technologies, the feature to be processed needs to be split into two smaller features of size (6,2) and (1,2), and calculated separately. Specifically, this includes: first, for the smaller feature (6,2), it is input into the processing array, occupying the first two columns of the array for calculation, to obtain the first convolution result; second, for the smaller feature (1,2), it is also input into the processing array, occupying any row of the first two columns for calculation, to obtain the second convolution result; finally, the two convolution results are superimposed to obtain the final calculation result. In this process, since the processing array performs two calculations, if each calculation process takes 'a', the total time is 2a. When there are N features to be processed, calculating the convolution of these N features takes 2a*N time.

[0045] In the above processing, especially in the process of processing array to calculate small feature (1,2), only 2 processing units are required, and the remaining 34 processing units are idle. Therefore, on the one hand, the processing array is not fully utilized and the utilization rate of processing resources is low; on the other hand, the processing time is long and the processing efficiency is relatively low.

[0046] In view of this, in the embodiments of this disclosure, when performing convolution operations, multiple small features with smaller dimensions are simultaneously input into the processing array for calculation, so as to increase the number of processing units in the calculation state during a single operation, thereby improving the utilization rate of processing resources. At the same time, this parallel processing method can also effectively shorten the processing time and improve the processing efficiency.

[0047] In some optional implementations, when the computational cost of the feature to be processed exceeds the computational threshold of the processing array, it is not possible to directly use the processing array to perform convolution operations on the feature to be processed. Therefore, in step S11, the feature to be processed is split to obtain a first type of sub-feature and a second type of sub-feature with relatively smaller sizes. The number of features to be processed can be one or more, and the sizes of each feature to be processed can be the same or different. For the second type of sub-feature, it can be a sub-feature with the same computational cost (e.g., the size of the second type of sub-feature is (2,2)), or it can include multiple sub-features with different computational costs (e.g., at least some of the second type of sub-features have a size of (2,2), and the remaining second type of sub-features have a size of (1,2)).

[0048] In some optional implementations, the computational cost is determined based on the feature's channel size, and the computation threshold is determined based on the size of the processing array; where the channel size includes the input channel size and the output channel size, and the processing array size includes the input size and the output size. Therefore, the splitting of the feature to be processed includes at least splitting from the input channel and splitting from the output channel.

[0049] In some optional implementations, each feature to be processed is split into at least one first-class sub-feature and at least one second-class sub-feature, including: based on the input channel, the input channel size of the feature to be processed is larger than the input size of the processing array.

[0050] For example, the input channel size of the feature to be processed is larger than the input size of the processing array. Therefore, the feature to be processed is split in the input channel, and the resulting first type of sub-features and second type of sub-features have the same output channel size.

[0051] In some optional implementations, each feature to be processed is split into at least one first-class sub-feature and at least one second-class sub-feature, including: based on the output channel, the output channel size of the feature to be processed is larger than the output size of the processing array.

[0052] For example, the output channel size of the feature to be processed is larger than the output size of the processing array. Therefore, the feature to be processed is split in the output channel, and the resulting first type of sub-features and second type of sub-features have the same input channel size.

[0053] In summary, by splitting the features to be processed, one or more first-class sub-features and one or more second-class sub-features can be obtained. The first-class sub-features are larger in size, making it easier for the processing array to perform direct calculations. The second-class sub-features are smaller in size, allowing the processing array to calculate multiple second-class sub-features simultaneously, thereby shortening the computation time and improving the utilization rate of the processing array.

[0054] It should be noted that the multiple features to be processed can be features of the same size or features of different sizes. When the multiple features to be processed have the same size, the processing method of this disclosure is suitable for scenarios of grouped convolution. When the multiple features to be processed have different sizes, the processing method of this disclosure is suitable for processing multiple convolution operations in parallel.

[0055] In some optional implementations, the sum of the computational costs of any two first-class sub-features is greater than the preset computational threshold of the processing array, and the sum of the computational costs of at least two second-class sub-features is less than or equal to the computational threshold of the processing array.

[0056] In some optional implementations, the sum of the computational amounts of any two first-class sub-features is greater than the preset computational threshold of the processing array, including: the sum of the input channel sizes of any two first-class sub-features is greater than the input size of the processing array, and the sum of the output channel sizes of the two first-class sub-features is less than or equal to the output size of the processing array; or, the sum of the output channel sizes of any two first-class sub-features is greater than the output size of the processing array, and the sum of the input channel sizes of the two first-class sub-features is less than or equal to the input size of the processing array.

[0057] In some optional implementations, the sum of the computational costs of at least two second-type sub-features is less than or equal to the computational threshold of the processing array, including: the sum of the input channel sizes of at least two second-type sub-features is less than or equal to the input size of the processing array, and the sum of the output channel sizes of the two second-type sub-features is less than or equal to the output size of the processing array; or, the sum of the output channel sizes of at least two second-type sub-features is less than or equal to the output size of the processing array, and the sum of the input channel sizes of the two second-type sub-features is less than or equal to the input size of the processing array.

[0058] For example, when the input channel size of the feature to be processed is greater than the input size of the processing array and the output channel size is less than the output size of the processing array, the feature to be processed is split from the input channel dimension, and the input channel size of the first type of sub-feature is equal to the input size of the processing array, and the sum of the input channel sizes of at least two second type of sub-features is less than or equal to the input size of the processing array.

[0059] For example, the processing array has a size of 6*6, and there are 3 features to be processed, each with a size of (7,2). For any feature to be processed, it is split into a first-class sub-feature (6,2) and a second-class sub-feature (1,2). For the above 3 features to be processed, a total of 3 first-class sub-features (6,2) and 3 second-class sub-features (1,2) are obtained.

[0060] For example, if the processing array is 6*6 and there are 3 features to be processed, each with a size of (9,2), then for any one of these features, it can be split into one first-class sub-feature (6,2) and two second-class sub-features with sizes of (2,2) and (1,2), respectively. For the 3 features, this results in 3 first-class sub-features (6,2), 3 second-class sub-features (2,2), and 3 second-class sub-features (1,2). Alternatively, the feature (9,2) can also be split into one first-class sub-feature (6,2) and one second-class sub-feature (3,2). In this case, the 3 features (9,2) will yield 3 first-class sub-features (6,2) and 3 second-class sub-features (3,2).

[0061] For example, the processing array has a size of 6*6, and there are three features to be processed with sizes (7,2), (8,2), and (9,2). Feature (7,2) is split into one first-class sub-feature (6,2) and one second-class sub-feature (1,2). Feature (8,2) is split into one first-class sub-feature (6,2) and one second-class sub-feature (2,2). Feature (9,2) is split into one first-class sub-feature (6,2) and one second-class sub-feature (3,2). For these three features, a total of three first-class sub-features (6,2), one second-class sub-feature (1,2), one second-class sub-feature (2,2), and one second-class sub-feature (3,2) are obtained.

[0062] For example, when the input channel size of the feature to be processed is smaller than the input size of the processing array and the output channel size is larger than the output size of the processing array, the feature to be processed is split from the output channel dimension. The output channel size of the first type of sub-feature is equal to the output size of the processing array, and the sum of the output channel sizes of any two second type of sub-features is less than or equal to the output size of the processing array.

[0063] For example, if the processing array is 6*6 and there are 3 features to be processed, each with a size of (2,7), then for any one feature, it is split into a first-class sub-feature (2,6) and a second-class sub-feature (2,1). For the 3 features to be processed, a total of 3 first-class sub-features (2,6) and 3 second-class sub-features (2,1) are obtained. Other cases are similar and will not be described in detail here.

[0064] It should be noted that since the sum of the computational costs of any two first-class sub-features is greater than the computational threshold of the processing array, only one first-class sub-feature can be input during a single processing cycle of the processing array. For second-class sub-features, when processing them, multiple second-class sub-features that can be combined can be determined first, and then these multiple second-class sub-features can be simultaneously input into the processing array for computation. This allows multiple second-class sub-features to undergo convolution operations simultaneously in a single processing cycle, thereby improving the utilization rate of the processing array, reducing processing time, and improving processing efficiency.

[0065] In some optional implementations, in step S12, each first-type sub-feature is individually input into the processing array for calculation to obtain the first calculation result of each feature to be processed. Therefore, if the processing time required for one processing cycle of the processing array is 'a', then the calculation time required for each first-type sub-feature is also 'a'.

[0066] In some optional implementations, in step S13, multiple second-type sub-features are combined and then input into the processing array for calculation to obtain the second calculation result of each feature to be processed. Here, the combined multiple second-type sub-features refer to those whose computational complexity is less than or equal to the computational threshold of the processing array; that is, the sum of the input channel sizes of the combined multiple second-type sub-features is less than or equal to the input size of the processing array, and the sum of the output channel sizes is also less than or equal to the output size of the processing array.

[0067] In some optional implementations, step S13 includes: combining multiple second-type sub-features whose sum of computational amounts is less than or equal to a computational threshold to obtain at least one combined feature; and inputting each combined feature individually into a processing array for computation to obtain a second computational result for each feature to be processed.

[0068] Therefore, in the process of calculating the second type of sub-features, instead of independently inputting each second type of sub-feature into the processing array for calculation each time, multiple second type of sub-features whose sum of computational costs is less than or equal to the computational threshold of the processing array are simultaneously input into the processing array for parallel computation. Compared to independently calculating each second type of sub-feature each time, in the processing method of this embodiment, the number of computing units occupied by the processing array in each calculation process is increased, thereby reducing the waste of processing array resources, improving the utilization rate of processing resources, and also reducing processing time and improving processing efficiency.

[0069] It should be noted that, assuming the processing time required for one cycle of the processing array is *a*, if two second-type sub-features are simultaneously input into the processing array for calculation (the sum of the computational costs of these two second-type sub-features is less than a computational threshold), then in this process, the number of processing units occupied in the processing array corresponds to the two second-type sub-features, and only *a* time is required to obtain the second calculation result corresponding to these two second-type sub-features. Conversely, in related technologies, the two second-type sub-features need to be input into the processing array separately for two calculations to obtain the corresponding second calculation result. In each calculation, the number of processing units occupied in the processing array corresponds to one second-type sub-feature, and the computation time required is 2*a*. Clearly, the processing method of this embodiment not only improves the utilization rate of the processing array but also shortens the processing time and improves the task processing effect.

[0070] In some optional implementations, the processing array includes multiple processing units arranged in an array. Each processing unit has multiplication and addition operation functions. There is a correspondence between the first type of sub-features and the second type of sub-features obtained by dividing the same feature to be processed. Correspondingly, multiple second type of sub-features are combined and then input into the processing array for calculation to obtain the second calculation result of each feature to be processed. This includes: inputting each second type of sub-feature into the target row of the processing array for calculation, wherein the target row is the row or column in the processing array where the processing unit that calculates the first type of sub-feature that has a correspondence with the second type of sub-feature is located.

[0071] The purpose of this setup is twofold: firstly, the first and second types of sub-features can share the weight information of the convolutional kernel or the original input information based on the processing units of the target row; secondly, after obtaining the second calculation result, the corresponding first calculation result can be conveniently obtained from the current array unit or a nearby array unit (i.e., the array unit in the target row), so that the first and second calculation results can be superimposed to obtain the target calculation result of the feature to be processed. Here, the corresponding first and second calculation results refer to the first and second types of sub-features corresponding to them being obtained by splitting the same feature to be processed.

[0072] It should be noted that in some optional implementations, multiple second-type sub-features can be directly combined and input into the processing array for calculation to obtain the second calculation results of each feature to be processed, without needing to consider whether they are input into the target row. Correspondingly, when determining the target calculation result, it is necessary to retrieve the first calculation result corresponding to each second calculation result from a preset storage space, and then superimpose the second calculation result with the corresponding first calculation result to obtain the target calculation result. Compared to the method of calculating the second calculation result using the target row, the current implementation requires reading data from the storage space, so the time consumption may be relatively long. However, when inputting the second-type sub-features into the processing array, it is not necessary to consider whether they are input into the target row, thus the logic is simpler. In practical applications, any implementation method can be selected according to requirements, and this disclosure embodiment does not impose any restrictions on this.

[0073] In some optional implementations, the features to be processed are grouped convolutional features, where the channel sizes of the first type of sub-features of each grouped convolutional feature are consistent, and the channel sizes of the second type of sub-features of each grouped convolutional feature are consistent.

[0074] In some optional implementations, for grouped convolution, the target calculation result of each feature to be processed is determined based on the first and second calculation results of each feature to be processed. This includes: for each feature to be processed, superimposing the first and second calculation results of the feature to be processed to obtain the intra-group convolution result of each feature to be processed; and concatenating multiple intra-group convolution results to obtain the grouped convolution result.

[0075] In some optional implementations, for scenarios that process multiple convolution operations in parallel, the target calculation result of each feature to be processed is determined based on the first and second calculation results of each feature to be processed, including: superimposing the first and second calculation results that have a corresponding relationship to obtain the convolution result of each feature to be processed.

[0076] The following is combined Figure 3-6 The processing method of the embodiments of this disclosure will be described in detail.

[0077] Figure 3 This is a schematic diagram illustrating a processing method provided in an embodiment of this disclosure. (Refer to...) Figure 3 The processing array has a size of 6*6, and grouped convolution is performed based on this array. The size of the grouped convolution feature T is (21, 6), and the number of groups is 3. Based on the parameters of the grouped convolution, the number of features to be processed is determined to be 3: the first feature T1, the second feature T2, and the third feature T3 (i.e., feature T is grouped to obtain the above three features), and each feature has a size of (7, 2). Since the input channel size is larger than the input size of the processing array, each feature to be processed needs to be split.

[0078] First, split T1 to obtain a first-class sub-feature t11 and a second-class sub-feature t12, with t11 having a size of (6,2) and t12 having a size of (1,2). Then, split T2 to obtain a first-class sub-feature t21 and a second-class sub-feature t22, with t21 having a size of (6,2) and t22 having a size of (1,2). Finally, split T3 to obtain a first-class sub-feature t31 and a second-class sub-feature t32, with t31 having a size of (6,2) and t32 having a size of (1,2). Next, t11 is input into the first and second columns of the processing array for calculation to obtain the first calculation result s11 corresponding to t11. Each row of the processing array corresponds to an input channel of the feature, and each column of the processing array corresponds to an output channel of the feature. t21 is input into the third and fourth columns of the processing array for calculation to obtain the first calculation result s21 corresponding to t21. t31 is input into the fifth and sixth columns of the processing array for calculation to obtain the first calculation result s31 corresponding to t31. Then, t12 is input into the first row of the first and second columns of the processing array, t22 is input into the second row of the third and fourth columns of the processing array, and t32 is input into the third row of the fifth and sixth columns of the processing array. Through parallel operations of the processing array on the above-mentioned second type of sub-features, the second calculation result s12 corresponding to t12, the second calculation result s22 corresponding to t22, and the second calculation result s32 corresponding to t32 are obtained. Finally, the first calculation result s11 and the second calculation result s12 are superimposed to obtain the intra-group convolution result S1, the first calculation result s21 and the second calculation result s22 are superimposed to obtain the intra-group convolution result S2, and the first calculation result s31 and the second calculation result s32 are superimposed to obtain the intra-group convolution result S3. Finally, the above three combined convolution results are concatenated to obtain the grouped convolution result S = (S1, S2, S3).

[0079] like Figure 3 As shown, Figure 3 (1) The mapping relationship between t11 and the processing array is shown (the shaded part represents the processing unit used to process t11). Figure 3 (2) shows the mapping relationship between t21 and the processing array. Figure 3 (3) shows the mapping relationship between t31 and the processing array. Figure 3 (4) shows the mapping relationship between t12, t22, and t32 and the processing array. If the processing time of each processing step of the processing array is 'a', then the total processing time of the above process is approximately 4a.

[0080] It should be noted that in related technologies, when calculating t12, t22, and t32, the calculation is usually similar to the mapping method for the first type of sub-features, with each second type of sub-feature being calculated separately. Therefore, the total time consumption for the above-mentioned grouped convolution is close to 6a. This further illustrates that the processing method of the present disclosure embodiment can shorten the processing time and improve the utilization rate of the processing array and the task processing efficiency.

[0081] Figure 4 This is a schematic diagram illustrating a processing method provided in an embodiment of this disclosure. (Refer to...) Figure 4 The processing array has a size of 6*6, and grouped convolution is performed based on this array. The size of the grouped convolution feature T is (6, 21), and the number of groups is 3. Based on the parameters of the grouped convolution, the number of features to be processed is determined to be 3: the first feature T1, the second feature T2, and the third feature T3 (i.e., feature T is grouped to obtain the above three features), and each feature has a size of (2, 7). Since the output channel size is larger than the output size of the processing array, each feature needs to be split.

[0082] First, split T1 to obtain a first-class sub-feature t11 and a second-class sub-feature t12, with t11 having a size of (2,6) and t12 having a size of (2,1). Then, split T2 to obtain a first-class sub-feature t21 and a second-class sub-feature t22, with t21 having a size of (2,6) and t22 having a size of (2,1). Finally, split T3 to obtain a first-class sub-feature t31 and a second-class sub-feature t32, with t31 having a size of (2,6) and t32 having a size of (2,1). Next, t11 is input into the first and second rows of the processing array for calculation to obtain the first calculation result s11 corresponding to t11. Each row of the processing array corresponds to an input channel of the feature, and each column of the processing array corresponds to an output channel of the feature. t21 is input into the third and fourth rows of the processing array for calculation to obtain the first calculation result s21 corresponding to t21. t31 is input into the fifth and sixth rows of the processing array for calculation to obtain the first calculation result s31 corresponding to t31. Then, t12 is input into the first column of the first and second rows of the processing array, t22 is input into the second column of the third and fourth rows of the processing array, and t32 is input into the third column of the fifth and sixth rows of the processing array. Through parallel operations of the processing array on the above-mentioned second type of sub-features, the second calculation result s12 corresponding to t12, the second calculation result s22 corresponding to t22, and the second calculation result s32 corresponding to t32 are obtained. Finally, the first calculation result s11 and the second calculation result s12 are superimposed to obtain the intra-group convolution result S1, the first calculation result s21 and the second calculation result s22 are superimposed to obtain the intra-group convolution result S2, and the first calculation result s31 and the second calculation result s32 are superimposed to obtain the intra-group convolution result S3. Finally, the above three combined convolution results are concatenated to obtain the grouped convolution result S = (S1, S2, S3).

[0083] like Figure 4 As shown, Figure 4 (1) shows the mapping relationship between t11 and the processing array. Figure 4 (2) shows the mapping relationship between t21 and the processing array. Figure 4 (3) shows the mapping relationship between t31 and the processing array. Figure 4 (4) shows the mapping relationship between t12, t22, and t32 and the processing array. If the processing time of each processing step of the processing array is 'a', then the total processing time of the above process is about 4a. Compared with related technologies, this can shorten the processing time and improve the utilization rate of the processing array and the efficiency of task processing.

[0084] Figure 5 This is a schematic diagram illustrating a processing method provided in an embodiment of this disclosure. (Refer to...) Figure 5The processing array has a size of 6*6, and grouped convolution is performed based on this array. The size of the grouped convolution feature T is (27, 6), and the number of groups is 3. Based on the parameters of the grouped convolution, the number of features to be processed is determined to be 3: the first feature T1, the second feature T2, and the third feature T3 (i.e., feature T is grouped to obtain the above three features), and each feature has a size of (9, 2). Since the input channel size is larger than the input size of the processing array, each feature needs to be split.

[0085] First, split T1 to obtain a first-class sub-feature t11 and a second-class sub-feature t12, with t11 having a size of (6,2) and t12 having a size of (3,2). Then, split T2 to obtain a first-class sub-feature t21 and a second-class sub-feature t22, with t21 having a size of (6,2) and t22 having a size of (3,2). Finally, split T3 to obtain a first-class sub-feature t31 and a second-class sub-feature t32, with t31 having a size of (6,2) and t32 having a size of (3,2). Next, t11 is input into the first and second columns of the processing array for calculation to obtain the first calculation result s11 corresponding to t11. Each row of the processing array corresponds to an input channel of the feature, and each column of the processing array corresponds to an output channel of the feature. t21 is input into the third and fourth columns of the processing array for calculation to obtain the first calculation result s21 corresponding to t21. t31 is input into the fifth and sixth columns of the processing array for calculation to obtain the first calculation result s31 corresponding to t31. Then, input t12 into the first three rows of the first and second columns of the processing array, and input t22 into the last three rows of the third and fourth columns of the processing array. Through parallel operations on the two second-type sub-features by the processing array, the second calculation result s12 corresponding to t12 and the second calculation result s22 corresponding to t22 are obtained. Then, input t32 into the last three rows of the fifth and sixth columns of the processing array (any three rows are acceptable). Through operations on the second-type sub-feature by the processing array, the second calculation result s32 corresponding to t32 is obtained. Finally, the first calculation result s11 and the second calculation result s12 are superimposed to obtain the intra-group convolution result S1. The first calculation result s21 and the second calculation result s22 are superimposed to obtain the intra-group convolution result S2. The first calculation result s31 and the second calculation result s32 are superimposed to obtain the intra-group convolution result S3. Finally, the above three combined convolution results are concatenated to obtain the grouped convolution result S = (S1, S2, S3).

[0086] like Figure 5 As shown, Figure 5 (1) shows the mapping relationship between t11 and the processing array. Figure 5(2) shows the mapping relationship between t21 and the processing array. Figure 5 (3) shows the mapping relationship between t31 and the processing array. Figure 5 (4) shows the mapping relationship between t12, t22 and the processing array. Figure 5 (5) shows the mapping relationship between t32 and the processing array. If the processing time of each processing step of the processing array is a, then the total processing time of the above process is about 5a. Compared with related technologies, the processing time has been reduced and the utilization rate of the processing array has been improved.

[0087] Figure 6 This is a schematic diagram illustrating a processing method provided in an embodiment of this disclosure. (Refer to...) Figure 6 The processing array has a size of 6*6, and convolution operations are performed on three features to be processed based on this array. These features include a first feature T1, a second feature T2, and a third feature T3, with sizes of (7,2), (8,2), and (9,2), respectively. Since the input channel size is larger than the input size of the processing array, each feature needs to be split.

[0088] First, split T1 to obtain a first-class sub-feature t11 and a second-class sub-feature t12, with t11 having a size of (6,2) and t12 having a size of (1,2). Then, split T2 to obtain a first-class sub-feature t21 and a second-class sub-feature t22, with t21 having a size of (6,2) and t22 having a size of (2,2). Finally, split T3 to obtain a first-class sub-feature t31 and a second-class sub-feature t32, with t31 having a size of (6,2) and t32 having a size of (3,2). Next, t11 is input into the first and second columns of the processing array for calculation to obtain the first calculation result s11 corresponding to t11. Each row of the processing array corresponds to an input channel of the feature, and each column of the processing array corresponds to an output channel of the feature. t21 is input into the third and fourth columns of the processing array for calculation to obtain the first calculation result s21 corresponding to t21. t31 is input into the fifth and sixth columns of the processing array for calculation to obtain the first calculation result s31 corresponding to t31. Then, t12 is input into the first row of the first and second columns of the processing array, t22 is input into the second and third rows of the third and fourth columns of the processing array, and t32 is input into the last three rows of the fifth and sixth columns of the processing array. Through parallel operation of the processing array on the above multiple second-type sub-features, the second calculation result s12 corresponding to t12, the second calculation result s22 corresponding to t22, and the second calculation result s32 corresponding to t32 are obtained. Finally, the first calculation result s11 and the second calculation result s12 are superimposed to obtain the target convolution result S1 corresponding to the feature to be processed T1. The first calculation result s21 and the second calculation result s22 are superimposed to obtain the target convolution result S2 corresponding to the feature to be processed T2. The first calculation result s31 and the second calculation result s32 are superimposed to obtain the target convolution result S3 corresponding to the feature to be processed T3.

[0089] like Figure 6 As shown, Figure 6 (1) shows the mapping relationship between t11 and the processing array. Figure 6 (2) shows the mapping relationship between t21 and the processing array. Figure 6 (3) shows the mapping relationship between t31 and the processing array. Figure 6 (4) shows the mapping relationship between t12, t22, and t32 and the processing array. If the processing time of each processing step of the processing array is 'a', then the total processing time of the above process is approximately 4a.

[0090] It should be noted that in related technologies, when calculating the convolution results of the three features to be processed with sizes (7,2), (8,2), and (9,2) respectively, for T1(7,2), it is necessary to split it into two smaller features and calculate them separately, which takes about 2 years. The process is similar for T1(8,2) and (9,2). Therefore, the total time taken is about 6 years. Therefore, compared with related technologies, the processing method of this embodiment reduces the processing time and improves the utilization rate of the processing array.

[0091] In summary, the present embodiment does not change the method of convolution operation, but optimizes the configuration of processing resources by changing the feature splitting and the mapping relationship between features and processing array, thereby effectively improving the resource utilization of processing array, shortening processing time, and thus improving task processing efficiency.

[0092] It should be noted that in some optional implementations, if the remainder obtained by modulo the size of the feature to be processed and the size of the processing array is close to the size of the processing array, it may not be possible to split the feature to be processed into the first type of sub-features and the second type of sub-features that satisfy the embodiments of this disclosure. In this case, it may not be possible to further reduce the processing time. For example, if the size of the feature to be processed is (11,2), the quantity is 3, and the processing array is a 6*6 array, it is impossible to split it into the first type of sub-features and the second type of sub-features of this disclosure, thus making it impossible to reduce the processing time.

[0093] Figure 7 A flowchart illustrating a processing method based on a processing array, provided as an embodiment of this disclosure. (Refer to...) Figure 7 The processing method includes:

[0094] In step S71, the acquired original features are split into at least one first split feature and at least one second split feature, and the second split feature is used as the feature to be processed.

[0095] In this embodiment, the computational cost of the first splitting feature is less than or equal to the computational threshold of the processing array, while the computational cost of the second splitting feature is greater than the computational threshold of the processing array. In other words, the first splitting feature can be directly input into the processing array for computation without further splitting, while the second splitting feature still cannot be directly input into the processing array for computation. However, it can be further split using the processing method of this embodiment to enable the processing array to perform computation.

[0096] In some optional implementations, when the input channel size of the original feature is larger than the input size of the processing array and the output channel size is also larger than the output size of the processing array, at least one first split feature with the same size as the processing array can be extracted from the original feature. In the remaining feature parts, the input channel size of some feature regions is smaller than or equal to the input size of the processing array, and the output channel size of some feature regions is smaller than or equal to the output size of the processing array. By further splitting the remaining feature parts from the input channel or the output channel, at least one second split feature can be obtained, or one or more first split features and at least one second split feature can be obtained.

[0097] Figure 8 This is a schematic diagram illustrating feature segmentation as provided in an embodiment of this disclosure. (Refer to...) Figure 8 The original feature has a size of (7,7), and the processing array has a size of (6,6). The original feature is then split to obtain the first split feature (a), with a size of (6,6). The remaining feature is then split to obtain the second split feature (b), with a size of (1,6). The remaining part is the second split feature (c), with a size of (7,1). The first split feature can be directly calculated by the processing array. The second split feature can be further split to obtain corresponding first-class and second-class sub-features, and then subsequent convolution operations are performed.

[0098] Figure 9 This is a schematic diagram illustrating feature segmentation as provided in an embodiment of this disclosure. (Refer to...) Figure 9 The original feature has a size of (7,7), and the processing array has a size of (6,6). The original feature is then split to obtain the first split feature (a), with a size of (6,6). The remaining feature is then split to obtain the second split feature (d), with a size of (6,1). The remaining part is the second split feature (e), with a size of (1,7). The first split feature can be directly calculated by the processing array. The second split feature can be further split to obtain corresponding first-class and second-class sub-features, and then subsequent convolution operations are performed.

[0099] Figure 10 This is a schematic diagram illustrating feature segmentation as provided in an embodiment of this disclosure. (Refer to...) Figure 10The original feature has a size of (14,7), and the processing array has a size of (6,6). The original feature is then split into two first-level features, (f) and (g), both with a size of (6,6). The remaining feature is then split into two second-level features, (h) and (i), both with a size of (6,1). The remaining part is the second-level feature (j), with a size of (2,7). The first-level features can be directly calculated by the processing array. The second-level features can be further split to obtain corresponding first-class and second-class sub-features, and then subsequent convolution operations are performed.

[0100] In summary, the purpose of step S71 is to first split the original feature when both the input channel size and the output channel size are larger than the size of the processing array, to obtain several first split features that can be directly calculated by the processing array, and several second split features that cannot be directly calculated by the processing array but can be calculated by further splitting using the processing method of this embodiment.

[0101] In step S72, each feature to be processed is obtained and split into at least one first type of sub-feature and at least one second type of sub-feature.

[0102] In step S73, each first-class sub-feature is individually input into the processing array for calculation to obtain the first calculation result of each feature to be processed.

[0103] In step S74, multiple second-type sub-features are combined and then input into the processing array for calculation to obtain the second calculation result of each feature to be processed.

[0104] In step S75, the target calculation result of each feature to be processed is determined based on the first calculation result and the second calculation result of each feature to be processed.

[0105] In step S76, the calculation result of the original feature is obtained based on the target calculation result and the calculation result of the first split feature.

[0106] The calculation result of the first splitting feature is obtained by directly inputting the first splitting feature into the processing array for calculation.

[0107] In some optional implementations, the calculation results of the original features can be obtained by superimposing and / or splicing the calculation results of the target calculation results and the calculation results of the first split features, based on information such as the relative positional relationship between each feature to be processed and the first split feature in the original features.

[0108] According to embodiments of this disclosure, it is applicable to convolution processing of various large-sized features, while improving the utilization of the processing array, shortening the processing time, and improving task processing efficiency.

[0109] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0110] Figure 11 This is a block diagram of a processing apparatus provided in an embodiment of the present disclosure.

[0111] Reference Figure 11 This disclosure provides a processing apparatus 1100, which includes:

[0112] The splitting module 1110 is used to split each feature to be processed into at least one first-class sub-feature and at least one second-class sub-feature;

[0113] The first processing module 1120 is used to input each first type of sub-feature into the processing array for calculation to obtain the first calculation result of each feature to be processed.

[0114] The second processing module 1130 is used to combine multiple second-type sub-features and input them into the processing array for calculation to obtain the second calculation result of each feature to be processed.

[0115] The determination module 1140 is used to determine the target calculation result of each feature to be processed based on the first calculation result and the second calculation result of each feature to be processed.

[0116] In this embodiment, a splitting module splits each feature to be processed into at least one first-type sub-feature and at least one second-type sub-feature; a first processing module individually inputs each first-type sub-feature into a processing array for calculation, obtaining a first calculation result for each feature to be processed; a second processing module combines multiple second-type sub-features and inputs them into the processing array for calculation, obtaining a second calculation result for each feature to be processed; finally, a determining module determines the target calculation result for each feature to be processed based on the first and second calculation results. Firstly, the reasonable splitting of the features to be processed provides a foundation for subsequent feature combination; secondly, by combining multiple second-type sub-features before inputting them into the processing array for calculation, it is equivalent to performing the calculation of multiple second-type sub-features in parallel during a single processing step. Compared to directly calculating each second-type sub-feature separately, this effectively improves the utilization rate of the processing array, while also shortening the calculation time and improving computational efficiency.

[0117] In addition, this disclosure also provides electronic devices and computer-readable storage media, all of which can be used to implement any of the processing array-based processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section and will not be repeated here.

[0118] Figure 12 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0119] Reference Figure 12 This disclosure provides an electronic device comprising: at least one processor 1201; at least one memory 1202; and one or more I / O interfaces 1203 connected between the processor 1201 and the memory 1202; wherein the memory 1202 stores one or more computer programs executable by the at least one processor 1201, the one or more computer programs being executed by the at least one processor 1201 to enable the at least one processor 1201 to perform the above-described processing array-based processing method.

[0120] Figure 13 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0121] Reference Figure 13 This disclosure provides an electronic device that includes multiple processing cores 1301 and an on-chip network 1302. The multiple processing cores 1301 are all connected to the on-chip network 1302, and the on-chip network 1302 is used to exchange data between the multiple processing cores and external data.

[0122] One or more processing cores 1301 store one or more instructions, and the one or more instructions are executed by one or more processing cores 1301 to enable one or more processing cores 1301 to perform the above-described processing array-based processing method.

[0123] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the above-described processing array-based processing method. The computer-readable storage medium may be volatile or non-volatile.

[0124] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described processing method based on a processing array.

[0125] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0126] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0127] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0128] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0129] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0130] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0131] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0132] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0134] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A processing method based on a processing array, characterized by, The method comprises the following steps: obtaining each to-be-processed feature and splitting it into at least one first-type sub-feature and at least one second-type sub-feature, wherein the to-be-processed feature comprises a feature obtained based on at least one of image data, text data, and audio data; inputting each first-type sub-feature into the processing array for calculation separately to obtain a first calculation result of each to-be-processed feature; combining a plurality of second-type sub-features and then inputting them into the processing array for calculation to obtain a second calculation result of each to-be-processed feature; determining a target calculation result of each to-be-processed feature according to the first calculation result and the second calculation result of each to-be-processed feature; the sum of the calculation amounts of any two first-type sub-features is greater than a preset calculation threshold of the processing array, and the sum of the calculation amounts of at least two second-type sub-features is less than or equal to the calculation threshold of the processing array, wherein the calculation threshold is determined according to the size of the processing array.

2. The treatment method according to claim 1, characterized in that, Before the step of obtaining each to-be-processed feature and splitting it into at least one first-type sub-feature and at least one second-type sub-feature, the method further comprises the following steps: splitting an obtained original feature into at least one first split feature and at least one second split feature, and taking the second split feature as the to-be-processed feature; wherein the calculation amount of the first split feature is less than or equal to the calculation threshold of the processing array, and the calculation amount of the second split feature is greater than the calculation threshold of the processing array.

3. The treatment method of claim 1, wherein The to-be-processed feature is a grouped convolution feature, the channel size of the first-type sub-feature of each grouped convolution feature is consistent, and the channel size of the second-type sub-feature of each grouped convolution feature is consistent.

4. The treatment method according to claim 3, characterized in that, The step of determining a target calculation result of each to-be-processed feature according to the first calculation result and the second calculation result of each to-be-processed feature comprises the following steps: for each to-be-processed feature, superimposing the first calculation result and the second calculation result of the to-be-processed feature to obtain an intra-group convolution result of the to-be-processed feature; splicing a plurality of intra-group convolution results to obtain a grouped convolution result.

5. The treatment method of claim 1, wherein The calculation amount is determined according to the channel size of the feature, and the calculation threshold is determined according to the size of the processing array; wherein the channel size comprises an input channel size and an output channel size, and the size of the processing array comprises an input size and an output size.

6. The treatment method according to claim 5, characterized in that, The step of obtaining each to-be-processed feature and splitting it into at least one first-type sub-feature and at least one second-type sub-feature comprises the following steps: based on an input channel, the input channel size of the to-be-processed feature is greater than the input size of the processing array.

7. The treatment method of claim 5, wherein The step of obtaining each to-be-processed feature and splitting it into at least one first-type sub-feature and at least one second-type sub-feature comprises the following steps: based on an output channel, the output channel size of the to-be-processed feature is greater than the output size of the processing array.

8. The processing method of claim 5, wherein, The step that the sum of the calculation amounts of any two first-type sub-features is greater than a preset calculation threshold of the processing array comprises the following steps: The sum of input channel sizes of any two first-type sub-features is greater than the input size of the processing array, and the sum of output channel sizes of the two first-type sub-features is less than or equal to the output size of the processing array. Or, the sum of output channel sizes of any two first-type sub-features is greater than the output size of the processing array, and the sum of input channel sizes of the two first-type sub-features is less than or equal to the input size of the processing array.

9. The processing method of claim 5, wherein, The sum of calculation amounts of the at least two second-type sub-features is less than or equal to the calculation threshold of the processing array, including: The sum of input channel sizes of at least two second-type sub-features is less than or equal to the input size of the processing array, and the sum of output channel sizes of the two second-type sub-features is less than or equal to the output size of the processing array. Or, the sum of output channel sizes of at least two second-type sub-features is less than or equal to the output size of the processing array, and the sum of input channel sizes of the two second-type sub-features is less than or equal to the input size of the processing array.

10. The treatment method of claim 1, wherein, The combining of multiple second-type sub-features and inputting into the processing array for calculation to obtain the second calculation result of each to-be-processed feature, including: Combining multiple second-type sub-features with a sum of calculation amounts less than or equal to a calculation threshold to obtain at least one combined feature; Inputting each combined feature into the processing array for calculation to obtain the second calculation result of each to-be-processed feature.

11. The treatment method of claim 1, wherein, The processing array includes multiple processing units arranged in an array, and the processing units have a multiply-add operation function, and there is a corresponding relationship between the first-type sub-features and the second-type sub-features obtained by dividing the same to-be-processed feature; The combining of multiple second-type sub-features and inputting into the processing array for calculation to obtain the second calculation result of each to-be-processed feature, including: Inputting each second-type sub-feature into a target row of the processing array for calculation, wherein the target row is a row or a column in which a processing unit for calculating a first-type sub-feature corresponding to the second-type sub-feature is located.

12. A processing device, characterized by Including: A splitting module is configured to split each to-be-processed feature into at least one first-type sub-feature and at least one second-type sub-feature, the to-be-processed feature including a feature obtained based on at least one of image data, text data, and audio data; A first processing module is configured to input each first-type sub-feature into a processing array for calculation to obtain a first calculation result of each to-be-processed feature; A second processing module is configured to combine multiple second-type sub-features and input into the processing array for calculation to obtain a second calculation result of each to-be-processed feature; A determining module is configured to determine a target calculation result of each to-be-processed feature according to the first calculation result and the second calculation result of each to-be-processed feature. A sum of calculation amounts of any two of the first type of sub-features is greater than a preset calculation threshold of the processing array, and a sum of calculation amounts of at least two of the second type of sub-features is less than or equal to the calculation threshold of the processing array, the calculation threshold being determined according to a size of the processing array.

13. An electronic device, comprising: Comprise: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory has stored thereon one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the processing method based on the processing array as claimed in any one of claims 1-11.

14. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the processing method based on the processing array as claimed in any one of claims 1-11.

Images