Data processing method, data processing device, and computer-readable storage medium
By fusing and splitting the convolutional input data and converting it into single-row convolutional computation, the problem of low conversion efficiency between different data layout formats is solved, thereby improving the performance of convolutional computation and the speed of device operation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2022-12-30
- Publication Date
- 2026-05-29
AI Technical Summary
In convolutional computation, existing techniques are inefficient when converting between different data layout formats, and the numerous and complex constraints on convolutional layers lead to reduced vector computation performance.
By obtaining the fusion parameters of the convolution input data, the data is fused and split, converted into single-row convolution calculation, and combined with the preset data layout format, the efficiency of vector calculation is improved.
It improves the performance of convolution calculations, simplifies the processing flow, is suitable for various occasions, and increases the running speed on programmable devices.
Smart Images
Figure CN116129241B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer deep learning technology, and in particular to a data processing method, a data processing apparatus, and a computer-readable storage medium. Background Technology
[0002] Deep learning is a popular field in machine learning, and convolutional layers are an important structure in deep learning networks. The performance of convolutional computation has a significant impact on the performance of deep learning. On some programmable devices (such as DSPs), convolutional computation is usually implemented using vector instructions to achieve highly parallel computing.
[0003] To maximize the efficiency of vector instructions, different data arrangement formats are introduced: for example, feature data with a large width W uses NCHW arrangement, while feature data with a small width W but a large number of channels C uses NHWC arrangement. Thus, different arrangement formats may be used during the computation of the same network, requiring the selection of the optimal arrangement format and data conversion between different arrangement formats.
[0004] If convolutional computation uses the NCHW layout, with vector computation along the W direction, the performance of vector computation will significantly decrease when the amount of data in the W dimension is small, or just a few data points beyond the vector length, because its effective computational cost may only account for a small fraction of the vector length. If the layout is dynamically selected based on feature dimension information during convolutional computation, data conversion between different layouts (such as NCHW and NHWC) is required. Furthermore, NHWC layout requires the amount of data in the C dimension (i.e., the number of channels) to be aligned to a multiple of the vector length for high utilization of vector computation instructions. Ensuring high performance for vector operations imposes numerous constraints on the convolutional layers, which are difficult to meet in practice and involve complex processing. Summary of the Invention
[0005] This application provides a data processing method, a data processing apparatus, and a computer-readable storage medium.
[0006] This application provides a data processing method, the data processing method comprising:
[0007] Convolutional input data is obtained based on the image to be processed, wherein the convolutional input data is arranged according to a preset data layout format;
[0008] Based on the convolution parameters, obtain the fusion parameters of the convolution input data;
[0009] Based on the fusion parameters and the size parameters of the convolution output data, the data volume of the convolution output data is obtained;
[0010] When the amount of data meets the discrimination condition, the convolutional input data is fused based on the fusion parameters to obtain fused convolutional data.
[0011] Perform single-row convolution calculation on the fused convolutional data according to the convolution parameters to obtain single-row output data;
[0012] The single-row output data is split based on the fusion parameters to obtain the convolution output data of the convolution input data. The convolution output data is used to perform image recognition on the image to be processed.
[0013] The fusion parameters include the amount of invalid data;
[0014] The step of obtaining the fusion parameters of the convolutional input data based on the convolutional parameters includes:
[0015] Obtain the padding parameter and kernel size from the convolution parameters;
[0016] Based on the padding parameters and the convolution kernel size, the amount of invalid data is obtained.
[0017] The step of obtaining the data volume of the convolutional output data based on the fusion parameters and the size parameters of the convolutional output data includes:
[0018] Obtain the first and second size parameters of the convolution output data;
[0019] Obtain the sum of the second size parameter and the invalid data amount;
[0020] The data volume of the convolution output data is obtained based on the difference between the product of the first size parameter and the sum of the parameters and the amount of invalid data.
[0021] The dimension of the invalid data volume is the same as the dimension of the second size parameter.
[0022] The step of obtaining the amount of invalid data based on the padding parameters and the convolution kernel size includes:
[0023] Obtain the span parameter from the convolution parameters;
[0024] Based on the third size parameter of the convolution input data, as well as the span parameter and the padding parameter, the amount of connected data is obtained;
[0025] Based on the amount of connected data, the second size parameter, and the third size parameter, the amount of invalid data is obtained;
[0026] Wherein, the dimension of the second dimension parameter is the same as the dimension of the third dimension parameter.
[0027] The step of performing single-row convolution calculation on the fused convolutional data according to the convolution parameters includes:
[0028] Obtain the hole parameter from the convolution parameters;
[0029] The data location of each fused convolutional data is located according to the hole parameter and the span parameter;
[0030] The offset of each fused convolutional data is obtained based on the hole parameter, the span parameter, the third size parameter, and the amount of connected data;
[0031] Extract the fused convolutional data according to the data position and the offset, and perform single-row convolution calculation.
[0032] The step of fusing the convolutional input data based on the fusion parameters to obtain fused convolutional data includes:
[0033] The convolutional input data is padded according to the padding parameters.
[0034] The convolutional input data is divided into several groups according to the span parameter to obtain fused convolutional data, wherein adjacent rows of data within each group are connected using zero data connections of the connection data volume;
[0035] The number of groups is determined by the span parameter.
[0036] The step of fusing the convolutional input data based on the fusion parameters to obtain fused convolutional data includes:
[0037] The convolutional input data is divided into several groups according to the span parameter to obtain fused convolutional data, wherein adjacent rows of data within each group are connected using zero data connections of the connection data volume;
[0038] The number of groups is determined by the span parameter;
[0039] The step of performing single-row convolution calculation on the fused convolutional data according to the convolution parameters includes:
[0040] Each group of data in the fused convolutional data is padded according to the padding parameters.
[0041] Perform single-row convolution calculations on the expanded and fused convolutional data according to the convolution parameters.
[0042] The step of splitting the single-row output data based on the fusion parameters to obtain the convolutional output data of the convolutional input data includes:
[0043] Obtain the first and second size parameters of the convolution output data;
[0044] According to the data order of the single-row output data, the output data corresponding to the second size parameter is obtained as the first row of the convolution output data;
[0045] According to the data order of the single-line output data, discard the output data corresponding to the invalid data volume;
[0046] According to the data order of the single row output data, the output data corresponding to the second size parameter is obtained as the second row output of the convolution output data;
[0047] The output data is generated by obtaining the number of rows corresponding to the first size parameter.
[0048] The data processing method further includes:
[0049] Get the device's vector length;
[0050] Based on the amount of data in the convolutional output data and the vector length, the first discrimination parameter is obtained;
[0051] A second discrimination parameter is obtained based on a preset judgment factor, the first and second size parameters of the convolution output data, and the vector length;
[0052] When the first discrimination parameter is less than the second discrimination parameter, the data volume is determined to meet the discrimination condition.
[0053] This application also provides a data processing apparatus, which includes a processor and a memory, wherein the memory stores program data, and the processor executes the program data to implement the data processing method described above.
[0054] This application also provides a computer-readable storage medium for storing program data, which, when executed by a processor, is used to implement the above-described data processing method.
[0055] The beneficial effects of this application are as follows: The data processing device acquires convolutional input data based on the image to be processed, wherein the convolutional input data is arranged according to a preset data layout format; based on the convolutional parameters, it acquires the fusion parameters of the convolutional input data; based on the fusion parameters and the size parameters of the convolutional output data, it acquires the data volume of the convolutional output data; when the data volume meets the discrimination condition, it performs fusion processing on the convolutional input data based on the fusion parameters to acquire fused convolutional data; it performs single-row convolution calculation on the fused convolutional data according to the convolutional parameters to acquire single-row output data; it splits the single-row output data based on the fusion parameters to acquire the convolutional output data of the convolutional input data, and the convolutional output data is used for image recognition of the image to be processed. Through the above method, the data processing device combines the fusion of input data and the splitting of output data, converting multi-row convolution calculation into single-row calculation, thereby improving the calculation speed when using vectors for calculation. Attached Figure Description
[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:
[0057] Figure 1 This is a flowchart illustrating an embodiment of the data processing method provided in this application;
[0058] Figure 2 This is a schematic diagram of the overall process of the data processing method provided in this application;
[0059] Figure 3 This is a schematic diagram of the structure of an embodiment of the data processing apparatus provided in this application;
[0060] Figure 4 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation
[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0062] Vector length refers to the number of elements that a vector register can store, and it is also the maximum amount of data that a vector instruction can process in a single execution. Vector length is usually determined by the register size and the data size. For example, for a 512-bit register, if the data size is 8 bits (e.g., char, unsigned char), the vector length is 512 / 8 = 64; if the data size is 32 bits (e.g., int32, unsigned int32, float), the vector length is 512 / 32 = 16. Typically, the data size, register size, and vector length are all integer powers of 2.
[0063] Loading and storing vector data: When loading data from RAM into a vector register, or storing data from a vector register into RAM, it is generally required that the address be aligned to the vector length and the data be stored contiguously in RAM. Even if some instructions can access unaligned or non-contiguous RAM data, their access efficiency will decrease. To improve the efficiency of vector computation, it is usually necessary to store data in RAM in the same order as the vector operation. For example, in convolution operations, if the vector operation processes data within the same row (continuously increasing in the w direction), the NCHW format can be used in RAM; if the vector operation processes data from different channels (continuously increasing in the c direction), the NHWC format can be used in RAM. Here, this application refers to W and C as the dimensions of vector processing in two different cases. When the number of data in the dimension of vector processing exceeds the length of the vector, it needs to be split and looped according to the length of the vector, and the amount of data processed in the last step may be less than the length of the vector. In particular, if the total amount of data in the dimension of vector processing is less than the length of the vector, then only one processing is needed, and the effective amount of data is the amount of data in that dimension.
[0064] Based on the above concepts, this application utilizes the NCHW data arrangement format in convolution operations and employs a unique multi-row fusion technique. This allows data from different rows to be processed in a single vector operation, thereby increasing the proportion of effective data and improving the performance of convolution computation. This technique has no specific requirements on the dimensionality of the feature data, is applicable to various scenarios, effectively improves the running speed of various convolutional neural networks on programmable vector devices, and is relatively simple to process.
[0065] In other embodiments, the data layout format of NCWH can also be used, and the possible cases will not be listed here.
[0066] Please refer to the details. Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating an embodiment of the data processing method provided in this application. Figure 2 This is a schematic diagram of the overall process of the data processing method provided in this application.
[0067] The data processing method of this application is applied to a data processing apparatus, which can be a server or a system consisting of a server and terminal devices working together. Accordingly, the various parts of the data processing apparatus, such as units, subunits, modules, and submodules, can be all located in the server, or they can be located separately in the server and the terminal devices.
[0068] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed servers, or as a single software program or software module; no specific limitation is made here. In some possible implementations, the data processing method of this application embodiment can be implemented by a processor calling computer-readable instructions stored in memory.
[0069] Specifically, such as Figure 1 As shown, the data processing method in this application embodiment specifically includes the following steps:
[0070] Step S11: Obtain convolution input data based on the image to be processed, wherein the convolution input data is arranged according to a preset data layout format.
[0071] In this embodiment, the data processing device obtains convolutional input data arranged in NCHW from the image to be processed through feature extraction. In other embodiments, if the preset data arrangement format is NCWH, the data processing device obtains convolutional input data arranged in NCWH.
[0072] Step S12: Based on the convolution parameters, obtain the fusion parameters of the convolution input data.
[0073] In this embodiment of the application, the general calculation formula for two-dimensional convolution is given as follows:
[0074]
[0075] Where O represents the convolution output data, wt represents the weight data, and I represents the convolution input data. The subscripts of O and I are the index values of dimensions N, C, H, and W, respectively. b represents the batch processing index value, C o h represents the output index on the channel. o Indicates the output index along dimension H, w o Indicates the index of the output in dimension W; C I H represents the number of input channels. K W represents the size of the convolution kernel in the H direction.K This indicates the size of the convolution kernel in the W direction.
[0076] s h and s w p is the stride parameter in the H and W directions of convolution. h and p w These are the padding parameters in the H and W directions of the convolution, d h and d w These are the dilation parameters for the convolution in the H and W directions. This formula requires parameters for b and c. o ,h o ,w o Perform a traversal to obtain all the output results.
[0077] This application uses the NCHW data format. Loops in N and C dimensions can be considered as outer loops. The data processing device can fuse H and W dimensions. To simplify the explanation of the entire scheme, the following description assumes that the batch size, the number of input data channels, and the number of output data channels are all 1. It should be noted that the entire data processing process can be extended to cases where the batch size and the number of input / output channels are not 1.
[0078] In the simplified case, since O, I, and wt are all two-dimensional data, the above convolution calculation formula can be simplified to:
[0079]
[0080] The height of the convolution output data is H. O Width is W O The height of the convolution input data is H. I The input width is W I .
[0081] It should be noted that when the subscript of I in the calculation formula is outside its valid range, the boundary padding value 0 is used.
[0082] According to the above calculation formula, when h o When the values remain constant (i.e., the convolution outputs are in the same row), the result depends only on H of the convolution input data. K Therefore, in single-row calculations, this application can include the weights and H. K The starting address of the row data is used as the input parameter of this function. With other parameters remaining constant, its computational complexity will be similar to ceil(W). O It is directly proportional to / L).
[0083] Where L represents the vector length, ceil() means taking the smallest integer not less than the input parameter, and it is assumed that each vector processing (which can be a series of instruction combinations) produces output data of length L.
[0084] This application first considers the span parameter s of the convolution. h s w and the void parameter d h d w In the simple case where both values are 1, the fusion processing of the convolutional input data is as follows: first, p is added to the top and bottom of the input data respectively. h The data is divided into rows of zeros, and then each pair of rows is concatenated using pw zeros. The formula for calculating the size of the merged convolutional data is:
[0085] I total =(H I +2*p h )*(W I +p w )-p w
[0086] The formula for calculating the amount of invalid data in the fusion parameters is:
[0087] extra_num = W I +p w -W O =W K -1-p w
[0088] Where extra_num represents the number of invalid data obtained by merging rows between the two rows of data in the actual output, i.e., the amount of invalid data.
[0089] Next, consider the span parameter s of the convolution. h and s w It is not 1, but the hole parameter d h and d w In the case where the value is still 1, the fusion processing method degenerates to s. h and s w When the value is 1, the processing method is exactly the same as before.
[0090] The line fusion processing method for the input data is as follows: first, p is added to the top and bottom of the input data respectively. h Take zero data, then expand the data and group it according to the row number, calculated as group number = row number mod s. h Here, mod is the modulo operator.
[0091] The group numbers start from zero and have a maximum value of s. h-1; the row number is the padded row number, also starting from zero. For each group of data, every two rows are connected using mid_zero zeros, ultimately resulting in s. h The input data after group fusion, where mid_zero, i.e., the amount of concatenated data, is calculated as follows:
[0092] mid_zero = ceil((W I +p w ) / s w )*s w -W I
[0093] That is, the number of zeros inserted in the middle must be no less than p. w There are 1, and the starting address between two adjacent rows is s. w Integer multiples of. When performing the merged single-row convolution operation, the h-th... k The data in row h is located at the hth row. k mod s h In the group of data, the offset of the first data is floor(h) k / s h )*(W I +mid_zero), here h k The value range is 0, 1, ..., H K -1. The amount of invalid data between every two rows of output data after convolution in the fusion mode is:
[0094] extra_num = (W I +mid_zero) / s w -W O
[0095] Finally, consider d. h and d w In the case where the input value is not 1, the fusion processing of the input data is exactly the same as in the previous case, except that the position of the input data acquisition changes during convolution calculation. The h-th... k The row data is located at the hth k *d h mod s h In the group of data, the offset of the first data is floor(h) k *d h / s h )*(W I +mid_zero), the calculation of other quantitative relationships remains unchanged. When d h and d w When it degenerates to 1, the formula here is the same as before.
[0096] The fusion parameters such as the amount of invalid data and the amount of connected data in the embodiments of this application can be used to split the convolution output data, and can also be used to calculate O. total And it is used in the judgment process.
[0097] Step S13: Based on the fusion parameters and the size parameters of the convolution output data, obtain the data volume of the convolution output data.
[0098] In this embodiment of the application, the formula for calculating the data volume of the convolution output data after convolution calculation based on the fused convolution data is as follows:
[0099] O total = O *(W O +extra_)-extra_
[0100] The first size parameter of the convolution output data is H. O The second dimension parameter is W O The amount of invalid data is extra_num.
[0101] Step S14: When the amount of data meets the discrimination condition, perform fusion processing on the convolution input data based on the fusion parameters to obtain fused convolution data.
[0102] In this embodiment, because multi-line fusion requires additional operations—namely, fusion of the inputs and splitting of the outputs—this application introduces a judgment factor λ when determining whether to use fusion processing. When ceil(O) is satisfied... total / L)<λH O *ceil(W O When / L), convolution calculation is performed using multi-row fusion; when the above criteria are not met, then as follows: Figure 2 As shown, convolution is performed row by row to obtain the output data (NCHW) for each row.
[0103] The first discriminant parameter is ceil(O) total / L), the second discriminant parameter is λH O *ceil(W O / L).
[0104] If multi-line merging is not used, and each line is processed separately, assuming the vector length is L, then each line needs to be ceil(W). O / L) vector processing steps, requiring a total of H steps O *ceil(W O / L) times vector processing.
[0105] It should be noted that λ <= 1 here, and can be adjusted according to the characteristics of the platform and other parameters in the convolution.
[0106] Step S15: Perform single-row convolution calculation on the fused convolutional data according to the convolution parameters to obtain single-row output data.
[0107] In this embodiment of the application, during the single-row convolution calculation process, the data processing device uses h when calculating the convolution output data. k The starting address of the row input is offset from the fused convolutional data by 0, (W) I +p w ), (W I +p w The data processing device performs single-row convolution calculations based on the offsets in the fused convolutional data to obtain single-row output data.
[0108] Step S16: Split the single-row output data based on the fusion parameters to obtain the convolution output data of the convolution input data. The convolution output data is used for image recognition of the image to be processed.
[0109] In this embodiment of the application, the data processing device's output splitting operation is also very simple: extract W from a single line of output data. O The first row of output data is taken as the first data, skipping the extra_num data before taking W. O The first data point is output as the second line, and this process is repeated until all H data points are obtained. O The output data is processed in rows to obtain the final convolutional output data. The final convolutional output data can be used for image recognition of the image to be processed, and specific applications include, but are not limited to: image target recognition, target detection, target tracking, and target classification.
[0110] In summary, when the data processing device uses a fusion method to calculate the output, a total of ceil(O) is required. total / L) times of vector processing. Compared to the number of vector processing times H without using multi-row fusion. O *ceil(W O / L), which clearly shows a significant reduction in vector processing resources.
[0111] To address the above situation, the following example will specifically analyze the difference between using multiline blending and not using multiline blending:
[0112] For example, a 3x3 convolution with a 7x7 input and a vector length of 64 can be performed in two ways: without boundary expansion and with boundary expansion (with a padding parameter of 1).
[0113] Without boundary padding, the output is 5x5, requiring 5 vector processing steps without fusion. After fusion, extra_num = 2, and the total output data size is 5*(5+2)–2 = 33. Only 1 vector processing step is needed during the convolution calculation, which is 1 / 5 of the original.
[0114] When performing boundary padding, the output is 7x7. Without boundary fusion, 7 vector processing steps are required. After fusion, extra_num = 1, and the total output data size is 7*(7+1)–1 = 55. During the convolution calculation, only 1 vector processing step is required, which is 1 / 7 of the original.
[0115] In summary, as Figure 2 As shown, Figure 1 Steps S11 to S16 of the data processing method can be uniformly described as follows:
[0116] Calculate the fusion parameters based on the convolution parameters:
[0117] mid_zero = ceil((W I +p w ) / s w )*s w -W I
[0118] extra_num = (W I +mid_zero) / s w -W O
[0119] O total =H O *(W O +extra_num)-extra_num
[0120] Using the discriminant condition: ceil(O total / L)<λH O *ceil(W O If the condition is met, the row fusion scheme is selected; if the condition is not met, the fusion scheme is not selected.
[0121] The input data is then fused: first, p is added to the top and bottom of the input data respectively. h Rows with zero data are then processed according to the group number = row number modulo 1. h The expanded data is divided into s h Groups, with each pair of rows connected by mid_zero zeros.
[0122] Performing single-row convolution calculations: The calculation requires H K Line input data, where the h-th line is... kThe starting address of the line is located at h k *d h mod s h The offset within the group is floor(h) k *d h / s h )*(W I The position of +mid_zero is calculated, and the output is O. total Data.
[0123] Split the output: from O total From the calculation results, take the first W. O The first line of output is taken as the first line, discarding extra_num invalid data, and then taking W. O The first H is output as the second line, and this process is repeated until all H are extracted. O Output data in rows.
[0124] Additionally, when the fill parameter p w If the value is not zero, p can be added to the beginning and end of each set of input data. w The presence of zeros eliminates the need for padding parameters during single-row convolution calculations, allowing for processing as if no padding was used. Padding can be applied before or after group partitioning, as detailed below:
[0125] Before group division: The convolutional input data is expanded according to the padding parameter; the convolutional input data is divided into several groups according to the span parameter to obtain fused convolutional data.
[0126] This occurs after group division: the convolutional input data is divided into several groups according to the span parameter to obtain fused convolutional data; each group of fused convolutional data is padded according to the padding parameter.
[0127] On specific devices, the fusion of convolutional input data and the splitting of convolutional output data can be achieved through DMA data transfer, or by modifying the parameters in the original DMA operation to perform data transfer and transformation simultaneously. This application does not specify the operation for this purpose; it pertains to the specific implementation optimization of the corresponding operation.
[0128] Furthermore, this application does not specify which vector instructions should be used to perform single-row convolution calculations. It simply converts multi-row output convolution calculations into single-row outputs based on the premise that single-row convolution calculations can be performed using vector instructions, and completes the calculations using the same vector instructions. If there are no suitable vector instructions to perform the calculations under certain modes or parameters, then using a multi-row fusion scheme will not change this situation.
[0129] In this embodiment, the data processing device acquires convolutional input data based on the image to be processed, wherein the convolutional input data is arranged according to a preset data layout format; based on the convolutional parameters, it acquires fusion parameters of the convolutional input data; based on the fusion parameters and the size parameters of the convolutional output data, it acquires the data volume of the convolutional output data; when the data volume meets the discrimination condition, it performs fusion processing on the convolutional input data based on the fusion parameters to acquire fused convolutional data; it performs single-row convolution calculation on the fused convolutional data according to the convolutional parameters to acquire single-row output data; and it splits the single-row output data based on the fusion parameters to acquire the convolutional output data of the convolutional input data. The convolutional output data is used for image recognition of the image to be processed. Through the above method, the data processing device combines the fusion of input data and the splitting of output data, converting multi-row convolution calculations into single-row calculations, thereby improving the calculation speed when using vectors for calculation.
[0130] This application, during convolution calculations using vector instructions, leverages NCHW data arrangement to process data belonging to different output rows within a single operation. This increases the amount of valid data in the vector, reduces the total number of vector operations, and thus improves convolution calculation performance. The core computation part can reuse convolution operations with single-row output data and uses only the NCHW data arrangement format, reducing code development and maintenance costs.
[0131] The data processing method of this application combines the fusion of input data and the splitting of output data, converting multi-row convolution calculations into single-row calculations to improve the computation speed when using vectors. Before fusion, a judgment is made that the fusion strategy is only activated if it can effectively reduce the number of vector calculations; otherwise, a non-fusion row-by-row calculation mode is used. The fusion of input data includes padding, grouping, and multi-row concatenation and merging processes and related parameters. The position calculation of the input row is performed when performing single-row convolution calculations. Finally, the method for splitting and restoring the output data is described.
[0132] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.
[0133] To implement the data processing method of the above embodiments, this application also proposes a data processing apparatus, which can be found in detail below. Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the data processing apparatus provided in this application.
[0134] The data processing apparatus 300 of this application embodiment includes a memory 31 and a processor 32, wherein the memory 31 and the processor 32 are coupled together.
[0135] The memory 31 is used to store program data, and the processor 32 is used to execute the program data to implement the data processing method described in the above embodiments.
[0136] In this embodiment, processor 32 can also be referred to as a CPU (Central Processing Unit). Processor 32 may be an integrated circuit chip with signal processing capabilities. Processor 32 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 32 can be any conventional processor.
[0137] To implement the data processing method of the above embodiments, this application also provides a computer-readable storage medium, such as... Figure 4 As shown, the computer-readable storage medium 400 is used to store program data 41, which, when executed by a processor, is used to implement the data processing method as described in the above embodiments.
[0138] This application also provides a computer program product, wherein the computer program product includes a computer program operable to cause a computer to perform the data processing method as described in the embodiments of this application. The computer program product may be a software installation package.
[0139] The data processing method described in the above embodiments of this application, when implemented as a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0140] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A data processing method based on image processing, characterized in that, The data processing method includes: Convolutional input data is obtained based on the image to be processed, wherein the convolutional input data is arranged according to a preset data layout format; Based on the convolution parameters, obtain the fusion parameters of the convolution input data; Based on the fusion parameters and the size parameters of the convolution output data, the data volume of the convolution output data is obtained; When the amount of data meets the discrimination condition, the convolutional input data is fused based on the fusion parameters to obtain fused convolutional data. Perform single-row convolution calculation on the fused convolutional data according to the convolution parameters to obtain single-row output data; Based on the fusion parameters, the single-row output data is split to obtain the convolutional output data of the convolutional input data. The convolutional output data is used to perform image recognition on the image to be processed. The step of splitting the single-row output data based on the fusion parameters to obtain the convolutional output data of the convolutional input data includes: Obtain the first and second size parameters of the convolution output data; According to the data order of the single-row output data, the output data corresponding to the second size parameter is obtained as the first row of the convolution output data; According to the data order of the single-line output data, discard the output data corresponding to the invalid data volume; According to the data order of the single row output data, the output data corresponding to the second size parameter is obtained as the second row output of the convolution output data; The output data is generated by obtaining the number of rows corresponding to the first size parameter.
2. The data processing method according to claim 1, characterized in that, The fusion parameters include the amount of invalid data; The step of obtaining the fusion parameters of the convolutional input data based on the convolutional parameters includes: Obtain the padding parameter and kernel size from the convolution parameters; Based on the padding parameters and the convolution kernel size, the amount of invalid data is obtained.
3. The data processing method according to claim 2, characterized in that, The step of obtaining the data volume of the convolutional output data based on the fusion parameters and the size parameters of the convolutional output data includes: Obtain the first and second size parameters of the convolution output data; Obtain the sum of the second size parameter and the invalid data amount; The data volume of the convolution output data is obtained based on the difference between the product of the first size parameter and the sum of the parameters and the amount of invalid data. The dimension of the invalid data volume is the same as the dimension of the second size parameter.
4. The data processing method according to claim 3, characterized in that, The step of obtaining the amount of invalid data based on the padding parameters and the convolution kernel size includes: Obtain the span parameter from the convolution parameters; Based on the third size parameter of the convolutional input data, as well as the span parameter and the padding parameter, the amount of connected data is obtained; Based on the amount of connected data, the second size parameter, and the third size parameter, the amount of invalid data is obtained; Wherein, the dimension of the second dimension parameter is the same as the dimension of the third dimension parameter.
5. The data processing method according to claim 4, characterized in that, The step of performing single-row convolution calculation on the fused convolutional data according to the convolution parameters includes: Obtain the hole parameter from the convolution parameters; The data location of each fused convolutional data is located according to the hole parameter and the span parameter; The offset of each fused convolutional data is obtained based on the hole parameter, the span parameter, the third size parameter, and the amount of connected data; Extract the fused convolutional data according to the data position and the offset, and perform single-row convolution calculation.
6. The data processing method according to claim 4 or 5, characterized in that, The step of fusing the convolutional input data based on the fusion parameters to obtain fused convolutional data includes: The convolutional input data is padded according to the padding parameters. The convolutional input data is divided into several groups according to the span parameter to obtain fused convolutional data, wherein adjacent rows of data within each group are connected using zero data connections of the connection data volume; The number of groups is determined by the span parameter.
7. The data processing method according to claim 4 or 5, characterized in that, The step of fusing the convolutional input data based on the fusion parameters to obtain fused convolutional data includes: The convolutional input data is divided into several groups according to the span parameter to obtain fused convolutional data, wherein adjacent rows of data within each group are connected using zero data connections of the connection data volume; The number of groups is determined by the span parameter; The step of performing single-row convolution calculation on the fused convolutional data according to the convolution parameters includes: Each group of data in the fused convolutional data is padded according to the padding parameters; Perform single-row convolution calculations on the expanded and fused convolutional data according to the convolution parameters.
8. The data processing method according to claim 1, characterized in that, The data processing method further includes: Get the device's vector length; Based on the amount of data in the convolutional output data and the vector length, the first discrimination parameter is obtained; A second discrimination parameter is obtained based on a preset judgment factor, the first and second size parameters of the convolution output data, and the vector length; When the first discrimination parameter is less than the second discrimination parameter, the data volume is determined to meet the discrimination condition.
9. A data processing apparatus, characterized in that, The data processing device includes a processor and a memory, the memory storing program data, and the processor executing the program data to implement the data processing method as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program data, which, when executed by a processor, is used to implement the data processing method according to any one of claims 1-8.