Data processing method, data processing device and electronic equipment

Abstract

The application provides a data processing method, a data processing device and an electronic equipment, and relates to the technical field of data operation. The method comprises the following steps: determining a to-be-input data stream of a corresponding input data type; determining a target operation method in a plurality of operation modules of a hardware circuit according to the data stream quantity of the to-be-input data stream; and performing calculation on the to-be-input data stream based on the target operation method, obtaining a target data stream of an output data type, and taking the target data stream as an operation factor of a neural network. The application sets a plurality of operation modules with different functions in the hardware circuit, determines a corresponding target operation method in the plurality of operation modules according to the data quantity of the to-be-input data stream, and calculates a target data stream of a required type. The resources in the hardware circuit can be reused to process data of different quantities of data streams, different input data types and output data types, the area of the hardware circuit is reduced, the cost of calculation is reduced, and the efficiency of calculation is improved.

Description

Data processing methods, data processing devices and electronic equipment Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data processing method, a data processing device, and an electronic device. Background Technology

[0002] The fused multiply-accumulate (fma) operation is a common operation in digital signal processors. It performs ∑(a*b)+c on three operands a, b, and c, simplifying an operation that would normally require several instructions into a single instruction.

[0003] Traditional fusion multiply-accumulate operations are simple in structure and limited in function, only performing addition and multiplication. However, in neural network computing applications, the input data streams and inputs have various data types. Current fusion multiply-accumulate operations cannot handle the diverse input data types in neural network computing, nor can they perform corresponding vector operations. This results in high hardware resource consumption, significant area constraints in hardware circuit design, high computational cost, and low efficiency. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of this application is to provide a data processing method, a data processing device, and an electronic device to improve the problem of high computational cost and low efficiency when hardware circuits perform fusion multiplication and addition operations in the prior art.

[0005] To address the aforementioned problems, firstly, embodiments of this application provide a data processing method, the method comprising:

[0006] Determine the input data stream of the corresponding input data type;

[0007] Based on the number of data streams to be input, the target operation method is determined among multiple operation modules of the hardware circuit;

[0008] The target operation method is used to calculate the input data stream to obtain the target data stream of the output data type, and the target data stream is used as the operation factor of the neural network.

[0009] In the above implementation process, by limiting the data type of the input data stream, subsequent calculations are performed, improving computational efficiency. Furthermore, by setting up multiple computation modules with different functions in the hardware circuit for data processing, the appropriate target computation method can be determined among multiple computation modules based on the amount of data in the input data stream. The input data stream is then calculated according to the target computation method to obtain the target data stream of the required output type. This allows for the reuse of resources from multiple computation modules in the hardware circuit to process data streams of different quantities, input data types, and output data types, implementing various computational functions. This improves computational accuracy, reduces instruction operations during processing, thereby improving the efficiency of hardware circuit compilation. It also reduces the area pressure during hardware circuit design, thus reducing the area of ​​the hardware circuit and computational costs, and ultimately improving computational efficiency.

[0010] Optionally, the input data stream includes: a first input data stream, and / or a second input data stream, and / or a third input data stream;

[0011] The step of determining the input data stream of the corresponding input data type includes:

[0012] If the input data stream includes the first input data stream or the second input data stream, determine the first input data type of the first input data stream and the second input data stream;

[0013] If the input data stream includes the third input data stream, determine the second input data type of the third input data stream.

[0014] In the above implementation process, to achieve the related calculations of fused multiplication, accumulation, and addition operations, the input data stream can include three data streams. Before calculation, the data types of the three data streams can be set accordingly. The first input data stream and the second input data stream are set to the same first input data type, and the third input data stream is set to a second input data type that is the same as or different from the first input data type. By limiting the input data types, subsequent unified calculation and processing can be performed, thereby improving the efficiency of the calculation and making it suitable for various FPGA or integrated circuit IP designs.

[0015] Optionally, determining the target computation method among multiple computation modules of the hardware circuit based on the number of data streams in the input data stream includes:

[0016] The number of data streams in the data stream to be input is determined; wherein, if the data stream to be input includes the first input data stream, the second input data stream, or the third input data stream, the number of data streams is 1; if the data stream to be input includes the first input data stream and the second input data stream, or the first input data stream and the third input data stream, or the second input data stream and the third input data stream, the number of data streams is 2; if the data stream to be input includes the first input data stream, the second input data stream, and the third input data stream, the number of data streams is 3.

[0017] If the number of data streams is 1, then among the multiple computing modules, the single-path computing method is determined as the target computing method;

[0018] If the number of data streams is 2, the dual-path operation method is determined as the target operation method among the multiple operation modules;

[0019] If the number of data streams is 3, the three-way operation method is determined as the target operation method among the multiple operation modules;

[0020] The hardware circuit includes multiple arithmetic modules such as a product module, an accumulation module, an addition module, a constant module, a shift module, a channel module, and a truncation module.

[0021] In the above implementation process, the number of data streams input to the hardware circuit for computation in actual computing scenarios is uncertain. Therefore, to process different numbers of data streams, the number of input data streams to be processed can be determined first based on whether each input data stream is enabled. Then, based on the different numbers of data streams, the appropriate module can be selected from multiple computation modules in the hardware circuit for computation, forming various computational methods. By setting up computational modules with different functions in the hardware circuit, various different operations and corresponding functions can be implemented, such as multiplication, accumulation, addition, adding constants, shifting, etc. It can also process and truncate channels of clock cycles for various input and output data types, thus supporting mixed operations with different sign bits for different numbers of data streams, performing various types of calculations and processing, improving the versatility and accuracy of computation.

[0022] Optionally, if the target computation method is the single-channel computation method, then the step of performing computation on the input data stream based on the target computation method to obtain the target data stream of the output data type includes:

[0023] The accumulation module calculates the input data stream to obtain the accumulated data.

[0024] The addition module and the constant module are used to perform addition calculations on the accumulated data and the preset constant to obtain the addition data;

[0025] The addition data is shifted and rounded according to the shift module to obtain shifted data;

[0026] Based on the channel module, the number of channels in the clock cycle of the shifted data is processed to obtain the transmitted data;

[0027] Based on the output data type, a corresponding target truncation module is selected from among the multiple truncation modules to process the transmitted data and obtain the target data stream.

[0028] In the above implementation, when only one input data stream is enabled, only that one input data stream needs to be processed. During the operation, various operations can be performed on the input data stream, such as accumulation, adding constants, shifting as needed, processing based on the number of channels in the data clock cycle, and selecting the appropriate truncation module to output data of the corresponding structure based on the set output data type. This allows for corresponding calculations and processing of a single input data stream as needed, improving the computational efficiency and accuracy of a single input data stream.

[0029] Optionally, if the target computation method is the dual-path computation method, and the input data stream includes the first input data stream and the second input data stream; then the step of performing computation on the input data stream based on the target computation method to obtain the target data stream of the output data type includes:

[0030] According to the product module, the first input data stream and the second input data stream are multiplied to obtain product data;

[0031] The product data is calculated using the accumulation module to obtain the accumulated data.

[0032] The addition module and the constant module are used to perform addition calculations on the accumulated data and the preset constant to obtain the addition data;

[0033] The addition data is shifted and rounded according to the shift module to obtain shifted data;

[0034] Based on the channel module, the number of channels in the clock cycle of the shifted data is processed to obtain the transmitted data;

[0035] Based on the output data type, a corresponding target truncation module is selected from among the multiple truncation modules to process the transmitted data and obtain the target data stream.

[0036] In the above implementation process, when two input data streams are enabled, and these two input data streams are a first input data stream and a second input data stream with the same input data type, various operations can be performed on the two input data streams, such as multiplication, accumulation, adding constants, shifting as needed, processing according to the number of channels in the data clock cycle, and selecting the appropriate truncation module to output data of the corresponding structure according to the set output data type. This allows for corresponding calculations and processing of two input data streams with the same input data type as required, improving the calculation efficiency and accuracy of the dual-input data streams.

[0037] Optionally, if the target computation method is the dual-path computation method, and the input data stream includes the first input data stream or the second input data stream, and the third input data stream; then the step of calculating the input data stream based on the target computation method to obtain the target data stream of the output data type includes:

[0038] The accumulation module calculates the accumulated data by processing the first input data stream, the second input data stream, and the third input data stream.

[0039] The addition module and the constant module are used to perform addition calculations on the accumulated data and the preset constant to obtain the addition data;

[0040] The addition data is shifted and rounded according to the shift module to obtain shifted data;

[0041] Based on the channel module, the number of channels in the clock cycle of the shifted data is processed to obtain the transmitted data;

[0042] Based on the output data type, a corresponding target truncation module is selected from among the multiple truncation modules to process the transmitted data and obtain the target data stream.

[0043] In the above implementation process, when two input data streams are enabled, and these two input data streams are either a first input data stream, a second input data stream, or a third input data stream with potentially different data types, various operations can be performed on the two input data streams, such as accumulation, adding constants, shifting as needed, processing according to the number of channels in the data clock cycle, and selecting the appropriate truncation module to output data of the corresponding structure according to the set output data type. This allows for corresponding calculations and processing of two input data streams with potentially different data types as needed, improving the calculation efficiency and accuracy of dual-input data streams.

[0044] Optionally, if the target computation method is the three-way computation method, then the step of performing computation on the input data stream based on the target computation method to obtain the target data stream of the output data type includes:

[0045] According to the product module, the first input data stream and the second input data stream are multiplied to obtain product data;

[0046] The product data is calculated using the accumulation module to obtain the accumulated data.

[0047] According to the addition module, the accumulated data, the third input data stream and the preset constant are added together to obtain the addition data;

[0048] The addition data is shifted and rounded according to the shift module to obtain shifted data;

[0049] Based on the channel module, the number of channels in the clock cycle of the shifted data is processed to obtain the transmitted data;

[0050] Based on the output data type, a corresponding target truncation module is selected from among the multiple truncation modules to process the transmitted data and obtain the target data stream.

[0051] In the above implementation, when all three input data streams are enabled, multiplication and accumulation operations can be performed on the first and second input data streams. Combined with the third data stream, various operations can be performed, including adding constants, shifting as needed, processing based on the number of channels in the data clock cycle, and selecting the appropriate truncation module to output data of the corresponding structure based on the set output data type. This allows for corresponding calculations and processing of the three input data streams as required, improving the computational efficiency and accuracy of the three input data streams.

[0052] Optionally, the step of processing the number of channels in the clock cycle of the shifted data according to the channel module to obtain the transmitted data includes:

[0053] The maximum channel value for the clock cycle is determined based on the bus width;

[0054] The channel threshold of the channel module is determined based on the maximum channel value;

[0055] If the number of channels exceeds the channel threshold, the shifted data is processed to reduce parallelism based on the channel module to obtain the transmitted data;

[0056] If the number of channels does not exceed the channel threshold, the shifted data is transmitted transparently based on the channel module to obtain the transmitted data.

[0057] In the above implementation process, when the channel module of the hardware circuit processes the shifted data, it can compare the number of channels of the shifted data with the channel threshold determined based on the maximum channel value to determine whether the shifted data can pass directly. This approach can reduce the parallelism of data with a large number of channels and can also perform multi-level processing through multiple channel modules to ensure that the final output data meets the preset output data type. It is suitable for processing data of various input data types and meeting the requirements of various output data types.

[0058] Optionally, the step of shifting and rounding the addition data according to the shift module to obtain shifted data includes:

[0059] Determine the shift direction and degree of the addition data;

[0060] According to the shift module, the addition data is shifted based on the shift direction and the shift degree to obtain initial shift data;

[0061] Select a target numerical rounding algorithm from among several numerical rounding algorithms to perform numerical rounding on the initial shifted data, thereby obtaining the shifted data.

[0062] In the above implementation process, when shifting data, the direction and degree of shift can be determined according to the requirements, and a suitable numerical rounding algorithm can be selected to perform a single numerical rounding process on the obtained initial shifted data. This can ensure the accuracy of calculation while realizing the operation functions of multiple vector operations.

[0063] Secondly, embodiments of this application also provide a data processing apparatus, the apparatus comprising: a type unit, a determination unit, and a calculation unit;

[0064] The type unit is used to determine the input data stream of the corresponding input data type;

[0065] The determining unit is used to determine the target operation method among multiple operation modules of the hardware circuit based on the number of data streams of the data stream to be input;

[0066] The computing unit is used to perform calculations on the input data stream based on the target operation method to obtain a target data stream of output data type, and to use the target data stream as an operation factor of the neural network.

[0067] In the above implementation process, the type unit limits the data type of the input data stream to facilitate subsequent calculations and improve computational efficiency. Multiple computation modules with different functions are set up in the hardware circuit for data processing. The determining unit determines the appropriate target computation method among these modules based on the amount of data in the input data stream. The computation unit then performs calculations on the input data stream according to the target computation method to obtain the target data stream of the required output type.

[0068] Thirdly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above-described data processing methods.

[0069] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform steps in any of the above-described implementations of the data processing method.

[0070] In summary, the embodiments of this application provide a data processing method, a data processing device, and an electronic device that can reuse the resources of multiple computing modules in a hardware circuit to process data of different quantities, different input data types, and different output data types, thereby realizing a variety of different computing functions, improving the accuracy of calculations, reducing instruction operations during processing, improving the efficiency of hardware circuit compilation, and reducing the area pressure during hardware circuit design, thereby reducing the area of ​​the hardware circuit and the cost of calculation, and improving the efficiency of calculation. Attached Figure Description

[0071] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 is a block diagram of an electronic device provided in an embodiment of this application;

[0073] Figure 2 is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0074] Figure 3 is a detailed flowchart of step S300 provided in an embodiment of this application;

[0075] Figure 4 is a detailed flowchart of the first step S400 provided in the embodiment of this application;

[0076] Figure 5 is a detailed flowchart of the second step S400 provided in the embodiment of this application;

[0077] Figure 6 is a detailed flowchart of the third step S400 provided in the embodiment of this application;

[0078] Figure 7 is a detailed flowchart of the fourth step S400 provided in the embodiment of this application;

[0079] Figure 8 is a detailed flowchart of one of the steps 414, S425, S434 or S445 provided in an embodiment of this application.

[0080] Figure 9 is a detailed flowchart of one of the steps 413, S424, S433 or S444 provided in an embodiment of this application.

[0081] Figure 10 is a schematic diagram of the structure of a data processing device provided in an embodiment of this application.

[0082] Icons: 100 - Electronic device; 111 - Memory; 112 - Memory controller; 113 - Processor; 114 - Peripheral interface; 115 - Input / output unit; 116 - Display unit; 500 - Data processing device; 510 - Type unit; 520 - Determining unit; 530 - Calculation unit. Detailed Implementation

[0083] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0084] Current hardware circuits that integrate multiplication and accumulation operations typically only implement addition and multiplication, and can only process and output data of specific data types. The applicant has found that in neural network computing applications, due to the diverse data types of input and output data streams—such as data streams of different bit sizes, signed or unsigned numbers—existing integrated multiplication and accumulation algorithms cannot process these different input data types, nor can they limit the data type of the output data stream. Furthermore, because current integrated multiplication and accumulation operation modules only have simple addition and multiplication functions, they cannot perform corresponding vector operations such as left and right shifts, resulting in a large number of instructions, high hardware resource consumption, significant area constraints in hardware circuit design, high computational cost, and low efficiency.

[0085] To address the aforementioned issues, this application provides a data processing method applied to a server. The server can be an electronic device with logical computing capabilities, such as a personal computer (PC), tablet computer, smartphone, or personal digital assistant (PDA). This method enables the implementation of corresponding operations for different data streams, reducing hardware resources required for computation and improving computational efficiency.

[0086] Optionally, please refer to Figure 1, which is a block diagram of an electronic device provided in an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that the structure shown in Figure 1 is merely illustrative and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include more or fewer components than shown in Figure 1, or have a different configuration than that shown in Figure 1.

[0087] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0088] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs. After receiving execution instructions, the processor 113 executes the programs. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0089] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0090] Optionally, the processor 113 can be equipped with corresponding hardware circuits, which include multiple arithmetic modules with different operational functions, such as product modules, accumulation modules, addition modules, constant modules, shift modules, channel modules, and truncation modules. Each arithmetic module has corresponding input / output ports and complete connection methods. For example, the product module, accumulation module, addition module, constant module, and shift module are connected sequentially. Furthermore, due to the diversity of the number of channels in the clock cycle of the data stream, multiple channel modules can also be set up, with the first channel module connected to the shift module, and subsequent channel modules connected sequentially to the preceding channel module. Due to the diversity of output data types, multiple truncation modules can also be set up, and these modules can be connected to corresponding channel modules or shift modules according to their specific output data types.

[0091] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0092] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 can be, but is not limited to, a mouse and keyboard, etc.

[0093] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing. In this embodiment, the display unit 116 can display the input data stream to be input and the output target data stream.

[0094] The electronic device in this embodiment can be used to execute various steps in the data processing methods provided in the embodiments of this application. The implementation process of the data processing methods is described in detail below through several embodiments.

[0095] Please refer to Figure 2, which is a flowchart illustrating a data processing method provided in an embodiment of this application. The method may include steps S200-S400.

[0096] Step S200: Determine the input data stream of the corresponding input data type.

[0097] In order to realize the related calculations of fusion multiplication, accumulation, and addition, the input data stream may include three or more data streams. This application only describes three data streams. The calculation method of other numbers of data streams is the same as that of three data streams and is also within the scope of protection of this application. Therefore, the input data stream may include: a first input data stream, and / or a second input data stream, and / or a third input data stream, that is, the input data stream may include any number of input data streams, such as single, dual, or triple streams.

[0098] For example, the three input data streams can be denoted as a, b, and c, respectively.

[0099] Before performing calculations, the data types of the three data streams can be set accordingly. If the input data stream includes either a first or a second input data stream, the first input data type of the first and second input data streams is determined; if the input data stream includes a third input data stream, the second input data type of the third input data stream is determined. By limiting the input data types, subsequent unified calculations and processing can be performed, thereby improving the efficiency of the calculation.

[0100] Optionally, the first input data type and the second input data type can be the same or different. Furthermore, the second input data type can be determined based on the actual situation of the first input data type. For example, if the first input data type of the first or second input data stream is u8 (8-bit unsigned number) or s8 (8-bit signed number), then the second input data type of the third input data stream can be u8 / s8, or u16 (16-bit unsigned number) or s16 (16-bit signed number). If the data types of the first two channels are u16 or s16, the data type of the third channel can be u16 or s16, or u32 (32-bit unsigned number) or s32 (32-bit signed number).

[0101] Step S300: Determine the target operation method from multiple operation modules of the hardware circuit based on the number of data streams to be input.

[0102] In order to perform a variety of different operations, multiple operation modules with different functions can be set in the hardware circuit. Based on the actual number of data streams to be input, the appropriate module can be selected from the multiple operation modules to form the corresponding target operation method.

[0103] It should be noted that the multiple operation modules of the hardware circuit can include various modules with different functions, such as a product module (mult module), an accumulation module (reduce module), an addition module (add module), a constant module (bias module), a shift module (shift round module), a channel module (narrow module), and a clip module (clip module). These modules can respectively implement operations such as product, accumulation, addition, adding a constant, and shifting. They can also process and truncate the channels of various input and output data types during clock cycles. Each operation module has corresponding input and output ports and complete connection methods. For example, the product module, accumulation module, addition module, constant module, and shift module are connected sequentially. Furthermore, due to the diversity of the number of channels in the data stream clock cycle, multiple channel modules can be set up. The first channel module is connected to the shift module, and subsequent channel modules are connected to the preceding channel module in sequence. Due to the diversity of output data types, multiple clipping modules can also be set up. These multiple clipping modules can be connected to the corresponding channel modules or shift modules according to their specific output data types.

[0104] Step S400: Calculate the input data stream based on the target operation method to obtain the target data stream of the output data type.

[0105] The acquired target data stream can serve as a computational factor for the neural network. Depending on the desired output results, corresponding output data types can be set. For example, six data types—u8 / s8, u16 / s16, and u32 / s32—can be selected as output data types. During computation, the input data stream can be calculated accordingly based on the target computation method, and the appropriate path can be selected for output to obtain the target data stream. For instance, when the output data type format is u16, the target data stream output data type is u16, and the final output can be a 256-bit data stream (typically 256 bits is chosen as the AXI bus width).

[0106] In the embodiment shown in Figure 2, the resources of multiple computing modules in the hardware circuit can be reused to process data of different quantities, different input data types, and different output data types, thereby realizing a variety of different computing functions, improving the accuracy of calculation, reducing instruction operations during processing, improving the efficiency of hardware circuit compilation, and reducing the area pressure during hardware circuit design, thereby reducing the area of ​​the hardware circuit and the cost of calculation, and improving the efficiency of calculation.

[0107] Optionally, please refer to Figure 3, which is a detailed flowchart of step S300 provided in an embodiment of this application. Step S300 may also include steps S310-S340.

[0108] Step S310: Determine the number of data streams in the data stream to be input.

[0109] In actual computing scenarios, the number of data streams input into the hardware circuit for computation is uncertain. Therefore, in order to process different numbers of data streams, the number of data streams to be input can be determined first based on whether each input data stream is enabled.

[0110] For example, if the input data stream includes a first input data stream, a second input data stream, or a third input data stream, then the number of data streams is 1; if the input data stream includes a first input data stream and a second input data stream, or a first input data stream and a third input data stream, or a second input data stream and a third input data stream, then the number of data streams is 2; if the input data stream includes a first input data stream, a second input data stream, and a third input data stream, then the number of data streams is 3.

[0111] Step S320: If the number of data streams is 1, determine the single-path operation method as the target operation method among multiple operation modules.

[0112] When only one data stream is enabled, the single-path computation method for computing the single data stream is determined among multiple computation modules.

[0113] Step S330: If the number of data streams is 2, determine the dual-path operation method as the target operation method among multiple operation modules.

[0114] When two data streams are enabled, the dual-processing method for computing the two data streams is determined among multiple processing modules.

[0115] Step S340: If the number of data streams is 3, determine the three-way operation method as the target operation method among multiple operation modules;

[0116] When all three data streams are enabled, the three-way operation method for calculating the three data streams is determined among multiple operation modules.

[0117] In the embodiment shown in Figure 3, by setting up different functional operation modules in the hardware circuit, a variety of different operations and corresponding functions can be realized, thereby enabling various types of calculations and processing for different amounts of data streams, improving the versatility and accuracy of the calculation.

[0118] Optionally, please refer to Figure 4, which is a detailed flowchart of the first step S400 provided in the embodiment of this application. If the target operation method is a single-path operation method, step S400 may also include steps S411-S415.

[0119] Step S411: Calculate the input data stream according to the accumulation module to obtain the accumulated data.

[0120] When there is only one of the input data streams a, b, and c, a single data stream cannot be multiplied with other data streams. Therefore, the single data stream can be directly accumulated by the accumulation module, and a signed number can be output to obtain the corresponding accumulated data. The accumulated data can be denoted as ∑a, ∑b, or ∑c.

[0121] For example, the first input data stream a can be s8 / u8 (32 channels of data input per clock cycle), or a can be u16 / s16 (16 channels of data input per clock cycle); the second input data stream b can be s8 / u8 (32 channels of data input per clock cycle), or b can be u16 / s16 (16 channels of data input per clock cycle); the third input data stream c can be s8 / u8 (32 channels of data input per clock cycle), or c can be u16 / s16 (16 channels of data input per clock cycle).

[0122] Optionally, the accumulation module can support the accumulation of m input numbers, where m can be a minimum of 1 and a maximum of 1024.

[0123] Step S412: Based on the addition module and the constant module, perform addition calculations on the accumulated data and the preset constant to obtain the addition data.

[0124] The accumulated data can be added to a preset constant bias using the addition module and the constant module to obtain the corresponding addition data, which can be denoted as ∑a+bias, ∑b+bias, or ∑c+bias.

[0125] Step S413: According to the shift module, the addition data is shifted and the value is rounded to obtain the shifted data.

[0126] The shifting module can shift the addition data according to the needs of the calculation, and round the shifted data to obtain the corresponding shifted data. The shifted data can be denoted as (∑a+bias)>>n, (∑b+bias)>>n, or (∑c+bias)>>n.

[0127] Step S414: Based on the channel module, process the number of channels in the clock cycle of the shifted data to obtain the transmitted data.

[0128] Since the data types of shifted data are different, the number of channels in the corresponding clock cycles is also different. Therefore, a corresponding number of channel modules can be set to process some shifted data with a large number of channels, thereby obtaining the transmitted data.

[0129] Optionally, the number of channels can be calculated based on the AXI bus width and data type. For example, when the bus width is 256 and the data type has 8 bits, the number of channels is 256 / 8, which is 32. When the data type has 16 bits, the number of channels is 256 / 16, which is 16.

[0130] Step S415: Select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmitted data and obtain the target data stream.

[0131] Due to the limitations of the output data type, a corresponding target truncation module can be selected from multiple truncation modules for processing. This allows data exceeding the upper bound to be truncated to the upper bound, and values ​​below the lower bound to be truncated to the lower bound, outputting the target data stream of the corresponding output data type. The target data stream can be denoted as clip((∑a+bias)>>n, upper, lower), clip((∑b+bias)>>n, upper, lower), or clip((∑c+bias)>>n, upper, lower).

[0132] In the embodiment shown in Figure 4, the single-channel input data stream can be calculated and processed according to requirements, thereby improving the calculation efficiency and accuracy of the single-channel input data stream.

[0133] Optionally, please refer to Figure 5, which is a detailed flowchart of the second step S400 provided in the embodiment of this application. If the target operation method is a dual-path operation method and the input data stream includes a first input data stream and a second input data stream, then step S400 may also include steps S421-S426.

[0134] Step S421: According to the product module, the first input data stream and the second input data stream are multiplied to obtain the product data.

[0135] Since the first input data stream and the second input data stream are two data streams with the same set input data type, the two data streams can be multiplied to obtain the corresponding product data, which can be denoted as a*b.

[0136] For example, the input data types of the first input data stream a and the second input data stream b need to be the same. They can both be s8 / u8 (32 channels of data input per clock cycle) or u16 / s16 (16 channels of data input per clock cycle).

[0137] Step S422: Calculate the product data using the accumulation module to obtain the accumulated data.

[0138] The accumulation module can accumulate the product data of the two data streams and output a signed number to obtain the corresponding accumulated data, which can be denoted as ∑(a*b).

[0139] Step S423: Based on the addition module and the constant module, perform addition calculation on the accumulated data and the preset constant to obtain the addition data.

[0140] The accumulated data can be added to the preset constant bias using the addition module and the constant module to obtain the corresponding addition data, which can be denoted as ∑(a*b)+bias.

[0141] Step S424: According to the shift module, the addition data is shifted and rounded to obtain shifted data.

[0142] The shift module can shift the addition data according to the needs of the calculation, and round the shifted data to obtain the corresponding shifted data, which can be denoted as (∑(a*b)+bias)>>n.

[0143] Step S425: Based on the channel module, process the number of channels in the clock cycle of the shifted data to obtain the transmitted data.

[0144] Since the data types of shifted data are different, the number of channels in the corresponding clock cycles is also different. Therefore, a corresponding number of channel modules can be set to process some shifted data with a large number of channels, thereby obtaining the transmitted data.

[0145] Step S426: Select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmitted data and obtain the target data stream.

[0146] Due to the limitations of the output data type, a corresponding target truncation module can be selected from multiple truncation modules for processing. This allows data exceeding the upper bound to be truncated to the upper bound, and values ​​below the lower bound to be truncated to the lower bound, outputting the target data stream of the corresponding output data type. The target data stream can be denoted as clip((∑(a*b)+bias)>>n, upper, lower).

[0147] In the embodiment shown in Figure 5, two input data streams with the same input data type can be calculated and processed according to requirements, thereby improving the calculation efficiency and accuracy of the dual input data streams.

[0148] Optionally, please refer to Figure 6, which is a detailed flowchart of the third step S400 provided in the embodiment of this application. If the target operation method is a dual-path operation method and the input data stream includes a first input data stream or a second input data stream and a third input data stream, then step S400 may also include steps S431-S435.

[0149] Step S431: According to the accumulation module, calculate the first input data stream or the second input data stream and the third input data stream to obtain the accumulated data.

[0150] In the case where two input data streams are enabled, and the two input data streams are either a and c, or b and c, the input data types of the two input data streams may not be the same. Therefore, the two data streams can be directly accumulated by the accumulation module, and a signed number can be output to obtain the corresponding accumulated data. The accumulated data can be denoted as ∑a+c or ∑b+c.

[0151] For example, when the data streams are a and c, if the data type of the first input data stream a is s8 / u8 (32 channels of data are input per clock cycle), then the second input data stream c can be either s8 / u8 (32 channels of data are input per clock cycle) or u16 / s16 (16 channels of data are input per clock cycle); if the data type of a is s16 / u16 (16 channels of data are input per clock cycle), then c can be either u16 / s16 (16 channels of data are input per clock cycle) or u32 / s32 (4 channels of data are input per clock cycle). When the data streams are b and c, if the data type of the second input data stream b is s8 / u8 (32 channels of data are input per clock cycle), then the third input data stream c can be either s8 / u8 (32 channels of data are input per clock cycle) or u16 / s16 (16 channels of data are input per clock cycle); if the data type of b is s16 / u16 (16 channels of data are input per clock cycle), then c can be either u16 / s16 (16 channels of data are input per clock cycle) or u32 / s32 (4 channels of data are input per clock cycle).

[0152] Step S432: Based on the addition module and the constant module, perform addition calculation on the accumulated data and the preset constant to obtain the addition data.

[0153] The accumulated data can be added to the preset constant bias using the addition module and the constant module to obtain the corresponding addition data, which can be denoted as ∑a+c+bias or ∑b+c+bias.

[0154] Step S433: According to the shift module, the addition data is shifted and rounded to obtain shifted data.

[0155] The shift module can shift the addition data according to the needs of the calculation, and round the shifted data to obtain the corresponding shifted data. The shifted data can be denoted as (∑a+c+bias)>>n or (∑b+c+bias)>>n.

[0156] Step S434: Based on the channel module, process the number of channels in the clock cycle of the shifted data to obtain the transmitted data.

[0157] Since the data types of shifted data are different, the number of channels in the corresponding clock cycles is also different. Therefore, a corresponding number of channel modules can be set to process some shifted data with a large number of channels, thereby obtaining the transmitted data.

[0158] Step S435: Select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmitted data and obtain the target data stream.

[0159] Due to the limitations of the output data type, a corresponding target truncation module can be selected from multiple truncation modules for processing. This allows data exceeding the upper bound to be truncated to the upper bound, and values ​​below the lower bound to be truncated to the lower bound, outputting the target data stream of the corresponding output data type. The target data stream can be denoted as clip((∑a+c+bias)>>n, upper, lower) or clip((∑b+c+bias)>>n, upper, lower).

[0160] In the embodiment shown in Figure 6, two input data streams with different input data types can be calculated and processed according to requirements, thereby improving the calculation efficiency and accuracy of the two input data streams.

[0161] Optionally, please refer to Figure 7, which is a detailed flowchart of the fourth step S400 provided in the embodiment of this application. If the target operation method is a three-way operation method, then step S400 may also include steps S441-S446.

[0162] Step S441: According to the product module, the first input data stream and the second input data stream are multiplied to obtain the product data.

[0163] When all three input data streams are enabled, since the first and second input data streams are two data streams with the same set input data type, the two data streams can be multiplied first to obtain the corresponding product data, which can be denoted as a*b.

[0164] For example, the data types of the first input data stream 'a' and the second input data stream 'b' need to be the same, for example, both can be s8 / u8 (32 channels of data input per clock cycle) or u16 / s16 (16 channels of data input per clock cycle). When 'a' and 'b' are s8 / u8, the third input data stream 'c' can be either s8 / u8 (32 channels of data input per clock cycle) or u16 / s16 (16 channels of data input per clock cycle). When 'a' and 'b' are s16 / u16, 'c' can be either u16 / s16 (16 channels of data input per clock cycle) or u32 / s32 (4 channels of data input per clock cycle). Taking the data types of 'a' and 'b' as u8 and the data type of 'c' as s16 as an example, there are 32 channels per clock cycle, meaning the hardware circuit can perform parallel calculations of 32 'a' and 'b' values ​​in one clock cycle, and the data type of the output product data is u16 (32 channels).

[0165] Step S442: Calculate the product data using the accumulation module to obtain the accumulated data.

[0166] The accumulation module can perform accumulation calculations on the product data of the two data streams and output a signed number to obtain the corresponding accumulated data, which can be denoted as ∑(a*b). In this embodiment, the data type of the accumulated data output by the accumulation module can be s27 (27-bit signed number) with 32 channels per clock cycle, or s43 (43-bit signed number) with 16 channels per clock cycle.

[0167] Step S443: According to the addition module, perform addition calculation on the accumulated data, the third input data stream and the preset constant to obtain the addition data.

[0168] Specifically, the addition module adds the accumulated data and the third input data stream to output ∑(a*b)+c. Then, the constant module adds ∑(a*b)+c to a preset constant bias to obtain the corresponding addition data, which can be denoted as ∑(a*b)+c+bias. Optionally, the data type of ∑(a*b)+c output by the addition module can be s28 (28-bit signed number), 32 channels; or s44 (44-bit signed number), 16 channels. The data type of the addition data output by the constant module can be s29 (29-bit signed number), 32 channels; or s45 (45-bit signed number), 16 channels.

[0169] Step S444: According to the shift module, the addition data is shifted and rounded to obtain shifted data.

[0170] The shift module can shift the addition data according to the needs of the calculation and round the shifted data to obtain the corresponding shifted data, which can be denoted as (∑(a*b)+c+bias)>>n. The output shifted data is of data type s29 (29-bit signed number), with 32 channels, and is the same as the input data type.

[0171] Step S445: Based on the channel module, process the number of channels in the clock cycle of the shifted data to obtain the transmitted data.

[0172] Since the data types of shifted data are different, the number of channels in the corresponding clock cycles is also different. Therefore, a corresponding number of channel modules can be set to process some shifted data with a large number of channels, thereby obtaining the transmitted data.

[0173] Step S446: Select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmitted data and obtain the target data stream.

[0174] Due to the limitations of the output data type, a corresponding target truncation module can be selected from multiple truncation modules for processing. This allows data exceeding the upper bound to be truncated to the upper bound, and values ​​below the lower bound to be truncated to the lower bound, outputting the target data stream of the corresponding output data type. The target data stream can be denoted as clip((∑(a*b)+c+bias)>>n, upper, lower).

[0175] In the embodiment shown in Figure 7, the three input data streams can be calculated and processed accordingly as needed, thereby improving the calculation efficiency and accuracy of the three input data streams.

[0176] Optionally, please refer to Figure 8, which is a detailed flowchart of steps 414, S425, S434 or S445 provided in an embodiment of this application. Steps 414, S425, S434 or S445 may also include steps S451-S454.

[0177] Step S451: Determine the maximum channel value of the clock cycle based on the bus width.

[0178] When the channel module of the hardware circuit processes the shifted data, the maximum channel value of the clock cycle of the data can be determined according to the bus width. For example, when the bus width is 256, since the smallest bit of data in the data stream is 8 bits, the maximum channel value of the clock cycle is 32 channels.

[0179] Step S452: Determine the channel threshold of the channel module based on the maximum channel value.

[0180] One approach is to halve the maximum channel value to determine the channel threshold in the channel module. For example, when the maximum channel value is 32 channels, the corresponding channel threshold is 16 channels.

[0181] Step S453: If the number of channels exceeds the channel threshold, the shifted data is processed to reduce parallelism based on the channel module to obtain the transmitted data;

[0182] When the number of channels for shifted data exceeds the channel threshold, the shifted data can be processed with reduced parallelism based on the channel module, so that the clock cycle of the shifted data is divided into two clock cycles for output, and the corresponding transmission data is obtained.

[0183] Step S454: If the number of channels does not exceed the channel threshold, the shifted data is transparently transmitted based on the channel module to obtain the transmitted data.

[0184] When the number of channels for shifted data does not exceed the channel threshold, the shifted data can be directly transmitted to the truncation module for processing as transmission data.

[0185] It should be noted that multiple channel modules and truncation modules can be set up to process the data. The function of the channel module is determined by the number of channels in the input data. For example, when the input is s29 (32 channels), it exceeds the channel threshold (16 channels). Therefore, it is necessary to reduce the parallelism of the data, that is, the input of one clock cycle is divided into two clock cycles and output separately, with the value remaining unchanged. The data type of the output is s29 (16-bit signed number), with 16 channels. When the input is s45 (16 channels), there is no need to reduce the parallelism, and it is directly passed through. Taking the embodiment shown in Figure 7 as an example, since the shifted data of the input channel module is s29 (32 channels), it is necessary to reduce the parallelism, and the output transmission data is s29 (16 channels). After the first channel module, a second channel module can also be set up to continue processing the transmission data. The channel threshold of the second channel module can be half of the threshold of the first channel module, that is, 8 channels. Therefore, the second channel module can perform the reduction of parallelism operation for all types of input. Furthermore, the appropriate channel module and truncation module can be selected for processing based on the output data type. Taking the embodiment shown in Figure 7 as an example, since the output data type is set to u16, the data stream does not pass through the second channel module. Multiple truncation modules, such as clip0, clip1, and clip2, can all perform truncation operations. However, the data bit width corresponding to each truncation module, i.e., the output data type, is different. For example, the clip0 module outputs s32 / u32 (8 channels), and the corresponding upper and lower are also s32 / u32; the clip1 module outputs s16 / u16 (16 channels), and the corresponding upper and lower are also s16 / u16; the clip2 module outputs s8 / u8 (8 channels), and the corresponding upper and lower are also s8 / u8, etc.

[0186] In the embodiment shown in Figure 8, it is possible to reduce the parallelism of data with a large number of channels, and it can also perform multi-level processing through multiple channel modules so that the final output data can meet the preset output data type. It is suitable for processing data of various different input data types and meeting the requirements of various different output data types.

[0187] Optionally, please refer to Figure 9, which is a detailed flowchart of step 413, step S424, step S433 or step S444 provided in an embodiment of this application. Step 413, step S424, step S433 or step S444 may also include steps S461-S463.

[0188] Step S461: Determine the shift direction and shift degree of the addition data.

[0189] When shifting data, the direction and extent of the shift can be determined according to the requirements. You can choose to shift left or right, and select the number of bits to shift, such as shifting right by 2 bits.

[0190] Step S462: According to the shift module, the addition data is shifted based on the shift direction and shift degree to obtain the initial shift data.

[0191] Specifically, the addition data is shifted according to the corresponding shift direction and shift degree to obtain the corresponding initial shift data.

[0192] Step S463: Select the target numerical rounding algorithm from multiple numerical rounding algorithms to perform numerical rounding on the initial shifted data to obtain the shifted data.

[0193] The available numerical rounding algorithms can include various options, such as directly discarding the least significant bit, carrying towards positive infinity (half_up), carrying towards negative infinity (half_down), carrying towards zero (half_towards_zero), carrying away from zero (half_away_from_zero), carrying towards even numbers (half_to_even), and carrying towards odd numbers (half_to_odd), where n can range from 0 to 15. Depending on the computational requirements, a suitable target numerical rounding algorithm can be selected from these options. For example, choosing the rounding method that carries towards even numbers allows for numerical rounding calculations on the initial shifted data to obtain the corresponding shifted data.

[0194] In the embodiment shown in Figure 9, numerical overflow is not considered, and only one numerical rounding is performed, which can ensure the accuracy of calculation while realizing the operation functions of multiple vector operations.

[0195] Please refer to Figure 10, which is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. The data processing device 500 may include: a type unit 510, a determination unit 520, and a calculation unit 530.

[0196] Type unit 510 is used to determine the input data stream of the corresponding input data type;

[0197] The determining unit 520 is used to determine the target operation method among multiple operation modules of the hardware circuit based on the number of data streams to be input;

[0198] The computing unit 530 is used to perform calculations on the input data stream based on the target operation method to obtain the target data stream of the output data type, and uses the target data stream as the operation factor of the neural network.

[0199] In an optional implementation, the input data stream includes: a first input data stream, and / or a second input data stream, and / or a third input data stream; the type unit 510 is specifically used to: if the input data stream includes the first input data stream or the second input data stream, determine the first input data type of the first input data stream and the second input data stream; if the input data stream includes the third input data stream, determine the second input data type of the third input data stream.

[0200] In an optional implementation, the determining unit 520 is specifically used to: determine the number of data streams in the input data stream; wherein, if the input data stream includes a first input data stream, a second input data stream, or a third input data stream, the number of data streams is 1; if the input data stream includes a first input data stream and a second input data stream, or a first input data stream and a third input data stream, or a second input data stream and a third input data stream, the number of data streams is 2; if the input data stream includes a first input data stream, a second input data stream, and a third input data stream, the number of data streams is 3; if the number of data streams is 1, a single-path operation method is determined as the target operation method among multiple operation modules; if the number of data streams is 2, a dual-path operation method is determined as the target operation method among multiple operation modules; if the number of data streams is 3, a three-path operation method is determined as the target operation method among multiple operation modules; wherein, the multiple operation modules of the hardware circuit include a product module, an accumulation module, an addition module, a constant module, a shift module, a channel module, and a truncation module.

[0201] In an optional implementation, if the target operation method is a single-channel operation method, the calculation unit 530 may further include a single-channel calculation subunit, which is used to calculate the input data stream according to the accumulation module to obtain accumulated data; to perform addition calculation on the accumulated data and a preset constant according to the addition module and the constant module to obtain added data; to shift and round the added data according to the shift module to obtain shifted data; to process the number of channels of the clock cycle of the shifted data according to the channel module to obtain transmitted data; and to select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmitted data to obtain the target data stream.

[0202] In an optional implementation, if the target computation method is a dual-channel computation method, and the input data stream includes a first input data stream and a second input data stream, the computation unit 530 may further include a first dual-channel computation subunit, used to perform product processing on the first input data stream and the second input data stream according to the product module to obtain product data; calculate the product data according to the accumulation module to obtain accumulated data; perform addition calculation on the accumulated data and a preset constant according to the addition module and the constant module to obtain addition data; shift and round the addition data according to the shift module to obtain shifted data; process the number of channels in the clock cycle of the shifted data according to the channel module to obtain transmission data; and select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmission data to obtain the target data stream.

[0203] In an optional implementation, if the target computation method is a dual-channel computation method, and the input data stream includes a first input data stream or a second input data stream, and a third input data stream, the computation unit 530 may further include a second dual-channel computation subunit, used to perform calculations on the first input data stream or the second input data stream, and the third input data stream according to the accumulation module to obtain accumulated data; perform addition calculations on the accumulated data and a preset constant according to the addition module and the constant module to obtain added data; perform shift and numerical reduction on the added data according to the shift module to obtain shifted data; process the number of channels in the clock cycle of the shifted data according to the channel module to obtain transmitted data; and select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmitted data to obtain the target data stream.

[0204] In an optional implementation, if the target operation method is a three-way operation method, the calculation unit 530 may further include a three-way calculation subunit, used to: multiply the first input data stream and the second input data stream according to the product module to obtain product data; calculate the product data according to the accumulation module to obtain accumulated data; perform addition calculation on the accumulated data, the third input data stream, and a preset constant according to the addition module to obtain addition data; shift and round the addition data according to the shift module to obtain shifted data; process the number of channels in the clock cycle of the shifted data according to the channel module to obtain transmission data; and select the corresponding target truncation module from multiple truncation modules according to the output data type to process the transmission data to obtain the target data stream.

[0205] In an optional implementation, the calculation unit 530 is specifically used to: determine the maximum channel value of the clock cycle based on the bus width; determine the channel threshold of the channel module based on the maximum channel value; if the number of channels exceeds the channel threshold, perform parallelism reduction processing on the shifted data based on the channel module to obtain the transmission data; if the number of channels does not exceed the channel threshold, perform transparent transmission of the shifted data based on the channel module to obtain the transmission data.

[0206] In an optional implementation, the calculation unit 530 is specifically used to: determine the shift direction and shift degree of the addition data; shift the addition data based on the shift direction and shift degree according to the shift module to obtain initial shift data; select a target numerical rounding algorithm from multiple numerical rounding algorithms to perform numerical rounding on the initial shift data to obtain shift data.

[0207] Since the principle of the data processing device 500 in this embodiment of the application is similar to that of the aforementioned data processing method embodiment, the implementation of the data processing device 500 in this embodiment can refer to the description in the above-mentioned data processing method embodiment, and the repeated parts will not be described again.

[0208] This application also provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, they perform the steps of any of the data processing methods provided in this embodiment.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0210] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0211] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0213] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0214] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A data processing method, characterized in that, The method includes: determining a data stream to be input of a corresponding input data type; determining a target operation method in multiple operation modules of a hardware circuit based on the number of data streams in the data stream to be input; calculating the data stream to be input based on the target operation method to obtain a target data stream of an output data type, using the target data stream as an operation factor of a neural network; wherein, the data stream to be input includes: a first input data stream, and / or a second input data stream, and / or a third input data stream; determining the data stream to be input of a corresponding input data type includes: if the data stream to be input includes the first input data stream or the second input data stream, determining a first input data type of the first input data stream and the second input data stream; if the data stream to be input includes the third input data stream, determining a second input data type of the third input data stream; determining the target operation method in multiple operation modules of a hardware circuit based on the number of data streams in the data stream to be input includes: determining the number of data streams in the data stream to be input; wherein If the input data stream includes the first input data stream, the second input data stream, or the third input data stream, then the number of data streams is 1; if the input data stream includes the first input data stream and the second input data stream, or the first input data stream and the third input data stream, or the second input data stream and the third input data stream, then the number of data streams is 2; if the input data stream includes the first input data stream, the second input data stream, and the third input data stream, then the number of data streams is 3; if the number of data streams is 1, a single-path operation method is determined as the target operation method among the multiple operation modules; if the number of data streams is 2, a dual-path operation method is determined as the target operation method among the multiple operation modules; if the number of data streams is 3, a three-path operation method is determined as the target operation method among the multiple operation modules; wherein, the multiple operation modules of the hardware circuit include a product module, an accumulation module, an addition module, a constant module, a shift module, a channel module, and a truncation module.

2. The method according to claim 1, characterized in that, If the target computation method is the single-channel computation method, then the step of calculating the input data stream based on the target computation method to obtain the target data stream of the output data type includes: calculating the input data stream according to the accumulation module to obtain accumulated data; performing addition calculation on the accumulated data and a preset constant according to the addition module and the constant module to obtain added data; shifting and rounding the added data according to the shift module to obtain shifted data; processing the number of channels in the clock cycle of the shifted data according to the channel module to obtain transmitted data; and selecting a corresponding target truncation module from among the multiple truncation modules according to the output data type to process the transmitted data to obtain the target data stream.

3. The method according to claim 1, characterized in that, If the target operation method is the dual-path operation method, and the input data stream includes the first input data stream and the second input data stream; The step of calculating the input data stream based on the target operation method to obtain the target data stream of the output data type includes: performing product processing on the first input data stream and the second input data stream according to the product module to obtain product data; The accumulation module calculates the product data to obtain accumulated data; the addition module and the constant module perform addition calculations on the accumulated data and a preset constant to obtain added data; the shift module shifts and rounds the added data to obtain shifted data; the channel module processes the number of channels in the clock cycle of the shifted data to obtain transmitted data; and the corresponding target truncation module is selected from multiple truncation modules according to the output data type to process the transmitted data and obtain the target data stream.

4. The method according to claim 1, characterized in that, If the target operation method is the dual-path operation method, and the input data stream includes the first input data stream or the second input data stream, and the third input data stream; The step of calculating the input data stream based on the target operation method to obtain the target data stream of the output data type includes: calculating the first input data stream, the second input data stream, and the third input data stream according to the accumulation module to obtain accumulated data; performing addition calculation on the accumulated data and a preset constant according to the addition module and the constant module to obtain added data; shifting and rounding the added data according to the shift module to obtain shifted data; processing the number of channels in the clock cycle of the shifted data according to the channel module to obtain transmitted data; and selecting a corresponding target truncation module from among the multiple truncation modules according to the output data type to process the transmitted data to obtain the target data stream.

5. The method according to claim 1, characterized in that, If the target computation method is the three-way computation method, then the step of calculating the input data stream based on the target computation method to obtain the target data stream of the output data type includes: multiplying the first input data stream and the second input data stream according to the product module to obtain product data; calculating the product data according to the accumulation module to obtain accumulated data; performing addition calculation on the accumulated data, the third input data stream, and a preset constant according to the addition module to obtain addition data; shifting and rounding the addition data according to the shift module to obtain shifted data; processing the number of channels in the clock cycle of the shifted data according to the channel module to obtain transmission data; and selecting a corresponding target truncation module from among the multiple truncation modules according to the output data type to process the transmission data to obtain the target data stream.

6. The method according to any one of claims 2-5, characterized in that, The step of processing the number of channels in the clock cycle of the shifted data according to the channel module to obtain the transmission data includes: determining the maximum channel value in the clock cycle based on the bus width; determining the channel threshold of the channel module based on the maximum channel value; if the number of channels exceeds the channel threshold, performing parallelism reduction processing on the shifted data based on the channel module to obtain the transmission data; if the number of channels does not exceed the channel threshold, performing pass-through transmission on the shifted data based on the channel module to obtain the transmission data.

7. The method according to any one of claims 2-5, characterized in that, The step of shifting and rounding the addition data according to the shift module to obtain shifted data includes: determining the shift direction and shift degree of the addition data; shifting the addition data based on the shift direction and shift degree according to the shift module to obtain initial shifted data; and selecting a target numerical rounding algorithm from multiple numerical rounding algorithms to perform numerical rounding on the initial shifted data to obtain the shifted data.

8. A data processing apparatus, characterized in that, The device includes: a type unit, a determination unit, and a calculation unit; the type unit is used to determine the input data stream of a corresponding input data type; the determination unit is used to determine a target operation method among multiple operation modules of the hardware circuit according to the number of data streams in the input data stream; the calculation unit is used to perform calculations on the input data stream based on the target operation method to obtain a target data stream of the output data type, using the target data stream as an operation factor of the neural network; wherein, the input data stream includes: a first input data stream, and / or a second input data stream, and / or a third input data stream; the type unit is specifically used to: if the input data stream includes the first input data stream or the second input data stream, determine the first input data type of the first input data stream and the second input data stream; if the input data stream includes the third input data stream, determine the second input data type of the third input data stream; the determination unit is specifically used to: determine the number of data streams in the input data stream; wherein, if the number of data streams in ... number of data streams in the input data stream includes the second input data stream, determine the second input data type of the third input data stream; the determination unit is specifically used to: determine the number of data streams in the input data stream; wherein, if the number of data streams in the input data stream includes the second input data stream, determine the second input data type of the third input data stream; if the number of data streams in the input data stream includes the second input data stream, determine the second input data type of the third input data stream; if the number of data streams in the input data stream includes the second input data If the input data stream includes the first input data stream, the second input data stream, or the third input data stream, then the number of data streams is 1; if the input data stream includes the first input data stream and the second input data stream, or the first input data stream and the third input data stream, or the second input data stream and the third input data stream, then the number of data streams is 2; if the input data stream includes the first input data stream, the second input data stream, and the third input data stream, then the number of data streams is 3; if the number of data streams is 1, a single-path operation method is determined as the target operation method among the multiple operation modules; if the number of data streams is 2, a dual-path operation method is determined as the target operation method among the multiple operation modules; if the number of data streams is 3, a three-path operation method is determined as the target operation method among the multiple operation modules; wherein, the multiple operation modules of the hardware circuit include a product module, an accumulation module, an addition module, a constant module, a shift module, a channel module, and a truncation module.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores program instructions, and when the processor executes the program instructions, it performs the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores computer program instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1-7.

Images