Data processing method and device and electronic equipment

Abstract

The invention provides a data processing method and device and electronic equipment. The data processing method comprises the following steps: acquiring a plurality of convolution kernels required by convolution operation by a convolution kernel control module; for the Nth group of data in the multiple groups of data used for convolution operation, a convolution kernel control module successively sends a plurality of convolution kernels to a convolution operation module in a first sequence, and N is greater than or equal to 1; respectively carrying out convolution operation on the Nth group of data and a plurality of convolution kernels according to a first sequence by a convolution operation module; for the (N + 1) th group of data adjacent to the Nth group of data in the multiple groups of data, the convolution kernel control module successively sends the multiple convolution kernels to the convolution operation module in a second sequence opposite to the first sequence; and respectively performing convolution operation on the (N + 1) th group of data and the plurality of convolution kernels according to a second sequence by the convolution operation module. According to the data processing method, the node jump of the convolution operation of the convolutional neural network can be reduced, and the dynamic power consumption of the circuit is reduced.

Description

Technical Field

[0001] This disclosure belongs to the field of integrated circuit design technology, and in particular relates to data processing methods and apparatus, and electronic equipment. Background Technology

[0002] Convolutional Neural Networks (CNNs) utilize convolutional computation as their core operational unit, efficiently extracting spatial features through local connectivity and weight sharing mechanisms. In traditional hardware implementations, convolutional layers typically rely on parallel multiply-accumulate (MAC) arrays to perform tensor operations. This process involves element-wise multiplication and accumulation of the input feature map with the convolutional kernel window, and data reuse is used to map convolution to matrix multiplication to improve parallel efficiency. Summary of the Invention

[0003] The purpose of this disclosure is to provide data processing methods, apparatus, and electronic devices to solve the problems of simultaneous input node jumps and high dynamic power consumption in the prior art during convolution operations.

[0004] In a first aspect, this disclosure provides a data processing method. The data processing method includes: obtaining multiple convolution kernels required for convolution operations by a convolution kernel control module; for the Nth group of data in a plurality of data sets used for convolution operations, the convolution kernel control module sequentially sends the multiple convolution kernels to a convolution operation module in a first order, where N≥1; the convolution operation module performs convolution operations on the Nth group of data and the multiple convolution kernels respectively in the first order; for the (N+1)th group of data adjacent to the Nth group of data in the plurality of data sets, the convolution kernel control module sequentially sends the multiple convolution kernels to the convolution operation module in a second order opposite to the first order; and the convolution operation module performs convolution operations on the (N+1)th group of data and the multiple convolution kernels respectively in the second order.

[0005] In one implementation of the first aspect, obtaining multiple convolution kernels required for the convolution operation by the convolution kernel control module includes: obtaining all convolution kernels required for the convolution operation by the convolution kernel control module and storing all convolution kernels in the convolution kernel control module.

[0006] In one implementation of the first aspect, the convolution kernel control module sending the plurality of convolution kernels sequentially to the convolution operation module in a first order includes: for a set of data to be convolved, the convolution kernel control module sending the plurality of convolution kernels to the convolution operation module in the first order according to the effective edge of the clock cycle, wherein the convolution kernel control module sending the plurality of convolution kernels sequentially to the convolution operation module in a second order opposite to the first order includes: for another set of data immediately following the first set of data, the convolution kernel control module sending the plurality of convolution kernels to the convolution operation module in the second order according to the effective edge of the clock cycle.

[0007] In one implementation of the first aspect, the convolution operation module performing convolution operations on the Nth group of data and the plurality of convolution kernels in the first order includes: the convolution operation module performing M convolution operations on the Nth group of data and M convolution kernels in the first order to generate M convolution operation values, where M represents the number of the plurality of convolution kernels; wherein the data processing method further includes: the output control module merging the M convolution operation values ​​to generate a convolution operation result for the Nth group of data.

[0008] In one implementation of the first aspect, the convolution operation module performing convolution operations on the (N+1)th group of data and the plurality of convolution kernels in the second order includes: the convolution operation module performing M convolution operations on the (N+1)th group of data and M convolution kernels in the second order to generate M convolution operation values, where M represents the number of the plurality of convolution kernels; wherein the data processing method further includes: the output control module merging the M convolution operation values ​​to generate a convolution operation result for the (N+1)th group of data.

[0009] In one implementation of the first aspect, the data processing method further includes: the output control module integrating the convolution operation results for the Nth group of data and the convolution operation results for the N+1th group of data to output the data operation result.

[0010] Secondly, this disclosure provides a data processing apparatus. The data processing apparatus includes: a data control module configured to acquire multiple sets of data for convolution operations; a convolution kernel control module configured to acquire multiple convolution kernels required for convolution operations, sequentially output the multiple convolution kernels in a first order for the Nth set of data, and sequentially output the multiple convolution kernels in a second order opposite to the first order for the (N+1)th set of data adjacent to the Nth set of data, where N≥1; and a convolution operation module configured to sequentially receive the multiple convolution kernels from the convolution kernel control module in the first order, and combine the Nth set of data with the multiple convolution kernels in the second order. A convolution operation is performed sequentially to generate multiple convolution operation values ​​for the Nth group of data. The convolution kernel control module receives the multiple convolution kernels sequentially according to a second order, and performs convolution operations between the N+1th group of data and the multiple convolution kernels in the second order to generate multiple convolution operation values ​​for the N+1th group of data. An output control module is configured to merge the multiple convolution operation values ​​for the Nth group of data to generate a convolution operation result for the Nth group of data, and to merge the multiple convolution operation values ​​for the N+1th group of data to generate a convolution operation result for the N+1th group of data.

[0011] In one implementation of the second aspect, the convolution kernel control module is configured to: acquire all convolution kernels required for the convolution operation and store all convolution kernels in the convolution kernel control module; for a set of data to be convolved, send the plurality of convolution kernels to the convolution operation module in the first order according to the effective edge of the clock cycle; and for another set of data immediately following the first set of data, send the plurality of convolution kernels to the convolution operation module in the second order according to the effective edge of the clock cycle.

[0012] In one implementation of the second aspect, the convolution operation module is configured to: receive the plurality of convolution kernels sequentially in the first order, and simultaneously perform M convolution operations on the Nth group of data and each of the received convolution kernels to generate M convolution operation values, where M represents the number of the plurality of convolution kernels; and receive the plurality of convolution kernels sequentially in the second order, and simultaneously perform M convolution operations on the (N+1)th group of data and each of the received convolution kernels to generate M convolution operation values.

[0013] In one implementation of the second aspect, the output control module is configured to: merge M convolution operation values ​​received from the convolution operation module for the Nth group of data to generate a convolution operation result for the Nth group of data; and merge M convolution operation values ​​received from the convolution operation module for the (N+1)th group of data to generate a convolution operation result for the (N+1)th group of data.

[0014] Thirdly, this disclosure provides an electronic device. The electronic device includes: a memory configured to store information associated with a convolution operation; and a processor electrically coupled to the memory and configured to perform a data processing method according to any one of the first aspects.

[0015] According to embodiments of this disclosure, the order in which the convolution kernels are sent from the convolution kernel control module to the convolution operation module is optimized. During data transitions, the convolution kernels are maintained, ensuring that only one side of the convolution calculation experiences a transition. This solution does not affect the speed of the convolution operation, does not require changes to the original convolution circuit, and only needs to change the input order of the convolution kernels. Thus, the convolution kernels remain unchanged during data transitions, reducing the power consumption of the convolutional neural network circuit. Attached Figure Description

[0016] Figure 1 This diagram illustrates the use of a convolutional neural network for convolution operations.

[0017] Figure 2A The diagram shows an environment schematic of the data processing method described in the embodiments of this disclosure.

[0018] Figure 2B This diagram illustrates the structure of the mid-cloud interaction scenario in these implementation methods.

[0019] Figure 3 The diagram shown is a flowchart illustrating the data processing method described in an embodiment of this disclosure.

[0020] Figure 4 The diagram shows a flowchart of the convolution operation described in an embodiment of this disclosure.

[0021] Figure 5 The diagram shows a flowchart of the convolution operation described in an embodiment of this disclosure.

[0022] Figure 6 The diagram shown is a schematic representation of the data processing method described in an embodiment of this disclosure.

[0023] Figure 7 The diagram shown is a structural schematic of the data processing apparatus described in an embodiment of this disclosure.

[0024] Figure 8The diagram shown is a structural schematic of the electronic device described in an embodiment of this disclosure. Detailed Implementation

[0025] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. This disclosure can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0026] It should be noted that in the embodiments of this disclosure, words such as "specifically" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "specifically" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "specifically" or "for example" is intended to present the relevant concepts in a concrete manner.

[0027] In this disclosure, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0028] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. Therefore, the drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0029] Existing data processing methods for neural network convolutional layers utilize a sufficient number of multipliers to perform multiple convolution operators simultaneously and reduce circuit power consumption by using zero-crossing techniques. However, in some multi-level convolutional neural network applications, the computational cost of convolution varies across different levels. With a fixed number of multipliers, multiple convolution operators cannot be performed simultaneously in certain levels and must be performed in separate steps. In this case, there is a problem of simultaneous jumps in data and convolution kernels. Figure 1 This is a schematic diagram illustrating convolution operations performed by a convolutional neural network. For example... Figure 1 As shown, when the computational power of a convolutional neural network circuit is insufficient to complete the computation of all convolutional kernels in one go, it needs to be processed in multiple steps. Each step performs a convolution operation on one convolutional kernel. After all convolutional kernels are completed, the same operation is performed on the next set of data. Switching data while simultaneously changing the convolutional kernels results in jumps on both sides of the convolution computation, leading to significant power consumption.

[0030] At least to address the aforementioned problems, this disclosure provides a data processing method. The data processing method includes: obtaining multiple convolution kernels required for convolution operations by a convolution kernel control module; for the Nth group of data in a plurality of data sets used for convolution operations, the convolution kernel control module sequentially sends the multiple convolution kernels to a convolution operation module in a first order, where N≥1; the convolution operation module performs convolution operations on the Nth group of data and the multiple convolution kernels respectively in the first order; for the (N+1)th group of data adjacent to the Nth group of data in the plurality of data sets, the convolution kernel control module sequentially sends the multiple convolution kernels to the convolution operation module in a second order opposite to the first order; and the convolution operation module performs convolution operations on the (N+1)th group of data and the multiple convolution kernels respectively in the second order.

[0031] In this embodiment, the order in which the convolution kernels are sent from the convolution kernel control module to the convolution operation module is optimized. The convolution kernels are maintained during data transitions, ensuring that only one side of the convolution calculation experiences a transition. This solution does not affect the speed of the convolution operation, does not require changes to the original convolution circuit, and only needs to change the input order of the convolution kernels. This ensures that the convolution kernels remain unchanged during data transitions, thereby reducing the power consumption of the convolutional neural network circuit.

[0032] Figure 2A The diagram shows an environment schematic for the data processing method described in this embodiment. Data processing system 1 can be used to implement the data processing method provided in this embodiment, but the application scenarios of the data processing method provided in this embodiment are not limited to... Figure 2A The data processing system 1 shown is as follows. Figure 2A As shown, the data processing system 1 includes a data storage device 11, a local processor 12, and a display terminal 13. The data processing method provided in this embodiment can be applied to the local processor 12 that carries out convolution operations of a convolutional neural network. In some embodiments, the local processor 12 may include a system-on-a-chip (SOC), and the SOC may include a neural network processor (NPU).

[0033] Figure 2AThe local processor 12 can be a single local processor, a cluster of multiple local processors, or a cloud computing center, etc., and is not specifically limited here. Although Figure 2A Only one data storage device 11, one local processor 12, and one display terminal 13 are shown in the diagram, but it should be understood that... Figure 2A The examples in this paper are only for understanding this solution. The specific number of local processors 12 and display terminals 13 should be flexibly determined based on the actual situation.

[0034] In some other implementations, the data processing system 1 may not include a display terminal 13, but only a local processor 12 with output function and a data storage device 11. The data processing method provided in this disclosure can be applied to the local processor 12 that carries out convolutional operations of a convolutional neural network. The local processor 12 may be included in tablet computers, PDAs, mobile phones, personal computers, and interactive devices, etc., and is not limited here.

[0035] In some other implementations, the data processing methods described in this disclosure can be applied to edge-cloud interaction scenarios. Figure 2B This diagram illustrates the structure of the endpoint-cloud interaction scenario in these implementation methods. For example... Figure 2B As shown, the end-to-cloud interactive system 2 includes a terminal 20 and a cloud server 21. The terminal 20 and the cloud server 21 can communicate with each other, and the communication method is not limited to wired or wireless. The data processing method described in this disclosure can be executed by the processor of the terminal 20 or by the processor of the cloud server 21.

[0036] Terminal 20 can be mobile or fixed. For example, terminal 20 can be a wireless terminal or a wired terminal. A wireless terminal can refer to a device with wireless transceiver capabilities, which can be deployed indoors, outdoors, and in industrial workshops. Terminal 20 can be a mobile phone, tablet computer, laptop computer, etc., and is not limited thereto. Cloud server 21 may include one or more servers, or one or more processing nodes, or one or more virtual machines running on the server. Cloud server 21 may also be referred to as a server cluster, management platform, data processing center, etc., and is not limited thereto in this embodiment.

[0037] The technical solutions in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0038] The following embodiments of this disclosure provide a data processing method, which, for example, can be implemented through... Figure 2A The local processor 12 shown or Figure 2B The cloud server 21 shown is used to implement this.

[0039] Figure 3 The diagram shown is a schematic flowchart of the data processing method described in an embodiment of this disclosure. Figure 3 As shown, the data processing method may include the following steps S11 to S15.

[0040] Step S11: The convolution kernel control module obtains multiple convolution kernels required for the convolution operation.

[0041] Step S12: For the Nth set of data in the multiple sets of data used for convolution operation, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in a first order, where N≥1.

[0042] Step S13: The convolution operation module performs convolution operations on the Nth group of data and the plurality of convolution kernels in the first order.

[0043] Step S14: For the N+1th group of data adjacent to the Nth group of data in the multiple groups of data, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in a second order that is the opposite of the first order.

[0044] Step S15: The convolution operation module performs convolution operations on the N+1th group of data and the plurality of convolution kernels in the second order.

[0045] In some possible implementations, for example, the convolution operation function of a convolutional neural network is implemented by multipliers and adders. When both sides of the input data of the convolution operation undergo a jump, it is equivalent to both sides of the input data of the multiplier undergoing a change. At this time, the flipping probability of the internal circuit nodes of the multiplier increases significantly, which also leads to a significant increase in the dynamic power consumption of the multiplier circuit. The convolution kernel control module obtains multiple convolution kernels required for the convolution operation and integrates the convolution kernels required for this operation into the convolution kernel control module. The number of convolution kernels is determined by the actual computational needs. For the first set of data in the multiple sets of data used for the convolution operation, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in a first order. The convolution operation module performs convolution operations on the first set of data and the multiple convolution kernels in the first order. For the second set of data adjacent to the first set of data, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in a second order, opposite to the first order. Sequentially, for the Nth group of data in the multiple groups of data used for convolution operations, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in a first order. For the (N+1)th group of data adjacent to the Nth group of data, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in a second order, opposite to the first order. Finally, the operation results of the first, third...Nth groups of data, processed in the first order, and the operation results of the second, fourth...N+1th groups of data, processed in the second order, opposite to the first order, are output. The N+1 group operation results are merged to output the final data operation result, where N is the number of data groups to be processed, N≥1.

[0046] It should be understood that the data control module 110, the convolution kernel control module 120, the convolution operation module 130, and the output control module 140 can also be configured to execute the corresponding steps or actions in the data processing method described in the above embodiments, which will not be repeated here.

[0047] In this embodiment, the order in which the convolution kernels are sent from the convolution kernel control module to the convolution operation module is optimized. The convolution kernels are maintained during data transitions, ensuring that only one side of the convolution calculation experiences a transition. This solution does not affect the speed of the convolution operation, does not require changes to the original convolution circuit, and only needs to change the input order of the convolution kernels. This ensures that the convolution kernels remain unchanged during data transitions, thereby reducing the power consumption of the convolutional neural network circuit.

[0048] In one embodiment of this disclosure, obtaining multiple convolution kernels required for convolution operations by the convolution kernel control module may include: obtaining all convolution kernels required for convolution operations by the convolution kernel control module and storing all convolution kernels in the convolution kernel control module.

[0049] In some possible implementations, the convolution kernel control module acquires eight convolution kernels required for the convolution operation. Therefore, a data transition occurs after every eight convolution kernels have completed their operations, and the eight convolution kernels are stored in the convolution kernel control module. After the eight convolution kernels corresponding to the first set of data complete the convolution operation in ascending order, only the input data is changed to the second set of data. The eight convolution kernels are then used in reverse order for the convolution operation of the second set of data, avoiding simultaneous data and convolution kernel transitions. Table 1 shows a power consumption comparison between commonly used convolution operation methods and the convolution operation described in this disclosure. As shown in Table 1, the total power consumption of other convolution model 0 is 860.456 uW, while the total power consumption of convolution model 1 of the data processing method described in this disclosure is 789.374 uW. Using the convolution operation module of this disclosure saves 8.2% of power consumption compared to existing convolution models.

[0050] Table 1: Power consumption comparison of different convolutional models

[0051] Convolution model Total power consumption Leakage current power consumption Dynamic power consumption Convolutional Model 0 860.465uW 1.096uW 859.369uW Convolution Model 1 789.374uW 1.096uW 788.278uW

[0052] Figure 4 The diagram shows a flowchart of the convolution operation described in an embodiment of this disclosure. Figure 4 As shown, step S12 may include steps S121 to S123.

[0053] Step S121: For a set of data to be convolved, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in the first order according to the effective edge of the clock cycle.

[0054] Step S122: For the next set of data immediately following the first set of data, the convolution kernel control module sends the plurality of convolution kernels to the convolution operation module in the second order according to the effective edge of the clock cycle.

[0055] In one embodiment of this disclosure, the convolution operation module performing convolution operations on the Nth group of data and the plurality of convolution kernels in the first order includes: the convolution operation module performing M convolution operations on the Nth group of data and M convolution kernels in the first order to generate M convolution operation values, where M represents the number of the plurality of convolution kernels. The data processing method may further include: the output control module merging the M convolution operation values ​​to generate a convolution operation result for the Nth group of data.

[0056] In some possible implementations, after the convolution operation starts, taking the Nth data group as the first data group and the first convolution kernel to the Mth convolution kernel as the first sequence, the first data group data0 and the first convolution kernel kernel0 based on the effective edge of the clock cycle are sent to the convolution operation module to output the first operation result 0_0 of the first data group. The second convolution kernel kernel1 based on the effective edge of the clock cycle is sent to the convolution operation module to perform convolution operation with the first data group data0 to output the second operation result 0_1 ​​of the first data group. And so on, the Mth convolution kernel based on the effective edge of the clock cycle is sent to the convolution operation module to perform convolution operation with the first data group data0 to output the Mth operation result 0_M of the first data group. The first operation result 0_0, the second operation result 0_1, ..., the Mth operation result 0_M of the first data group are merged to generate the convolution operation result of the first data group, where M is the number of convolution kernels required for the operation, M≥1.

[0057] Figure 5 The diagram shows a flowchart of the convolution operation described in an embodiment of this disclosure. Figure 5 As shown, step S13 may include the following steps S131 to S132.

[0058] Step S131: The convolution operation module performs M convolution operations on the N+1th group of data and M convolution kernels in the second order to generate M convolution operation values, where M represents the number of the multiple convolution kernels.

[0059] In step S132, the output control module merges the M convolution operation values ​​to generate the convolution operation result for the (N+1)th group of data.

[0060] In one embodiment of this disclosure, the convolution operation module performing convolution operations on the (N+1)th group of data and the plurality of convolution kernels in the second order may include: the convolution operation module performing M convolution operations on the (N+1)th group of data and M convolution kernels in the second order to generate M convolution operation values, where M represents the number of the plurality of convolution kernels. The data processing method further includes: the output control module merging the M convolution operation values ​​to generate a convolution operation result for the (N+1)th group of data.

[0061] Among some possible implementations, Figure 6 The diagram shown is a schematic representation of the data processing method described in an embodiment of this disclosure. Figure 6As shown, after the convolution operation starts, taking the (N+1)th data group as the next second data group adjacent to the first data group as an example, when the convolution operation of the first data group is completed, the last data sent to the convolution operation module is the first data group data0 and the Mth convolution kernel kernelM. At this time, the second data group data1 is sent to the convolution operation module, while keeping the convolution kernel kernelM already sent to the convolution operation module unchanged. The convolution operation module performs M convolution operations on the second data group data1 and the M convolution kernels in the second order to generate M convolution operation values. Specifically, the first convolution operation is performed on the second data group data1 and the Mth convolution kernel kernelM sent to the convolution operation module to output the first operation result 1_0 of the second data group. The second convolution operation is performed on the (M-1)th convolution kernel kernelM-1 sent to the convolution operation module and the second data group data1 to output the second operation result 1_1 of the second data group. Similarly, the first convolution kernel (kernel0) sent to the convolution operation module is convolved with the second set of data (data1) to output the M-th operation result (0_M) of the second set of data. The first operation result (1_M), the second operation result (1_M-1), ..., the M-th operation result (1_0) of the second set of data are merged to generate the convolution operation result for the second set of data, where M is the number of convolution kernels required for the operation, and M≥1.

[0062] In one embodiment of this disclosure, the data processing method may further include: integrating the convolution operation results for the Nth group of data and the convolution operation results for the N+1th group of data by the output control module to output the data operation result.

[0063] In some possible implementations, the convolution result of the Nth group of data is the result of the convolution kernel arranged in ascending order: N_0, N_1…N_M. The convolution result of the N+1th group of data is the result of the convolution kernel arranged in descending order: N+1_M, N+1_M-1…N+1_0. The convolution result of the N+1th group of data is then reordered to N+1_0, N+1_1…N+1_M-1, N+1_M, and the data operation result is output.

[0064] Figure 7 The diagram shown is a structural schematic of the data processing apparatus described in an embodiment of this disclosure. Figure 7 As shown, the data processing device 100 includes a data control module 110, a convolution kernel control module 120, a convolution operation module 130, and an output control module 140.

[0065] The data control module 110 is configured to acquire multiple sets of data for convolution operations.

[0066] The convolution kernel control module 120 is configured to acquire multiple convolution kernels required for convolution operations, and to output the multiple convolution kernels in a first order for the Nth group of data in the multiple groups of data, and to output the multiple convolution kernels in a second order opposite to the first order for the (N+1)th group of data adjacent to the Nth group of data in the multiple groups of data, where N≥1.

[0067] The convolution operation module 130 is configured to receive the plurality of convolution kernels sequentially from the convolution kernel control module in the first order, and perform convolution operations on the Nth group of data and the plurality of convolution kernels in the first order to generate a plurality of convolution operation values ​​for the Nth group of data; and to receive the plurality of convolution kernels sequentially from the convolution kernel control module in the second order, and perform convolution operations on the N+1th group of data and the plurality of convolution kernels in the second order to generate a plurality of convolution operation values ​​for the N+1th group of data.

[0068] The output control module 140 is configured to merge multiple convolution operation values ​​for the Nth group of data to generate a convolution operation result for the Nth group of data, and to merge multiple convolution operation values ​​for the (N+1)th group of data to generate a convolution operation result for the (N+1)th group of data.

[0069] In some possible implementations, the convolution operation function of the convolutional neural network is implemented by multipliers and adders. When both sides of the input data of the convolution operation change, it is equivalent to both sides of the input data of the multiplier changing. At this time, the flipping probability of the internal circuit nodes of the multiplier increases significantly, which also leads to a significant increase in the dynamic power consumption of the multiplier circuit. The data control module 110 is configured to acquire multiple sets of data for the convolution operation. The convolution kernel control module 120 is configured to acquire multiple convolution kernels required for the convolution operation and integrate the convolution kernels required for this operation into the convolution kernel control module 120. The number of convolution kernels is based on the actual operation requirements. For the first set of data in the multiple sets of data used for the convolution operation, the convolution kernel control module 120 sends the multiple convolution kernels to the convolution operation module 130 in a first order. The convolution operation module 130 is configured to perform convolution operations on the first set of data and the multiple convolution kernels in the first order. For the second group of data adjacent to the first group of data in the plurality of data sets, the convolution kernel control module 120 sends the plurality of convolution kernels to the convolution operation module 130 in a second order, which is the reverse of the first order. Sequentially, for the Nth group of data in the plurality of data sets used for convolution operations, the convolution kernel control module 120 sends the plurality of convolution kernels to the convolution operation module 130 in a first order. For the (N+1)th group of data adjacent to the Nth group of data in the plurality of data sets, the convolution kernel control module 120 sends the plurality of convolution kernels to the convolution operation module 130 in a second order, which is the reverse of the first order. Finally, the operation results of the first group, third group…Nth group of data operated in the first order and the operation results of the second group, fourth group…N+1th group of data operated in the second order, which is the reverse of the first order, are output. The output control module 140 is configured to merge the N+1 group operation results to output the data operation result, where N is the number of data sets to be operated on, and N≥1.

[0070] In this embodiment, the order in which the convolution kernels are sent from the convolution kernel control module 120 to the convolution operation module 130 is optimized. The convolution kernels are maintained during data transitions, ensuring that only one side of the convolution calculation experiences a transition. This approach does not affect the speed of the convolution operation and does not require changes to the original convolution circuit; only the input order of the convolution kernels needs to be altered. This ensures that the convolution kernels remain unchanged during data transitions, thereby reducing the power consumption of the convolutional neural network circuit.

[0071] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0072] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this disclosure, depending on actual needs. For example, the functional modules / units in the various embodiments of this disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0073] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0074] This disclosure also provides an electronic device. Figure 8 The diagram shown is a structural schematic of the electronic device 800 described in an embodiment of this disclosure. Figure 8 As shown, in this embodiment, the electronic device 800 includes a memory 810 and a processor 820.

[0075] The memory 810 is configured to store information associated with the convolution operation; the memory 810 includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card or optical disk.

[0076] Specifically, memory 810 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 810 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure. It is understood that memory 810 may be volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM) or programmable read-only memory (PROM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memories.

[0077] The processor 820 is electrically coupled to the memory 810 and is configured to perform the data processing method according to any embodiment of the present disclosure.

[0078] In some embodiments, processor 820 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. In other embodiments, processor 820 may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0079] In some embodiments, the electronic device 800 may further include a display 830. The display 830 is communicatively connected to the memory 810 and the processor 820, and is used to display a graphical user interface (GUI) of the data processing method described in this disclosure. The terms “component,” “module,” “system,” etc., used in this specification are used to denote computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in processes and / or execution threads, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components may execute from various computer-readable media on which various data structures are stored. Components may communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0080] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0081] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this disclosure should still be covered by the claims of this disclosure.

Claims

1. A data processing method, characterized in that, include: The convolution kernel control module obtains multiple convolution kernels required for the convolution operation; For the Nth set of data in a plurality of sets of data used for convolution operations, the convolution kernel control module sends the plurality of convolution kernels to the convolution operation module in a first order, where N≥1; The convolution operation module performs convolution operations on the Nth group of data and the plurality of convolution kernels in the first order; For the (N+1)th group of data that is adjacent to the Nth group of data in the multiple groups of data, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in a second order that is reversed from the first order. as well as The convolution operation module performs convolution operations on the N+1th group of data and the plurality of convolution kernels in the second order.

2. The data processing method according to claim 1, characterized in that, The convolution kernel control module obtains multiple convolution kernels required for the convolution operation, including: The kernel control module obtains all the kernels required for the convolution operation and stores all the kernels in the kernel control module.

3. The data processing method according to claim 1, characterized in that, The convolution kernel control module sends the plurality of convolution kernels sequentially to the convolution operation module in a first order, including: For a set of data to be convolutionally processed, the convolution kernel control module sends the plurality of convolution kernels to the convolution operation module in the first order according to the effective edge of the clock cycle. The convolution kernel control module sends the plurality of convolution kernels sequentially to the convolution operation module in a second order, which is the reverse of the first order. For the next set of data immediately following the first set of data, the convolution kernel control module sends the multiple convolution kernels to the convolution operation module in the second order according to the effective edge of the clock cycle.

4. The data processing method according to claim 1, characterized in that, The convolution operation module performs convolution operations on the Nth group of data and the plurality of convolution kernels in the first order, including: The convolution operation module performs M convolution operations on the Nth group of data and M convolution kernels in the first order to generate M convolution operation values, where M represents the number of the multiple convolution kernels; The data processing method further includes: The output control module merges the M convolution operation values ​​to generate the convolution operation result for the Nth group of data.

5. The data processing method according to claim 1, characterized in that, The convolution operation module performs convolution operations on the N+1th group of data and the plurality of convolution kernels in the second order, including: The convolution operation module performs M convolution operations on the N+1th group of data and M convolution kernels in the second order to generate M convolution operation values, where M represents the number of the multiple convolution kernels; The data processing method further includes: The output control module merges the M convolution operation values ​​to generate the convolution operation result for the (N+1)th group of data.

6. The data processing method according to claim 1, characterized in that, Also includes: The output control module integrates the convolution operation results for the Nth group of data and the convolution operation results for the (N+1)th group of data to output the data operation result.

7. A data processing apparatus, characterized in that, include: The data control module is configured to acquire multiple sets of data for convolution operations; The convolution kernel control module is configured to acquire multiple convolution kernels required for convolution operations, and to output the multiple convolution kernels in a first order for the Nth group of data in the multiple groups of data, and to output the multiple convolution kernels in a second order opposite to the first order for the N+1th group of data in the multiple groups of data, where N≥1; The convolution operation module is configured to receive the plurality of convolution kernels sequentially from the convolution kernel control module in a first order, and perform convolution operations on the Nth group of data with the plurality of convolution kernels in the first order to generate a plurality of convolution operation values ​​for the Nth group of data; and to receive the plurality of convolution kernels sequentially from the convolution kernel control module in a second order, and perform convolution operations on the N+1th group of data with the plurality of convolution kernels in the second order to generate a plurality of convolution operation values ​​for the N+1th group of data. The output control module is configured to merge multiple convolution operation values ​​for the Nth group of data to generate a convolution operation result for the Nth group of data, and to merge multiple convolution operation values ​​for the (N+1)th group of data to generate a convolution operation result for the (N+1)th group of data.

8. The data processing apparatus according to claim 7, characterized in that, The convolution kernel control module is configured as follows: Obtain all convolution kernels required for the convolution operation and store all convolution kernels in the convolution kernel control module; For a set of data to be convolved, the multiple convolution kernels are sent to the convolution operation module in the first order according to the effective edge of the clock cycle; as well as For the next set of data immediately following the first set of data, the plurality of convolution kernels are sent to the convolution operation module in the second order according to the effective edge of the clock cycle.

9. The data processing apparatus according to claim 7, characterized in that, The convolution operation module is configured as follows: The plurality of convolution kernels are received sequentially in the first order, and the Nth group of data is subjected to M convolution operations with each of the received convolution kernels to generate M convolution operation values, where M represents the number of the plurality of convolution kernels. as well as The multiple convolution kernels are received sequentially in the second order, and the (N+1)th group of data is subjected to M convolution operations with each of the received convolution kernels to generate M convolution operation values.

10. The data processing apparatus according to claim 9, characterized in that, The output control module is configured as follows: The M convolution operation values ​​received from the convolution operation module for the Nth group of data are merged to generate the convolution operation result for the Nth group of data; as well as The M convolution operation values ​​received from the convolution operation module for the (N+1)th group of data are merged to generate the convolution operation result for the (N+1)th group of data.

11. An electronic device, characterized in that, include: The memory is configured to store information associated with the convolution operation; as well as The processor is electrically coupled to the memory and configured to perform the data processing method according to any one of claims 1 to 6.

Images