Scaling method, chip, storage medium and electronic device

Abstract

The present disclosure provides a scaling method, a chip, a storage medium and an electronic device. The method is applied to a neural network processing unit and comprises: reading input data and a weight tensor; configuring at least one convolution kernel according to the weight tensor; performing convolution operation on the input data by at least one of the convolution kernels through a multiply-add array of the neural network processing unit; and outputting a result of the convolution operation as scaling data of the input data. The method can realize scaling operation on input data by using a neural network processing unit without adding an additional hardware scaling module.

Description

Technical Field

[0001] This disclosure pertains to the field of data processing and relates to a scaling operator implementation technique, particularly to scaling methods, chips, storage media, and electronic devices. Background Technology

[0002] To improve visual quality, image resizing is often necessary during image processing. Furthermore, many applications in artificial intelligence, such as object segmentation models, involve scaling intermediate data during inference. Scaling operators are typically implemented using specialized integrated circuits (ICs), such as application-specific integrated circuits (ASICs). Summary of the Invention

[0003] This disclosure provides scaling methods, chips, storage media, and electronic devices for implementing scaling operators through an NPU (Neural-network Processing Unit) to scale data.

[0004] In a first aspect, embodiments of this disclosure provide a scaling method applied to a neural network processing unit. The scaling method includes: reading input data and a weight tensor; configuring at least one convolutional kernel according to the weight tensor; performing convolution operations on the input data using at least one convolutional kernel through a multiply-accumulate array of the neural network processing unit; and outputting the result of the convolution operation as scaled data of the input data.

[0005] In one implementation of the first aspect, performing convolution operations on the input data using at least one of the convolution kernels includes: performing edge-padding processing on the input data to generate edge-padding data; combining each part of the edge-padding data corresponding to at least one of the convolution kernels with the input data to form data to be processed; and performing convolution operations on the data to be processed using at least one of the convolution kernels, and writing the results of the convolution operations on the data to be processed into buffers in a skip-over-offset manner.

[0006] In one implementation of the first aspect, outputting the result of the convolution operation as scaled data of the input data includes: outputting the results stored in the buffer and written in a skip-write manner across offsets as the scaled data.

[0007] In one implementation of the first aspect, configuring at least one convolutional kernel according to the weight tensor includes: configuring convolutional weights according to the weight tensor; and configuring Tdx×Tdy convolutional kernels, the size of each convolutional kernel being configured as K1×K2, where K1 and K2 are both positive integers, the input stride of each convolutional kernel in the horizontal and vertical directions being configured as Tsx and Tsy respectively, and the output stride of each convolutional kernel in the horizontal and vertical directions being configured as Tdx and Tdy respectively, wherein Tsx and Tdx are coprime and Tsx:Tdx=w_sx:w_dx, where w_sx and w_dx are the width of the input data and the width of the output data, respectively.

[0008] In one implementation of the first aspect, reading the input data and the weight tensor includes: reading the input data and the weight tensor and storing them in a buffer, and the scaling method further includes: performing edge-padding processing on the input data to generate edge-padding data and storing the edge-padding data in the buffer, wherein performing convolution operations on the input data using at least one of the convolution kernels includes: reading a portion of the edge-padding data corresponding to at least one of the convolution kernels from the buffer as data to be processed in a skip-read manner across offsets; performing convolution operations on the data to be processed using at least one of the convolution kernels; and writing the result of the convolution operation on the data to be processed into the buffer or an additional buffer different from the buffer in a skip-write manner across offsets.

[0009] In one implementation of the first aspect, the cross-offset jump read is performed via direct memory read access of the neural network processing unit, and the cross-offset jump write is performed via direct memory write access of the neural network processing unit.

[0010] In one implementation of the first aspect, the horizontal offset jump range of the cross-offset jump read is Tsx, and the horizontal offset jump range of the cross-offset jump write is Tdx, where Tsx and Tdx are coprime and Tsx:Tdx = w_sx:w_dx, w_sx and w_dx are the width of the input data and the width of the output data, respectively; and the vertical offset jump range of the cross-offset jump read is Tsy, and the vertical offset jump range of the cross-offset jump write is Tdy, where Tsy and Tdy are coprime and Tsy:Tdy = h_sy:h_dy, h_sy and h_dy are the height of the input data and the height of the output data, respectively.

[0011] In one implementation of the first aspect, performing convolution operations on the data to be processed using at least one of the convolution kernels includes: performing convolution operations on two or more of the data to be processed in parallel using the multiply-accumulate array.

[0012] In one implementation of the first aspect, the multiply-accumulate array of the neural network processing unit performs a parallel acceleration task of the neural network while performing convolution operations on the input data.

[0013] Secondly, embodiments of this disclosure provide a chip including a neural network processing unit, the neural network processing unit being configured to scale the input data using the scaling method described in any implementation of the first aspect of this disclosure to generate the scaled data.

[0014] Thirdly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed to implement the scaling method according to any implementation of the first aspect of this disclosure.

[0015] Fourthly, embodiments of this disclosure provide an electronic device, the electronic device comprising: a memory configured to store a computer program; and a processor configured to invoke the computer program to perform a scaling method according to any implementation of the first aspect of this disclosure.

[0016] As described above, the scaling method provided in this disclosure can implement the scaling operator through a neural network processing unit, thus eliminating the need for additional scaling hardware modules and facilitating further reduction in chip size. Furthermore, the scaling operator implementation algorithm utilizes a multiply-accumulate array in the neural network processing unit for scaling, resulting in high efficiency. It also supports various scaling parameter operations and is adaptable to different scaling algorithms, demonstrating good versatility. Attached Figure Description

[0017] Figure 1A The diagram shown is an example of an application scenario of the scaling method in this disclosure.

[0018] Figure 1B The flowchart shown is a scaling method in an embodiment of this disclosure.

[0019] Figure 1C The diagram shown is a schematic representation of the bilinear interpolation method in an embodiment of this disclosure.

[0020] Figure 2 The flowchart shown is a process of performing convolution processing on input data in an embodiment of this disclosure.

[0021] Figure 3 The flowchart shown is a process of performing convolution processing on input data in an embodiment of this disclosure.

[0022] Figure 4A The diagram shows the input and output data in an embodiment of this disclosure.

[0023] Figure 4B The flowchart shown is a scaling method in an embodiment of this disclosure.

[0024] Figures 4C to 4K The diagram shows the input and output data in the scaling method described in this embodiment of the present disclosure.

[0025] Figure 5 The diagram shown is a schematic representation of the chip structure in an embodiment of this disclosure.

[0026] Figure 6 The diagram shown is a schematic representation of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0027] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. This disclosure can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0028] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. Therefore, the drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0029] Traditional technologies require adding additional scaling hardware modules to the chip when implementing scaling operators, which hinders further reduction in chip size. To address this issue, this disclosure provides a scaling method that implements the scaling operator through an NPU, eliminating the need for additional scaling hardware modules and facilitating further reduction in chip size. Furthermore, the scaling operator implementation algorithm utilizes the MAC multiply-accumulate array in the NPU for scaling, resulting in high efficiency, support for various scaling parameter operations, and adaptability to different scaling algorithms, demonstrating good versatility.

[0030] The technical solutions in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0031] Figure 1AThis is a hardware architecture diagram illustrating an embodiment of the present disclosure. The scaling operator implementation algorithm provided in this embodiment of the present disclosure can be achieved through... Figure 1A The architecture shown is used to implement this. Figure 1A In this NPU, a data-parallel computing architecture is employed to accelerate neural network operations and improve chip efficiency during neural network computation. The NPU includes a MAC multiply-accumulate array, a controller, a cache, input circuitry, and output circuitry. The MAC multiply-accumulate array, composed of multiple multipliers and adders, is the core component for leveraging the NPU's computing power. A multiply-accumulator is an arithmetic unit that can perform multiple multiplications and additions within the same instruction cycle. The input circuitry reads data from external sources to be processed. The output circuitry outputs the processing results. The controller controls the input circuitry, output circuitry, and MAC multiply-accumulate array, enabling the NPU to implement the scaling method according to embodiments of this disclosure.

[0032] Figure 1B This is a flowchart illustrating a scaling method provided according to an embodiment of the present disclosure. For example... Figure 1B As shown, the scaling method provided in this embodiment includes the following steps S11 to S14.

[0033] In step S11, input data and a weight tensor are read. The input data can originate from images captured by a camera, NPU computation results, CPU computation results, GPU computation results, and / or RGA computation results, etc. The weight tensor is used to represent interpolation weights and convolution weights in step S12. In some embodiments, the input data and weight tensor are read through the NPU's input circuitry. In some embodiments, this input circuitry reads the input data and weight tensor from outside the NPU via RDMA (read direct memory access).

[0034] In step S12, at least one convolutional kernel is configured according to the weight tensor. In some embodiments, convolutional weights are configured according to the weight tensor, and convolutional kernels with corresponding sizes and numbers are configured. In some embodiments, the corresponding convolutional kernels are configured according to the weight tensor by the NPU controller.

[0035] In step S13, the input data is convolved using at least one convolution kernel via the NPU's MAC multiply-accumulate array. This convolution operation will be described in detail in a later embodiment.

[0036] In step S14, the result of the convolution operation is output as scaled data of the input data. In some embodiments, the scaled data can be output to external memory. In some embodiments, the convolution operation result as scaled data is output through the output circuit of the NPU. In some embodiments, the output circuit outputs the scaled data of the input data to the external NPU via WDMA (write direct memory access).

[0037] The scaling method provided in this embodiment supports multiple scaling modes, such as nearest neighbor interpolation, bilinear interpolation, and tricubic interpolation. Taking bilinear interpolation as an example... Figure 1C This is a schematic diagram illustrating the bilinear interpolation method. Points Q11(x1,y1), Q12(x1,y2), Q21(x2,y1), and Q22(x2,y2) are points in the input data. In this embodiment, linear interpolation of these four points in the x-direction can be performed using the following equations 1 and 2:

[0038]

[0039]

[0040] Where f(x,y1) represents the interpolation result of the point at position (x,y1) (i.e., R1), f(x,y2) represents the interpolation result of the point at position (x,y2) (i.e., R2), and f(Q) represents the value of point Q.

[0041] Based on the interpolation results in the x-direction, the following equation 3 can be used to interpolate in the y-direction to obtain the required estimate:

[0042]

[0043] Where f(x,y) is the final interpolation result.

[0044] As can be seen from the above process, the interpolation process of the scaling operator is a multiplication-addition operation, and multiplication-addition can be implemented through convolution. Therefore, the interpolation process of the scaling operator can be equivalent to a process of several convolution operations. Based on this principle, in the embodiments of this disclosure, step S12 can achieve the scaling processing of the input data by performing convolution operations on the input data.

[0045] Figure 2 The flowchart shown is an embodiment of this disclosure that demonstrates the convolution operation performed on the input data using at least one convolution kernel. For example... Figure 2 As shown, in this embodiment of the present disclosure, performing convolution operations on the input data includes the following steps S21 to S23.

[0046] In step S21, the input data is padded to generate padded data. Padded data refers to filling the boundaries of the input data with numerical values ​​to increase its size. The padded values ​​can be, for example, 0 or other values; this disclosure does not impose specific limitations. In some implementations, the padded data can be padded in advance in a first buffer. In other implementations, padded data can be padded during the convolution operation. In this case, the padded data does not need to be stored, thus eliminating the read / write task and improving efficiency.

[0047] In step S22, the data corresponding to at least one convolutional kernel in the edge-padding data are combined with the input data to form the data to be processed.

[0048] In step S23, at least one convolution kernel is used to perform convolution operations on the data to be processed, and the results of the convolution operations on the data to be processed are written into the buffers in a skip-over manner across offsets. This convolution operation can be implemented using a MAC multiply-accumulate array.

[0049] In some embodiments, cross-offset jump writes are performed via WDMA through the NPU.

[0050] In some embodiments, using the result of the output convolution operation as scaled data of the input data includes: using the results stored in the buffer and written separately in a skip-over manner as scaled data.

[0051] According to one embodiment of this disclosure, a center-aligned mode is adopted. Configuring at least one convolutional kernel based on a weight tensor includes: configuring convolutional weights based on the weight tensor; configuring Tdx × Tdy convolutional kernels, each kernel having a size of K1 × K2, where K1 and K2 are both positive integers, for example, both being 2. The input stride of each convolutional kernel in the horizontal and vertical directions is configured as Tsx and Tsy, respectively, and the output stride of each convolutional kernel in the horizontal and vertical directions is configured as Tdx and Tdy, respectively. Tsx and Tdx are coprime, Tsy and Tdy are coprime, and Tsx:Tdx = w_sx:w_dx, Tsy:Tdy = h_sy:h_dy, where w_sx and w_dx are the width of the input data and the width of the output data, respectively, and h_sy and h_dy are the height of the input data and the height of the output data, respectively.

[0052] In a specific example, if the input data size is 4×4 and the output data size is 8×8, that is, w_sx = 4, w_sy = 4, w_dx = 8, w_dy = 8. Therefore, Tsx = 1, Tsy = 1, Tdx = 2, Tdy = 2. Based on this, in this embodiment, 2×2 convolutional kernels C1, C2, C3, and C4 are configured, and each kernel has a size of 2×2. Each kernel has a horizontal stride of 1 in both the horizontal and vertical directions, and an output stride of 2 in both the horizontal and vertical directions. The periods in the horizontal and vertical directions are independent of each other.

[0053] According to one embodiment of this disclosure, reading the input data and the weight tensor includes: reading the input data and the weight tensor and storing them in a buffer. The scaling method provided in this disclosure further includes: performing edge-padding processing on the input data to generate edge-padding data and storing the edge-padding data in a buffer.

[0054] Figure 3 The flowchart shown is an embodiment of this disclosure that demonstrates the convolution operation performed on the input data using at least one convolution kernel. For example... Figure 3 As shown, in this embodiment of the present disclosure, the convolution operation on the input data using at least one convolution kernel includes the following steps S31 to S33.

[0055] In step S31, a portion of the input data and the padding data corresponding to at least one convolutional kernel are read from the buffer in a cross-offset skip read manner as the data to be processed.

[0056] In step S32, at least one convolution kernel is used to perform convolution operations on the data to be processed.

[0057] In step S33, the results of the convolution operation on the data to be processed are written to the buffer or an additional buffer different from the buffer in a cross-offset skip write manner.

[0058] In some embodiments, the data to be processed can be processed serially. Specifically, the first piece of data to be processed is read, a convolution operation is performed on it, and the result is written to a buffer. Then the second piece of data to be processed is read, a convolution operation is performed on it, and the result is written to a buffer. This process continues until all data to be processed has been processed.

[0059] In some embodiments, a serial pipeline approach can also be used to process each piece of data. Specifically, the process includes: reading the first piece of data to be processed; performing a convolution operation on the first piece of data to be processed while simultaneously reading the second piece of data to be processed; writing the convolution result of the first piece of data to be processed into a buffer, simultaneously performing a convolution operation on the second piece of data to be processed while reading the third piece of data to be processed; writing the convolution result of the second piece of data to be processed into a buffer, simultaneously performing a convolution operation on the third piece of data to be processed while reading the fourth piece of data to be processed. This process continues until all pieces of data to be processed have been calculated.

[0060] In some embodiments, the data to be processed can also be processed in parallel. In this case, m data points can be read in parallel, convolution operations can be performed on these m data points in parallel, and the results can be written to a buffer. Then, n data points can be read in parallel, convolution operations can be performed on these n data points in parallel, and the results can be written to a buffer. This process continues until all data points have been processed. Here, m and n are positive integers, and the values ​​of m and n can be the same or different.

[0061] In some embodiments, a parallel pipeline approach can also be used to process each piece of data. Specifically, the process includes: reading m pieces of data to be processed; performing convolution operations on the m pieces of data in parallel, while simultaneously reading n pieces of data to be processed; writing the convolution results of the m pieces of data to a buffer, while simultaneously performing convolution operations on the n pieces of data to be processed in parallel, and reading k pieces of data to be processed; writing the convolution results of the n pieces of data to a buffer, while simultaneously performing convolution operations on the k pieces of data to be processed in parallel, and reading j pieces of data to be processed. This process continues until all pieces of data to be processed have been calculated. Here, m, n, k, and j are all positive integers, and their values ​​can be the same or different.

[0062] In some embodiments, cross-offset skip reads are performed via the NPU's RDMA function; and / or, cross-offset skip writes are performed via the NPU's WDMA function in embodiments of this disclosure.

[0063] In some embodiments, the horizontal offset jump for a cross-offset jump read is Tsx, and the horizontal offset jump for a cross-offset jump write is Tdx, where Tsx and Tdx are coprime and Tsx:Tdx = w_sx:w_dx, where w_sx and w_dx are the width of the input data and the width of the output data, respectively. The vertical offset jump for a cross-offset jump read is Tsy, and the vertical offset jump for a cross-offset jump write is Tdy, where Tsy and Tdy are coprime and Tsy:Tdy = h_sy:h_dy, where h_sy and h_dy are the height of the input data and the height of the output data, respectively.

[0064] Specifically, for input data stored in the form of an N1×M1 matrix, when reading data of size n1×m1 using a cross-offset skip reading method, assume the coordinates of the current point read are (x_1, y_1). If x_1 + Tsx ≤ M1, no line break is needed, and the coordinates of the next point read are (x_1 + Tsx, y_1); if x_1 + Tsx > M1, a line break is needed, and the coordinates of the next point read are (x_1 + Tsx - M1, y_1 + Tsy). Starting from (x0, y0), reading n1×m1 points in the above manner yields one piece of data to be processed. Subsequently, starting from (x1, y1), reading n1×m1 points in the above manner yields another piece of data to be processed.

[0065] If the output data is stored in the form of an N2×M2 matrix, when writing the convolution result of a piece of data to be processed using the cross-offset skip write method, assume the coordinates of the current write point are (x_2, y_2). If x_2+Tdx≤M2, no newline write is needed, and the coordinates of the next write point are (x_2+Tdx, y_2); if x_2+Tdx>M2, a newline write is needed, and the coordinates of the next write point are (x_2+Tdx-M2, y_2+Tdy).

[0066] In some embodiments, performing convolution operations on the data to be processed using at least one convolution kernel includes: performing convolution operations on two or more data to be processed in parallel using a MAC multiply-accumulate array.

[0067] In some embodiments, the NPU's MAC multiply-accumulate array performs neural network acceleration tasks in parallel while performing convolution operations on the input data.

[0068] Specifically, MAC multiply-accumulate arrays typically have sufficient computing power, enabling them to perform convolution operations on two or more data to be processed in parallel, as well as to execute convolution operations on data to be processed and accelerate neural network tasks in parallel.

[0069] The process described above will be illustrated with a concrete example. In this example, the scaling operator is used to scale 2×2×C input data to 4×4×C output data, where C is a positive integer. The input and output data are as follows: Figure 4A As shown. In this specific instance, w_sx = 2, w_sy = 2, w_dx = 4, w_dy = 4, therefore, Tsx = 1, Tdx = 2, Tsy = 1, Tdy = 2. Figure 4B This is a flowchart illustrating the scaling process applied to the input data in this specific instance. For example... Figure 4B As shown, the scaling process for the input data in this specific example includes the following steps S41 to S48.

[0070] In step S41, the input data is padded with edges, such as... Figure 4C As shown. In some implementations, the input data can be padded in advance in the first buffer. In other implementations, the padded data can be padded during the convolution operation. In this case, the padded data does not need to be stored, so there is no task of reading and writing the padded data, which is more efficient.

[0071] In step S42, a convolution kernel is configured according to the weight tensor. The size of the convolution kernel is 2×2, the input stride is 1, and the output stride is 2. The convolution weights of the convolution kernel are determined by the weight tensor. In this embodiment of the disclosure, four different convolution kernels C1, C2, C3, and C4 can be configured using four interpolation weights.

[0072] In some embodiments, one implementation method for the above four convolution kernels can be: C1 = {0.25×0.25, 0.75×0.25, 0.25×0.75, 0.75×0.75}, C2 = {0.25×0.75, 0.75×0.75, 0.25×0.25, 0.75×0.25}, C3 = {0.75×0.25, 0.25×0.25, 0.75×0.75, 0.75×0.25}, C4 = {0.75×0.75, 0.25×0.75, 0.75×0.25, 0.25×0.25}.

[0073] In step S43, the 3×3 data in the upper left corner of the input data is read as the data to be processed using a cross-offset skip read method, and the data to be processed is processed using convolution kernel C1 to obtain 2×2 interpolation points, such as... Figure 4D As shown in the figure. The offset jump magnitude for cross-offset jump reading is 1. It should be noted that the 3×3 data read in this process and the interpolation points obtained after convolution processing can be stored in the same buffer or in different buffers.

[0074] In step S44, the interpolation point obtained in step S43 is written to the second buffer using a cross-offset skip write method, such as... Figure 4E As shown in the figure. The offset jump magnitude for cross-offset jump writes is 2.

[0075] In step S45, the next 3×3 data point of the current data to be processed is read using a cross-offset jump read method as the new data to be processed, and the new data to be processed is processed using convolution kernel C2 to obtain a new 2×2 interpolation point. Since the offset jump magnitude of the cross-offset jump read in this example is 1, the new data to be processed and the obtained new interpolation point are as follows: Figure 4F As shown.

[0076] In step S46, the new interpolation point obtained in step S45 is written to the second buffer using a cross-offset jump write method. Since the offset jump magnitude of the cross-offset jump write in this example is 1, after writing the new interpolation point from step S45 to the second buffer, as shown... Figure 4G As shown.

[0077] In step S47, the next 3×3 data point of the current data to be processed is read in a cross-offset skip read manner as the new data to be processed, and the new data to be processed is processed using convolution kernel C3 to obtain a new 2×2 interpolation point, such as... Figure 4H As shown, after writing the new interpolation point into the second buffer, as follows... Figure 4I As shown.

[0078] In step S48, the next 3×3 data point of the current data to be processed is read in a skip-over manner as the new data to be processed, and the new data to be processed is processed using convolution kernel C4 to obtain a new 2×2 interpolation point, such as... Figure 4J As shown, after writing the new interpolation point into the second buffer, as follows... Figure 4K As shown.

[0079] Furthermore, in this specific example, a MAC multiply-accumulate array can be used to perform convolution processing on two or more data to be processed simultaneously, that is, to achieve parallel processing of two or more data to be processed. For example, steps S43 and S47 can be executed in parallel, as can steps S43, S45, and S48.

[0080] The scope of protection of the scaling method described in this disclosure is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this disclosure is included within the scope of protection of this disclosure.

[0081] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed to implement the scaling method provided according to any embodiment of this disclosure.

[0082] In this disclosure, any combination of one or more storage media may be used. The storage media may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0083] This disclosure also provides a chip. Figure 5 The diagram shown is a schematic representation of the chip in an embodiment of this disclosure. The chip includes an NPU, which is configured to scale input data using the scaling method provided in any embodiment of this disclosure to generate scaled data.

[0084] This disclosure also provides an electronic device. Figure 6 The diagram shown is a structural schematic of the electronic device 600 in an embodiment of this disclosure. Figure 6 As shown, in this embodiment, the electronic device 600 includes a memory 610 and a processor 620.

[0085] The memory 610 is used to store computer programs. In some embodiments, the memory 610 includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk.

[0086] Specifically, memory 610 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic device 600 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 610 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0087] The processor 620 is connected to the memory 610 and is used to execute the computer program stored in the memory 610 so that the electronic device 600 performs the scaling method.

[0088] In some embodiments, the processor 620 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0089] In some embodiments, the electronic device 600 may further include a display 630. The display 630 is communicatively connected to the memory 610 and the processor 620, and is used to display the relevant GUI interactive interface of the scaling method.

[0090] In summary, the scaling method provided in this disclosure can implement the scaling operator through the NPU, thus eliminating the need for additional scaling hardware modules and facilitating further reduction in chip size. Furthermore, the scaling operator implementation algorithm utilizes the MAC multiply-accumulate array in the NPU for scaling, resulting in high efficiency, support for various scaling parameter operations, and adaptability to different scaling algorithms, demonstrating good versatility. Therefore, this disclosure effectively overcomes the various shortcomings of the prior art and possesses high industrial applicability.

[0091] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0092] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this disclosure should still be covered by the claims of this disclosure.

Claims

1. A scaling method, characterized in that, The scaling method, applied to a neural network processing unit, includes: Read the input data and the weight tensor, and store the input data and the weight tensor in a buffer; The input data is processed to generate edge-padded data, and the edge-padded data is stored in the buffer. Configure at least one convolutional kernel according to the weight tensor; The input data is convolutionally processed using the multiply-accumulate array of the neural network processing unit, with at least one convolution kernel applied to each data point; and The result of the convolution operation is output as the scaling data of the input data. The method of performing convolution operations on the input data using at least one of the convolution kernels includes: reading a portion of the input data and the padding data corresponding to at least one of the convolution kernels from the buffer as data to be processed in a skip-read manner across offsets; performing convolution operations on the data to be processed using at least one of the convolution kernels; and writing the result of the convolution operation on the data to be processed into the buffer or an additional buffer different from the buffer in a skip-write manner across offsets.

2. The scaling method according to claim 1, characterized in that, Performing convolution operations on the input data using at least one of the convolution kernels includes: The input data is processed to generate edge-padded data; The padded edge data corresponding to at least one of the convolutional kernels are combined with the input data to form the data to be processed; and The convolution operation is performed on the data to be processed using at least one of the convolution kernels, and the results of the convolution operation on the data to be processed are written into the buffers in a skip-offset manner.

3. The scaling method according to claim 2, characterized in that, The output of the convolution operation as scaled data of the input data includes: The results of writing the data stored in the buffer in a skip-offset manner are output as the scaling data.

4. The scaling method according to claim 1, characterized in that, Configuring at least one convolutional kernel based on the weight tensor includes: Configure convolution weights according to the weight tensor; and Configure Tdx×Tdy convolutional kernels, with each kernel having a size of K1×K2, where K1 and K2 are both positive integers. The input stride of each kernel in the horizontal and vertical directions is configured as Tsx and Tsy, respectively, and the output stride of each kernel in the horizontal and vertical directions is configured as Tdx and Tdy, respectively. Tsx and Tdx are coprime and Tsx:Tdx = w_sx:w_dx, where w_sx and w_dx are the width of the input data and the width of the output data, respectively.

5. The scaling method according to claim 1, characterized in that, The cross-offset jump read is performed via direct memory read access of the neural network processing unit, and the cross-offset jump write is performed via direct memory write access of the neural network processing unit.

6. The scaling method according to claim 1, characterized in that, The horizontal offset jump amplitude for the cross-offset jump read is Tsx, and the horizontal offset jump amplitude for the cross-offset jump write is Tdx, where Tsx and Tdx are coprime and Tsx:Tdx = w_sx:w_dx, where w_sx and w_dx are the widths of the input data and the output data, respectively; and The offset jump reading in the vertical direction has an offset jump magnitude of Tsy, and the offset jump writing in the vertical direction has an offset jump magnitude of Tdy, where Tsy and Tdy are coprime and Tsy:Tdy = h_sy:h_dy, and h_sy and h_dy are the height of the input data and the height of the output data, respectively.

7. The scaling method according to claim 1 or 2, characterized in that, Performing convolution operations on the data to be processed using at least one of the convolution kernels includes: performing convolution operations on two or more of the data to be processed in parallel using the multiply-accumulate array.

8. The scaling method according to claim 1, characterized in that, The multiply-accumulate array of the neural network processing unit performs convolution operations on the input data, thereby accelerating the neural network in parallel.

9. A chip, characterized in that, The system includes a neural network processing unit configured to scale the input data using the scaling method described in any one of claims 1 to 8 to generate the scaled data.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed to implement the scaling method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, The electronic device includes: Memory, configured to store computer programs; and The processor is configured to invoke the computer program to perform the scaling method according to any one of claims 1 to 8.

Images