Feature map caching device, method, equipment and storage medium
Through the design of the feature map cache device, the preset configuration parameters and the reading method of convolution kernel sliding order are used to solve the problem of input data expansion in neural networks, improve data reuse rate and reduce hardware costs.
Patent Information
- Application Number
- CN202411287574.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-14
AI Technical Summary
In the prior art, the input data of neural networks is expanded after img2col processing, resulting in low input data reuse rate, increasing storage and handling costs, and thus increasing hardware area and power consumption.
A feature map cache device is designed to generate storage addresses by preset configuration parameters, write data of feature image pixels to the cache module, and read data according to the sliding order of the convolution kernel, avoiding image-to-column processing. Improved data reuse rate.
It improves the multiplexing rate of feature map data, reduces storage and handling costs, reduces hardware area and power consumption, and provides a large output bandwidth, reducing access and bandwidth to external memory.
Smart Images

Figure CN118799168B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of chip technology, and in particular to a feature map cache device, method, equipment and storage medium. Background Art
[0002] The format of neural network input data is generally determined by the batch sample size, image height, image width, and image channel number. Among them, the image height, image width, and image channel number are the three dimensions of the image. The sizes of the dimensions of the input data of different neural networks are different, and they need to be convolved with the weight tensor. The format of the weight tensor is generally determined by the width of the convolution kernel, the height of the convolution kernel, the input channel, and the output channel. Generally, the MAC (Multiply-Accumulate) calculation array is used to calculate the input data, and the MAC calculation array can only perform addition and multiplication calculations, so the input data needs to be sorted according to the actual calculation method.
[0003] In related technologies, the image to column (img2col) method is used to organize input data, such as TPU (Tensor Processing Unit) and NPU (Neural Processing Unit).
[0004] However, the input data will expand exponentially after being processed by img2col, which makes the input data reuse rate low, increases the storage cost and transportation cost of the input data, and further increases the hardware area and power consumption. Summary of the invention
[0005] The embodiment of the present application provides a feature map cache device, method, equipment and storage medium, which can improve the reuse rate of input data, reduce the storage cost and transportation cost of input data, and thus reduce the hardware area and power consumption. The technical solution is as follows:
[0006] According to a first aspect of an embodiment of the present application, a feature map caching device is provided, the device comprising:
[0007] A write module, a cache module and a read module are electrically connected in sequence; the storage format of the cache module is determined based on preset configuration parameters; the cache module includes multiple storage areas; each storage area includes s sub-storage areas; the t-th sub-storage area includes multiple storage columns and multiple storage rows; the q-th storage area is used to store data of the q-th row of pixels of the feature map; the first storage address The corresponding storage block is used to store the corresponding pixel points data; Indicates the qth storage area; is the t-th sub-storage area; Indicates the address of the i-th storage row and the j-th storage column; Indicates the pixel point in the qth row; Indicates the qth storage area and the tth sub-storage area. The pixel points; s is the step size, s is an integer greater than or equal to 1; t is an integer greater than or equal to 0 and less than or equal to s; q is an integer greater than or equal to 0 and less than or equal to the number of rows of the feature map; i is an integer greater than or equal to 0 and less than or equal to (m-1); m is the number of storage rows; m is an integer greater than or equal to the width of the convolution kernel and less than or equal to the width of the feature map; j is an integer greater than or equal to 0;
[0008] The writing module is used to obtain the pixel point The first storage address is generated based on the preset configuration parameters Based on the first storage address , the pixel The data is written into the cache module;
[0009] The read module is used to generate the kth group of read addresses based on the preset configuration parameters; each of the read addresses indicates a first storage address; the data of n pixels corresponding to the kth group of the read addresses are read from the cache module; the gth row of values in the convolution kernel are calculated with the qth row of pixel data; the n pixels are the pixels calculated with the kth value; k is an integer greater than or equal to 0 and less than or equal to (the width of the convolution kernel - 1); n and g are integers greater than or equal to 1.
[0010] In a possible implementation, the preset configuration parameters include the width and precision of the convolution kernel, the height and width of the feature map, the amount of feature map data included in each pixel in the feature map, and the step size.
[0011] In a possible implementation, when each pixel point includes c pieces of feature map data; the t-th storage column includes c sub-storage columns;
[0012] Second storage address The corresponding storage block is used to store feature map data ; represents the address of the d-th sub-storage column in the i-th storage row and the j-th storage column; represents the dth feature map data; d is an integer greater than or equal to 0 and less than or equal to c; c is an integer greater than or equal to 0.
[0013] In a possible implementation, the writing module is further used for:
[0014] If the initial precision of the feature map data is greater than the precision, the initial precision is adjusted to the precision.
[0015] In one possible implementation, the read module is also used to output the feature map data corresponding to each read address based on the read address arrangement order when the kth group of read addresses are not in the same sub-storage column; the read address arrangement order is generated by the read module; the read address arrangement order is used to adjust the output order of the data of n pixels.
[0016] In a possible implementation, the device is electrically connected to a memory; the writing module is also used to obtain the feature map data from the memory.
[0017] In a possible implementation, the device is electrically connected to the memory via a reader;
[0018] The reader is used to obtain the characteristic map data from the memory and send the characteristic map data to the writing module.
[0019] In a possible implementation, the device further includes a control module, which is electrically connected to the write module, the cache module, and the read module respectively;
[0020] The control module is used to obtain a first storage state and a second storage state; the first storage state indicates whether the write module can write the characteristic map data; the second storage state indicates whether the read module can output the characteristic map data;
[0021] The control module is also used to control the cache module based on the first storage address , the pixel The data is written into the cache module;
[0022] The control module is further used to control the cache module to read the n pixel points corresponding to the kth group of the read addresses;
[0023] The control module is further used to control the writing module to generate a first signal based on the first storage state; the first signal indicates whether the characteristic map data can be written to the writing module;
[0024] The control module is also used to control the reading module to generate a second signal based on the second storage state; the second signal indicates whether the characteristic map data can be read from the reading module.
[0025] In a possible implementation, the reader is further used for:
[0026] When the first signal indicates that the characteristic graph data can be written to the writing module, the characteristic graph data is sent to the writing module based on the first signal.
[0027] In a possible implementation, the reading module is electrically connected to a calculator;
[0028] The control module is also used to control the reading module to send the second signal to the calculator, and based on the third signal, output n pixel points to the calculator; the third signal is generated by the calculator; the third signal indicates that the calculator is in a state where it can receive data.
[0029] According to a second aspect of an embodiment of the present application, a feature map caching method is provided, the method comprising:
[0030] Get pixel points The first storage address is generated based on the preset configuration parameters Based on the first storage address , the pixel The data is written into the cache module; the cache module includes multiple storage areas; each storage area includes s sub-storage areas; the t-th sub-storage area includes multiple storage columns and multiple storage rows; the q-th storage area is used to store the data of the q-th row of pixels of the feature map; the first storage address The corresponding storage block is used to store the corresponding pixel points data; Indicates the qth storage area; is the t-th sub-storage area; Indicates the address of the i-th storage row and the j-th storage column; Indicates the pixel point in the qth row; Indicates the qth storage area and the tth sub-storage area. The pixel points; s is the step size, s is an integer greater than or equal to 1; t is an integer greater than or equal to 0 and less than or equal to s; q is an integer greater than or equal to 0 and less than or equal to the number of rows of the feature map; i is an integer greater than or equal to 0 and less than or equal to (m-1); m is the number of storage rows; m is an integer greater than or equal to the width of the convolution kernel and less than or equal to the width of the feature map; j is an integer greater than or equal to 0;
[0031] A k-th group of read addresses is generated based on the preset configuration parameters; each of the read addresses indicates a first storage address; data of n pixels corresponding to the k-th group of read addresses are read from the cache module; the g-th row of values in the convolution kernel are calculated with the q-th row of pixel data; n pixels are the pixels calculated with the k-th value; k is an integer greater than or equal to 0 and less than or equal to (the width of the convolution kernel - 1); n and g are integers greater than or equal to 1.
[0032] According to a third aspect of an embodiment of the present application, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the above-mentioned feature map caching method.
[0033] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the above-mentioned feature map caching method.
[0034] In an embodiment of the present application, an embodiment of the present application provides a feature map cache device, the write module generates a first storage address according to the storage rule corresponding to the storage format, and stores the corresponding pixel data into the cache module according to the first storage address, thereby realizing the pre-arrangement of the pixel data in the feature map. The read module reads the feature map data in the cache module according to the order in which each value in the convolution kernel slides on the feature map. Due to the characteristics of the convolution calculation, the feature map data in the cache module will be repeatedly read out and then calculated, thereby greatly improving the reuse rate of the feature map data. In addition, the embodiment of the present application does not need to process the feature map from image to column, thereby avoiding the exponential expansion of the data volume, thereby reducing the handling of the feature map data, alleviating the bandwidth pressure of the feature map data, and reducing the storage pressure of the feature map data. In addition, the write module receives the data of one pixel each time, and the read module outputs the data of multiple pixels each time, that is, a smaller input bandwidth, which can provide a larger output bandwidth, thereby reducing the access to the external memory and the bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0036] Figure 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application;
[0037] Figure 2 is a schematic structural diagram of a first feature map caching device provided according to an embodiment of the present application;
[0038] Figure 3 is a schematic diagram of the structure of a second feature map cache device provided according to an embodiment of the present application;
[0039] Figure 4 is a structural diagram of a storage format provided according to an embodiment of the present application;
[0040] Figure 5 It is a schematic diagram of the structure of the storage method of the feature map in the memory;
[0041] Figure 6 is a schematic structural diagram of a third feature map caching device provided according to an embodiment of the present application;
[0042] Figure 7 It is a flowchart of a feature map caching method provided according to an embodiment of the present application;
[0043] Figure 8 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application;
[0044] Fig. 9 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0045] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0046] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.
[0047] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.
[0048] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.
[0049] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.
[0050] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application, and the implementation environment may include a terminal 101 and a server 102.
[0051] In the terminal 101, a feature map cache device is provided, for example, a feature map cache device provided on a computer or a feature map cache device provided on a smart phone.
[0052] The terminal 101 may be a smart phone with a feature map caching device, a wearable device, a personal computer, a laptop computer, a tablet computer, a smart TV, a vehicle-mounted terminal, etc.
[0053] The server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud processing center.
[0054] The terminal 101 is connected to the server 102 via a wired or wireless network.
[0055] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to any combination of local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, wired or wireless network, private network or virtual private network. In some embodiments, the data exchanged through the network is represented by technology and / or format including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technology can also be used to replace or supplement the above data communication technology.
[0056] The feature map involved in the embodiments of the present application can also be represented as input data of a neural network. Each feature map includes multiple pixels. For example, the feature map is a 3-channel RGB (Red Green Blue) image.
[0057] At present, before processing the feature map, it may be necessary to store and read the feature map. For example, for a feature map, each pixel in the feature map is stored in order from left to right and from top to bottom, so as to store the entire feature map. When it is necessary to read the stored feature map, the stored feature map is read according to the preset reading rules.
[0058] In addition, before storing the feature map, the format of the feature map is also sorted. For example, the format of the feature map is sorted using an image-to-column method. The image-to-column method converts multi-dimensional image data into a two-dimensional matrix form to facilitate efficient matrix operations.
[0059] However, because image-to-column converts multi-dimensional image data into a two-dimensional matrix form, the amount of data expands exponentially, resulting in a low input data reuse rate, increased storage and transportation costs of input data, and further increased hardware area and power consumption.
[0060] In order to solve the above technical problems, an embodiment of the present application provides a feature map caching device 200.
[0061] Figure 2 It is a structural diagram of a first feature map caching device 200 provided according to an embodiment of the present application.
[0062] Combine the following Figure 2 The structure of the feature map caching device 200 is exemplarily described.
[0063] In some embodiments, the feature map cache device 200 includes a write module 201, a cache module 202, and a read module 203 that are electrically connected in sequence.
[0064] In some embodiments, the writing module 201 is used to obtain a feature map and write the feature map into the cache module 202 ; the reading module 203 is used to read the feature map from the cache module 202 .
[0065] In one example, in order to save hardware area, the cache module 202 is a SRAM (Static Random Access Memory).
[0066] In some embodiments, the storage format of the cache module 202 is determined based on preset configuration parameters; the cache module 202 includes multiple storage areas; each storage area includes s sub-storage areas; the t-th sub-storage area includes multiple storage columns and multiple storage rows; the q-th storage area is used to store the data of the q-th row of pixels of the feature map; the first storage address The corresponding storage block is used to store the corresponding pixel points data; Indicates the qth storage area; is the t-th sub-storage area; Indicates the address of the i-th storage row and the j-th storage column; Represents the qth row of pixels; The tth sub-storage area in the qth storage area represents pixels; s is the step size, which is an integer greater than or equal to 1; t is an integer greater than or equal to 0 and less than or equal to s; q is an integer greater than or equal to 0 and less than or equal to the number of rows in the feature map; i is an integer greater than or equal to 0 and less than or equal to (m-1); m is the number of storage rows; m is an integer greater than or equal to the width of the convolution kernel and less than or equal to the width of the feature map; j is an integer greater than or equal to 0.
[0067] The storage format is equivalent to determining a storage rule. The storage rule can be understood as each pixel in the feature map matches a unique first storage address in the cache module 202. That is, in the cache module 202, the pixel needs to be stored in the storage block corresponding to the matching first storage address.
[0068] It can be understood that the storage areas correspond one-to-one to the rows in the feature map.
[0069] In some embodiments, before the cache device is running, or when the cache device is initialized, the cache module 202 obtains preset configuration parameters and generates a storage format based on the preset configuration parameters.
[0070] In one example, the preset configuration parameters are configured by a staff member based on an actually required computing method, for example, the actually required computing method is a convolution computing method.
[0071] In some embodiments, the preset configuration parameters include the width and precision of the convolution kernel, the height and width of the feature map, the amount of feature map data included in each pixel in the feature map, and the step size.
[0072] In general, the width of the convolution kernel is the same as the height of the convolution kernel. Therefore, the preset configuration parameters only need to consider the height of the convolution kernel. When the width of the convolution kernel is different from the height of the convolution kernel, the height of the convolution kernel needs to be considered.
[0073] In addition, the stride indicates the step size of the convolution kernel when sliding on the feature map. The stride determines the number of sub-storage areas in each storage area. The precision is the bit width of the input channel. Each pixel in the feature map corresponds to multiple input channels. The bit width of the pixel can be obtained based on the product of the number of input channels and the bit width of the input channel. The feature map data is a part of the corresponding pixel. The height of the feature map determines the number of rows of the feature map. The width of the feature map determines the number of pixels in each row.
[0074] For example, the size of the convolution kernel is 3×3 or 5×5, etc. That is, the width and height of the convolution kernel are 3 or 5 respectively.
[0075] For example: the precision is 4 bits, 8 bits, 16 bits, fp8 (8-bit floating-point number), fp16 (Half-precision floating-point), and bf16 (Brain Floating-Point Format).
[0076] For example, if one pixel corresponds to 32 input channels and the bit width of each input channel is 8 bits, then the bit width of the pixel is 256 bits.
[0077] In the embodiments of the present application, the feature map cache device supports mixed-precision feature maps, convolution kernels and step sizes, etc., and has strong applicability.
[0078] Figure 3 It is a structural diagram of a second feature map caching device 200 provided according to an embodiment of the present application.
[0079] Combine the following Figure 3 The structure of the cache module 202 is exemplarily described.
[0080] In some embodiments, the cache module 202 includes a plurality of cache units, and the cache units correspond to storage rows one by one.
[0081] For example, the address depth of each cache unit is 128 and the data width is 256 bits. That is, the size of each cache unit is 128×256 bits. The number of cache units is 8.
[0082] For example, the cache module 202 may include a 0th cache unit, a 1st cache unit, a 7th cache unit, and so on.
[0083] Next, continue to combine Figure 3 The structure of the writing module 201 is exemplarily described.
[0084] In some embodiments, the writing module 201 is used to obtain pixel points The first storage address is generated based on the preset configuration parameters ; Based on the first storage address , the pixel The data is written into the cache module 202.
[0085] For example, the bit width of the first storage address is 7 bits, corresponding to the address depth of each cache unit.
[0086] In some embodiments, the writing module 201 obtains pixel data from the feature map in a sequence from left to right and from top to bottom. Each time the writing module 201 obtains data of a pixel, it generates a matching first storage address based on preset configuration parameters and storage rules.
[0087] In one example, the writing module 201 includes a first generating unit 2011 and a first receiving unit 2012. The first generating unit 2011 is used to generate a first storage address and send the first storage address to the cache module 202. The first receiving unit 2012 is used to obtain pixel data and send the pixel data to the cache module 202.
[0088] In an embodiment of the present application, the writing module generates a first storage address according to a storage rule corresponding to the storage format, and stores the corresponding pixel data into the cache module 202 according to the first storage address, thereby realizing the organization of the pixel data in the feature map.
[0089] Next, continue to combine Figure 3 The structure of the reading module 203 is exemplarily described.
[0090] In some embodiments, the read module 203 is used to generate the kth group of read addresses based on preset configuration parameters; each read address indicates a first storage address; the data of n pixels corresponding to the kth group of read addresses are read from the cache module 202; the gth row of values in the convolution kernel are calculated with the data of the qth row of pixels; the n pixels are the pixels calculated with the kth value; k is an integer greater than or equal to 0 and less than or equal to (the width of the convolution kernel - 1); n and g are integers greater than or equal to 1.
[0091] In some embodiments, in order to obtain the result of the convolution calculation corresponding to the feature map, the reading module 203 can determine the pixel points that each value in the convolution kernel slides over in each row of the feature map based on the convolution kernel and the step size. Each value is calculated with the data of the pixel points that slide over, and the result of the convolution calculation of the feature map and the convolution kernel can be obtained. Therefore, the reading module reads the data of the pixel points that a value slides over in a row of the feature map each time.
[0092] In the first example, the reading module determines a value corresponding to the convolution kernel in the order from left to right and from top to bottom, and then reads the data of the pixel point where the value slides over a certain row. When the reading module determines the row corresponding to the value in the feature map, it also determines it in the order from top to bottom.
[0093] In the second example, when the height of the feature map is greater than the height of the convolution kernel, after calculating each value in the convolution kernel and the data of the pixel points sliding over the corresponding row, the process of the first example above is repeated until the data of all the pixels in the cache module 202 are read.
[0094] In one example, the reading module 203 includes a second generating unit 2031 and a second receiving unit 2032. The second generating unit 2031 and the second receiving unit 2032 are electrically connected. The second generating unit 2031 is used to generate multiple read addresses and send the multiple read addresses to the cache module 202. The second receiving unit 2032 is used to obtain the data of the pixel corresponding to each read address.
[0095] For example, the bit width of the read address is 7 bits, corresponding to the address depth of each cache unit.
[0096] In an embodiment of the present application, the write module receives data of one pixel each time, and the read module 203 outputs data of multiple pixels each time, that is, a smaller input bandwidth can provide a larger output bandwidth, thereby reducing access to external memory and bandwidth.
[0097] In some embodiments, when each pixel point includes c feature map data; the tth storage column includes c sub-storage columns.
[0098] Second storage address The corresponding storage block is used to store feature map data ; Represents the address of the d-th sub-storage column in the i-th storage row and the j-th storage column; represents the d-th feature map data; d is an integer greater than or equal to 0 and less than or equal to c; c is an integer greater than or equal to 0.
[0099] In one example, the number of storage rows in each sub-storage area is determined by the number of pixels that each value in the convolution kernel slides over in a row. That is, the reading module can read one sub-storage column from the cache module 202 each time.
[0100] Among them, for a value, after the reading module finishes reading all the characteristic graph data of the value sliding over a certain row, it reads all the characteristic graph data of the next value sliding over a certain row. That is, for a value, the characteristic graph data includes several characteristic graph data, and the reading module needs to read several times to finish reading all the characteristic graph data of the value sliding over a certain row.
[0101] Figure 4 It is a structural diagram of a storage format provided according to an embodiment of the present application.
[0102] Combine the following Figure 4 The writing module is described exemplarily.
[0103] For example: when the size of the convolution kernel is 3×3, the precision is 8 bits, the step size is 2, the width of the feature map is 17, each pixel includes two feature map data; the number of storage rows is 8; 8 is the number of pixels that each value in the convolution kernel slides over in a certain row.
[0104] The first receiving unit obtains the pixel point for the first time The generating unit generates a first storage address .
[0105] The first receiving unit obtains the pixel point for the second time The first generating unit generates a first storage address .
[0106] The first receiving unit obtains the pixel point for the third time The first generating unit generates a first storage address .
[0107] The first receiving unit obtains the pixel point for the fourth time The first generating unit generates a first storage address .
[0108] The first receiving unit obtains the pixel point for the fifth time The first generating unit generates a first storage address .
[0109] When the first receiving unit obtains the pixel point When the data is received, the first generating unit generates a first storage address .
[0110] The receiving unit obtains the pixel point The first generating unit generates a first storage address .
[0111] Next, continue to combine Figure 4 The reading module is described as an example.
[0112] For example, the second generation unit generates the following read address for the first time: , , , , , , as well as .
[0113] That is, the above eight read addresses correspond to eight first storage addresses of the 0th sub-storage column in the 0th storage column of the 0th sub-storage area in the 0th storage area.
[0114] The second receiving unit receives the following feature map data for the first time: , , , , , , as well as .
[0115] These feature map data need to be calculated with the values of the first row and first column in the convolution kernel.
[0116] The second generation unit generates the following read address for the second time: , , , , , , as well as .
[0117] That is, the above eight read addresses correspond to eight first storage addresses of the first sub-storage column in the 0th storage column of the 0th sub-storage area in the 0th storage area.
[0118] The second receiving unit receives the following feature map data for the second time: , , , , , , as well as .
[0119] These feature map data also need to be calculated with the values of the first row and first column in the convolution kernel.
[0120] For another example, the second generation unit generates the following read address for the third time: , , , , , , as well as .
[0121] That is, the above 8 read addresses correspond to: 8 first storage addresses of the 0th sub-storage column of the 0th storage column in the 1st sub-storage area in the 0th storage area.
[0122] The second receiving unit receives the following characteristic graph data for the third time: , , , , , , as well as .
[0123] These feature map data need to be calculated with the values in the first row and second column of the convolution kernel.
[0124] The second generation unit generates the following read address for the fourth time: , , , , , , as well as .
[0125] That is, the above eight read addresses correspond to eight first storage addresses of the first sub-storage column in the 0th storage column of the first sub-storage area in the 0th storage area.
[0126] The second receiving unit receives the following characteristic graph data for the fourth time: , , , , , , as well as .
[0127] These feature map data need to be calculated with the values in the first row and second column of the convolution kernel.
[0128] For another example, the second generation unit generates the following read address for the fifth time: , , , , , , as well as .
[0129] That is, the above 8 read addresses correspond to: 7 first storage addresses of the 0th sub-storage column of the 0th sub-storage area in the 0th storage area, and one first storage address of the 0th sub-storage column of the 1st storage column of the 0th sub-storage area in the 0th storage area.
[0130] The second receiving unit receives the following characteristic graph data for the fifth time: , , , , , , as well as .
[0131] These feature map data need to be calculated with the values in the first row and third column of the convolution kernel.
[0132] The second generation unit generates the following read address for the sixth time: , , , , , , as well as .
[0133] That is, the above 8 read addresses correspond to: 7 first storage addresses of the 0th sub-storage column of the 0th storage column of the 0th sub-storage area in the 0th storage area, and 1 first storage address of the 1st sub-storage column of the 1st storage column of the 0th sub-storage area in the 0th storage area.
[0134] The second receiving unit receives the following characteristic graph data for the sixth time: , , , , , , as well as .
[0135] These feature map data need to be calculated with the values in the first row and third column of the convolution kernel.
[0136] It can be seen from the above examples that under the above preset configuration parameters, the first receiving unit receives 256 bits of feature map data each time; the second receiving unit outputs 2048 bits of feature map data each time, thereby reducing access to external memory and bandwidth.
[0137] In some embodiments, the writing module is further configured to:
[0138] If the initial precision of the feature map data is greater than the accuracy, the initial precision is adjusted to the accuracy.
[0139] In one example, after the first receiving unit acquires the feature map data, if the initial accuracy of the feature map data is greater than the accuracy in the preset configuration parameters, the initial accuracy is split into the accuracy in the preset configuration parameters; or if the initial accuracy of the feature map data is less than or equal to the accuracy in the preset configuration parameters, no processing is performed.
[0140] For example, the precision is 8 bits; when the first receiving unit obtains 256 bits of feature map data and the initial precision is 4 bits, no processing is performed and the first receiving unit directly sends the feature map data to the cache module.
[0141] When the first receiving unit obtains 256 bits of feature map data with an initial precision of 16 bits, the original 16×16 is split into two 8×16, the bit width of the feature map data remains unchanged, and the precision becomes 8 bits. That is, the embodiment of the present application supports mixed precision feature map data.
[0142] In some embodiments, the read module is also used to output the feature map data corresponding to each read address based on the read address arrangement order when the kth group of read addresses is not in the same sub-storage column; the read address arrangement order is generated by the read module; the read address arrangement order is used to adjust the output order of the data of n pixels.
[0143] In one example, the second generating unit automatically generates a read address arrangement order. The read address arrangement order indicates the actual read order. The actual read order can be understood as the order in which the convolution kernel slides. However, since the order of the storage rows in the cache module is fixed, the 8 feature map data obtained by the second receiving unit are arranged in sequence from top to bottom according to the sequence number of the storage row.
[0144] Continue to combine Figure 4 When the eight feature map data are in the same sub-storage column, the arrangement order from top to bottom according to the sequence number of the storage row is correct, so there is no need to adjust the arrangement order of the eight feature map data.
[0145] For example, the storage rows are row 0, row 1, and row 2 from top to bottom.
[0146] The arrangement order of the feature map data received by the second receiving unit for the fourth time is: , , , , , , as well as .
[0147] However, when these 8 feature map data are not in the same sub-storage column, the feature map data that should have been passed by the convolution kernel is placed in front. In this way, when the values in the convolution kernel are calculated with these 8 feature map data, the calculation order will be wrong, resulting in wrong calculation results. Therefore, the arrangement order of these 8 feature map data needs to be adjusted.
[0148] For example, the correct arrangement order of the feature map data received by the second receiving unit for the fifth time is: , , , , , , as well as That is, among these 8 feature map data, the value in the convolution kernel should last slide over However, the actual order is , , , , , , as well as Therefore, it is necessary to sort the addresses in order. Adjust to After that, that is, the second receiving unit outputs first , and then output . So the value in the convolution kernel is first summed Calculate and then Calculation, thereby ensuring the correctness of the convolution calculation results.
[0149] In one example, when the reuse rate of the feature map data needs to be adjusted, the read address generated by the second generation unit is adjusted, that is, the preset configuration parameters are adjusted.
[0150] In the embodiment of the present application, the reading module reads the feature map data in the cache module according to the order in which each value in the convolution kernel slides on the feature map. Due to the characteristics of the convolution calculation, the feature map data in the cache module will be read out repeatedly and then calculated, thereby greatly improving the reuse rate of the feature map data. In addition, the embodiment of the present application does not need to process the feature map from image to column, thereby avoiding the exponential expansion of the data volume, thereby reducing the handling of the feature map data, alleviating the pressure on the bandwidth of the feature map data, and reducing the pressure on the storage of the feature map data.
[0151] Figure 5 It is a structural diagram of how feature maps are stored in memory.
[0152] Combine the following Figure 5 The storage format of the present application is described exemplarily.
[0153] In one example, the memory includes multiple rows and multiple columns. Each row is used to store data of a row of pixels in a feature map, and each column is used to store a column of feature map data in the feature map.
[0154] For example, the 0th row includes multiple pixels, and each pixel includes the 0th feature map data and the 1st feature map data.
[0155] Figure 6 It is a structural diagram of a third feature map caching device 200 provided according to an embodiment of the present application.
[0156] Next, continue to combine Figure 6 The control module 204 is described as an example.
[0157] In some embodiments, the device is electrically connected to the memory 300; the writing module 201 is also used to obtain feature map data from the memory 300.
[0158] In an example, the memory 300 outputs the feature map data to the first receiving unit 2012 of the writing module 201 in sequence from left to right and from top to bottom.
[0159] In some embodiments, the device is electrically connected to the memory 300 via the reader 400;
[0160] The reader 400 is used to obtain the characteristic map data from the memory 300 and send the characteristic map data to the writing module 201.
[0161] In an example, the reader 400 reads the feature map data from the memory 300 in a sequence from left to right and from top to bottom, and sends the read feature map data to the first receiving unit 2012 of the writing module 201 .
[0162] In one example, the reader 400 reads one characteristic map data at a time. The reader 400 reads the characteristic map data once per clock cycle.
[0163] For example, each pixel corresponds to 64 input channels, and the bit width of each input channel is 8 bits. Each pixel includes two feature map data, that is, each feature and data corresponds to 32 input channels. The reader 400 reads 256 bits from the memory 300 each time, that is, one feature map data.
[0164] The reader 400 is RDMA (Remote Direct Memory Access).
[0165] In some embodiments, the device further includes a control module 204, which is electrically connected to the write module 201, the cache module 202, and the read module 203;
[0166] The control module 204 is used to obtain a first storage state and a second storage state; the first storage state indicates whether the write module 201 can write the characteristic map data; the second storage state indicates whether the read module 203 can output the characteristic map data;
[0167] The control module 204 is further configured to control the cache module 202 based on the first storage address , the pixel The data is written into the cache module 202;
[0168] The control module 204 is further used to control the cache module 202 to read n pixels corresponding to the kth group of read addresses;
[0169] The control module 204 is further configured to control the writing module 201 to generate a first signal based on the first storage state; the first signal indicates whether the characteristic map data can be written to the writing module 201;
[0170] The control module 204 is further configured to control the reading module 203 to generate a second signal based on the second storage state; the second signal indicates whether the characteristic map data can be read from the reading module 203.
[0171] In one example, the control module 204 is further configured to send the preset configuration parameters to the cache module 202, so that the cache module 202 generates a storage format based on the preset configuration parameters.
[0172] In one example, the control module 204 is further configured to control the first generating unit 2011 to generate a first storage address and send the first storage address to the cache module 202. The control module 204 is further configured to control the first receiving unit 2012 to generate a first signal based on the first storage state.
[0173] In one example, the control module 204 is further configured to control the second generation unit 2031 to generate a read address and send the read address to the cache module 202. The control module 204 is further configured to control the second receiving unit 2032 to generate a second signal based on the second storage state.
[0174] In some embodiments, the reader 400 is further configured to:
[0175] When the first signal indicates that the feature map data can be written to the writing module 201 , the feature map data is sent to the writing module 201 based on the first signal.
[0176] In one example, when the reader 400 receives the first signal and detects that the fourth signal and the first signal are both valid, the characteristic graph data is sent to the first receiving unit 2012. The fourth signal is generated by the reader 400. The fourth signal indicates that the characteristic graph data sent by the reader 400 is valid.
[0177] For example, when the fourth signal and the first signal are at high level, it is valid, and when the fourth signal and the first signal are at low level, it is invalid.
[0178] In some embodiments, the reading module 203 is electrically connected to the calculator 500;
[0179] The control module 204 is also used to control the reading module 203 to send the second signal to the calculator 500, and output n pixels to the calculator 500 based on the third signal; the third signal is generated by the calculator 500; the third signal indicates that the calculator 500 is in a state of receiving data.
[0180] In one example, when the second receiving unit 2032 receives the third signal and detects that both the third signal and the second signal are valid, the characteristic diagram data is sent to the calculator 500 in the order of the read address arrangement.
[0181] For example, when the third signal and the second signal are at high level, it is valid. When the third signal and the second signal are at low level, it is invalid.
[0182] In one example, the calculator 500 calculates n feature map data in parallel each time. The calculator 500 performs one calculation per clock cycle.
[0183] In one example, the calculator 500 is also electrically connected to a numerical module, and the numerical module is used to provide the numerical value in the convolution kernel to the calculator 500. That is, the calculator 500 obtains n feature map data from the second receiving unit 2032, obtains a numerical value from the numerical module, calculates the n feature map data and the numerical value, and obtains a partial convolution calculation result corresponding to the feature map. After each numerical value in the convolution kernel is calculated with all the feature map data that slides over the feature map, the complete convolution calculation result corresponding to the feature map is obtained.
[0184] For example, the calculator 500 is a MAC calculation array, which includes a plurality of MAC units.
[0185] For example, the MAC calculation array calculates 8 feature map data per clock cycle.
[0186] It should be noted that the numerical module is not within the protection scope of the present application. Therefore, the embodiments of the present application will not further describe the specific structure of the numerical module.
[0187] In the embodiments of the present application, since the rates at which the reader sends characteristic graph data and the rate at which the calculator processes characteristic graph data may be different, the control module is required to control the first receiving unit to generate a first signal and control the second receiving unit to generate a second signal, thereby preventing the cache module from overflowing or reading empty.
[0188] In addition, when the feature map caching device provided in the above embodiment executes the corresponding steps, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0189] In the embodiment of the present application, the writing module generates a first storage address according to the storage rules corresponding to the storage format, and stores the corresponding pixel data into the cache module according to the first storage address, thereby realizing the pre-arrangement of the pixel data in the feature map. The reading module reads the feature map data in the cache module according to the order in which each value in the convolution kernel slides on the feature map. Due to the characteristics of the convolution calculation, the feature map data in the cache module will be repeatedly read out and then calculated, thereby greatly improving the reuse rate of the feature map data. In addition, the embodiment of the present application does not need to process the feature map from image to column, thereby avoiding the exponential expansion of the data volume, thereby reducing the handling of the feature map data, alleviating the bandwidth pressure of the feature map data, and reducing the storage pressure of the feature map data. In addition, the writing module receives the data of one pixel each time, and the reading module outputs the data of multiple pixels each time, that is, a smaller input bandwidth, which can provide a larger output bandwidth, thereby reducing the access to the external memory and the bandwidth.
[0190] Figure 7 is a flow chart of a feature map caching method provided according to an embodiment of the present application, such as Figure 7 As shown, in the embodiment of the present application, the application is described by taking the application on a terminal having a feature map cache device as an example. The method comprises the following steps:
[0191] In step 701, the terminal obtains pixel points The first storage address is generated based on the preset configuration parameters Based on the first storage address , the pixel The data is written into the cache module; the cache module includes multiple storage areas; each storage area includes s sub-storage areas; the t-th sub-storage area includes multiple storage columns and multiple storage rows; the q-th storage area is used to store the q-th row of pixels of the feature map; the first storage address The corresponding storage block is used to store the corresponding pixel points data; Indicates the qth storage area; is the t-th sub-storage area; Indicates the address of the i-th storage row and the j-th storage column; Represents the data of the pixel in the qth row; Indicates the qth storage area and the tth sub-storage area. The pixel points; s is the step size, s is an integer greater than or equal to 1; t is an integer greater than or equal to 0 and less than or equal to s; q is an integer greater than or equal to 0 and less than or equal to the number of rows of the feature map; i is an integer greater than or equal to 0 and less than or equal to (m-1); m is the number of storage rows; m is an integer greater than or equal to the width of the convolution kernel and less than or equal to the width of the feature map; j is an integer greater than or equal to 0;
[0192] In step 702, the terminal generates the kth group of read addresses based on the preset configuration parameters; each of the read addresses indicates a first storage address; the data of n pixels corresponding to the kth group of read addresses are read from the cache module; the gth row of values in the convolution kernel are calculated with the qth row of pixel data; the n pixels are the pixels calculated with the kth value; k is an integer greater than or equal to 0 and less than or equal to (the width of the convolution kernel - 1); n and g are integers greater than or equal to 1.
[0193] In some embodiments, the preset configuration parameters include the width and precision of the convolution kernel, the height of the feature map, the width of the feature map, the amount of feature map data included in each pixel in the feature map, and the step size.
[0194] In some embodiments, when each pixel point includes c pieces of the feature map data; the t-th storage column includes c sub-storage columns;
[0195] Second storage address The corresponding storage block is used to store feature map data ; represents the address of the d-th sub-storage column in the i-th storage row and the j-th storage column; represents the dth feature map data; d is an integer greater than or equal to 0 and less than or equal to c; c is an integer greater than or equal to 0.
[0196] In some embodiments, the method further comprises:
[0197] If the initial precision of the feature map data is greater than the precision, the initial precision is adjusted to the precision.
[0198] In some embodiments, the method further includes: when the kth group of read addresses are not in the same sub-storage column, outputting the feature map data corresponding to each read address based on the read address arrangement order; the read address arrangement order is generated by the read module; the read address arrangement order is used to adjust the output order of the data of n pixels.
[0199] In some embodiments, the method further includes obtaining the feature map data from the memory and sending the feature map data to the writing module.
[0200] In some embodiments, the method further includes: acquiring a first storage state and a second storage state; the first storage state indicates whether the write module can write the feature map data; the second storage state indicates whether the read module can output the feature map data;
[0201] Control the cache module based on the first storage address , the pixel The data is written into the cache module;
[0202] Control the cache module to read the n pixel points corresponding to the kth group of the read addresses;
[0203] Based on the first storage state, the writing module is controlled to generate a first signal; the first signal indicates whether the characteristic map data can be written to the writing module;
[0204] The read module is controlled to generate a second signal based on the second storage state; the second signal indicates whether the characteristic map data can be read from the read module.
[0205] In some embodiments, the method further includes: when the first signal indicates that the characteristic graph data can be written to the writing module, based on the first signal, sending the characteristic graph data to the writing module.
[0206] In some embodiments, the method further includes: controlling the reading module to send the second signal to the calculator, and based on the third signal, outputting n of the pixel points to the calculator; the third signal is generated by the calculator; and the third signal indicates that the calculator is in a state capable of receiving data.
[0207] It should be noted that the feature map caching device and the feature map caching method embodiment provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the device embodiment, which will not be repeated here.
[0208] The write module generates a first storage address according to the storage rule corresponding to the storage format, and stores the corresponding pixel data into the cache module according to the first storage address, thereby realizing the pre-arrangement of the pixel data in the feature map. The read module reads the feature map data in the cache module according to the order in which each value in the convolution kernel slides on the feature map. Due to the characteristics of the convolution calculation, the feature map data in the cache module will be read out repeatedly and then calculated, thereby greatly improving the reuse rate of the feature map data. In addition, the embodiment of the present application does not need to process the feature map from image to column, thereby avoiding the exponential expansion of the data volume, thereby reducing the handling of the feature map data, alleviating the bandwidth pressure of the feature map data, and reducing the storage pressure of the feature map data. In addition, the write module receives the data of one pixel each time, and the read module outputs the data of multiple pixels each time, that is, a smaller input bandwidth, which can provide a larger output bandwidth, thereby reducing the access to the external memory and the bandwidth.
[0209] An embodiment of the present application further provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the above method when executing the computer program.
[0210] Taking computer equipment as the terminal as an example, Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application, see Figure 8 The terminal 800 may be a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The terminal 800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.
[0211] Typically, the terminal 800 includes a processor 801 and a memory 802 .
[0212] The processor 801 may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0213] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one program code, which is used to be executed by the processor 801 to implement the process of terminal execution in the method for mixed-precision floating-point multiplication and accumulation operations provided in the method embodiment of the present application.
[0214] In some embodiments, the terminal 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 804, a camera assembly 805, an audio circuit 806 and a power supply 807.
[0215] The peripheral device interface 803 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 can be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.
[0216] The display screen 804 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 804 is a touch display screen, the display screen 804 also has the ability to collect touch signals on the surface or above the surface of the display screen 804. The touch signal can be input to the processor 801 as a control signal for processing. At this time, the display screen 804 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 804 can be one, set on the front panel of the terminal 800; in other embodiments, the display screen 804 can be at least two, respectively set on different surfaces of the terminal 800 or in a folding design; in other embodiments, the display screen 804 can be a flexible display screen, set on the curved surface or folding surface of the terminal 800. Even, the display screen 804 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 804 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).
[0217] The camera assembly 805 is used to capture images or videos. In some embodiments, the camera assembly 805 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 805 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.
[0218] The audio circuit 806 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 801 for processing. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 801 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 806 may also include a headphone jack.
[0219] The power supply 807 is used to power various components in the terminal 800. The power supply 807 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 807 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.
[0220] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.
[0221] Take the computer device as a server as an example. Fig. 9 It is a structural diagram of a server provided in an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 901 and one or more memories 902, wherein the one or more memories 902 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 901 to implement the above-mentioned feature map caching method. Of course, the server 900 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 900 may also include other components for implementing device functions, which will not be repeated here.
[0222] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned method for generating an image processing model. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.
[0223] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0224] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A feature map caching device, characterized in that: include: A write module, a cache module, and a read module electrically connected in sequence; The storage format of the cache module is determined based on preset configuration parameters; The cache module includes multiple storage areas; each of the storage areas includes s sub-storage areas; The t-th sub-storage area includes a plurality of storage columns and a plurality of storage rows; the q-th storage area is used to store the data of the q-th row of pixels of the feature map; the first storage address The corresponding storage block is used to store the corresponding pixel points data; represents the qth storage area; is the t-th sub-storage area; represents the address of the i-th storage row and the j-th storage column; represents the pixel point in the qth row; represents the first sub-storage area in the tth sub-storage area of the qth storage area. The pixel points; s is the step size, s is an integer greater than or equal to 1; t is an integer greater than or equal to 0 and less than or equal to s; q is an integer greater than or equal to 0 and less than or equal to the number of rows of the feature map; i is an integer greater than or equal to 0 and less than or equal to (m-1); m is the number of storage rows; m is an integer greater than or equal to the width of the convolution kernel and less than or equal to the width of the feature map; j is an integer greater than or equal to 0; The writing module is used to obtain the pixel point The first storage address is generated based on the preset configuration parameters ; Based on the first storage address , the pixel point Write the data into the cache module; The reading module is used to generate a kth group of reading addresses based on the preset configuration parameters; each of the reading addresses indicates one of the first storage addresses; The data of the n pixel points corresponding to the k-th group of the read address are read from the cache module; the g-th row value in the convolution kernel is calculated with the data of the pixel points in the q-th row; the n pixel points are the pixel points calculated with the k-th value; k is an integer greater than or equal to 0 and less than or equal to (the width of the convolution kernel - 1); n and g are integers greater than or equal to 1.
2. The device according to claim 1, characterized in that The preset configuration parameters include the width and precision of the convolution kernel, the height and width of the feature map, the number of feature map data included in each pixel in the feature map, and the step size.
3. The device according to claim 2, characterized in that When each of the pixel points includes c pieces of feature map data; the t-th storage column includes c sub-storage columns; Second storage address The corresponding storage block is used to store feature map data ; represents the address of the d-th sub-storage column in the i-th storage row and the j-th storage column; represents the dth feature map data; d is an integer greater than or equal to 0 and less than or equal to c; c is an integer greater than or equal to 0.
4. The device according to claim 3, characterized in that The writing module is also used for: If the initial precision of the feature map data is greater than the precision, the initial precision is adjusted to the precision.
5. The device according to claim 3, characterized in that The reading module is also used to output the feature map data corresponding to each read address based on the read address arrangement order when the read addresses of the kth group are not in the same sub-storage column; the read address arrangement order is generated by the reading module; the read address arrangement order is used to adjust the output order of the data of the n pixel points.
6. The device according to claim 3, characterized in that The device is electrically connected to the memory; the writing module is also used to obtain the characteristic map data from the memory.
7. The device according to claim 6, characterized in that The device is electrically connected to the memory via a reader; The reader is used to obtain the characteristic map data from the memory and send the characteristic map data to the writing module.
8. The device according to claim 7, characterized in that The device further includes a control module, wherein the control module is electrically connected to the writing module, the cache module and the reading module respectively; The control module is used to obtain a first storage state and a second storage state; the first storage state indicates whether the write module can write the characteristic map data; the second storage state indicates whether the read module can output the characteristic map data; The control module is further used to control the cache module based on the first storage address , the pixel point Write the data into the cache module; The control module is further used to control the cache module to read the n pixel points corresponding to the kth group of read addresses; The control module is further used to control the writing module to generate a first signal based on the first storage state; the first signal indicates whether the characteristic map data can be written to the writing module; The control module is further used to control the reading module to generate a second signal based on the second storage state; the second signal indicates whether the characteristic map data can be read from the reading module.
9. The device according to claim 8, characterized in that The reader is further configured to: When the first signal indicates that the characteristic map data can be written to the writing module, the characteristic map data is sent to the writing module based on the first signal.
10. The device according to claim 8, characterized in that The reading module is electrically connected to the calculator; The control module is also used to control the reading module to send the second signal to the calculator, and output n pixel points to the calculator based on a third signal; the third signal is generated by the calculator; the third signal indicates that the calculator is in a state where it can receive data.
11. A feature map caching method, characterized in that: include: Get pixel points The first storage address is generated based on the preset configuration parameters ; Based on the first storage address , the pixel point The data is written into the cache module; the cache module includes multiple storage areas; each storage area includes s sub-storage areas; the t-th sub-storage area includes multiple storage columns and multiple storage rows; the q-th storage area is used to store the data of the q-th row of pixels of the feature map; the first storage address The corresponding storage block is used to store the corresponding pixel points data; represents the qth storage area; is the t-th sub-storage area; represents the address of the i-th storage row and the j-th storage column; represents the pixel point in the qth row; represents the first sub-storage area in the tth sub-storage area of the qth storage area. The pixel points; s is the step size, s is an integer greater than or equal to 1; t is an integer greater than or equal to 0 and less than or equal to s; q is an integer greater than or equal to 0 and less than or equal to the number of rows of the feature map; i is an integer greater than or equal to 0 and less than or equal to (m-1); m is the number of storage rows; m is an integer greater than or equal to the width of the convolution kernel and less than or equal to the width of the feature map; j is an integer greater than or equal to 0; Generate a kth group of read addresses based on the preset configuration parameters; each of the read addresses indicates one of the first storage addresses; The data of the n pixel points corresponding to the k-th group of the read address are read from the cache module; the g-th row value in the convolution kernel is calculated with the data of the pixel points in the q-th row; the n pixel points are the pixel points calculated with the k-th value; k is an integer greater than or equal to 0 and less than or equal to (the width of the convolution kernel - 1); n and g are integers greater than or equal to 1.
12. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the feature map caching method as described in claim 11.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the feature map caching method as claimed in claim 11.
