Feature data pooling method, electronic device, storage medium, and program

Abstract

Embodiments of the present application disclose a feature data pooling method, an electronic device, a storage medium and a program, wherein the method comprises: carrying each target original input feature map to a local memory through a data transformation engine; performing one-dimensional pooling processing on each target original input feature map in parallel along a first spatial dimension direction of each target original input feature map in the vector register of each sub-thread of the local memory to obtain an intermediate pooling output result of each target original input feature map; performing one-dimensional pooling processing on the intermediate pooling output result of each target original input feature map in parallel along a second spatial dimension direction of the intermediate pooling output result of each target original input feature map to obtain an output feature map corresponding to each target original input feature map. The technical scheme of the embodiments of the present application can optimize the pooling calculation scheduling and the memory access mode, and further improve the operation efficiency and the calculation throughput of the pooling operator.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a method for pooling feature data, an electronic device, a storage medium, and a program. Background Technology

[0002] Pooling, as a core fundamental operator of deep convolutional neural networks, is widely used in computer vision tasks such as image classification, object detection, and semantic segmentation. The mainstream implementation forms include max pooling and average pooling, which achieve feature dimensionality reduction and key information preservation by performing maximum value extraction or mean calculation within a sliding window of the input feature map.

[0003] In existing pooling computation processes, constrained by the data arrangement format of the input feature map, a single pooling kernel struggles to perform a one-time vectorized read and operation on the complete window data, easily leading to fragmented access to window data and significantly reducing memory locality. Simultaneously, a single pooling window generates only one output data point after computation, resulting in low unit computation density and insufficient computational intensity. This fails to fully utilize the parallel processing capabilities of the hardware vector operation units, leading to low hardware resource utilization, limited overall throughput, and high computational latency, making it difficult to meet the real-time processing requirements of high-resolution vision tasks. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, storage medium, and program for pooling feature data, which can optimize pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of pooling operators.

[0005] According to one aspect of the present invention, a pooling method for feature data is provided, comprising: The original input feature maps of each target are moved to local memory through the data transformation engine; In the vector registers of each sub-thread in the local memory, one-dimensional pooling is performed in parallel on each of the original target input feature maps along the first spatial dimension direction to obtain intermediate pooling output results of each of the original target input feature maps; Along the second spatial dimension of the intermediate pooling output of each of the original target input feature maps, one-dimensional pooling is performed in parallel on the intermediate pooling output of each of the original target input feature maps to obtain the output feature map corresponding to each of the original target input feature maps.

[0006] According to another aspect of the present invention, a feature data pooling apparatus is provided, comprising: The target raw input feature map transport module is used to transport the target raw input feature maps to local memory through the data transformation engine; The first spatial dimension direction pooling processing module is used to perform one-dimensional pooling processing on each of the target original input feature maps in parallel along the first spatial dimension direction in the vector registers of each sub-thread in the local memory, so as to obtain the intermediate pooling output results of each of the target original input feature maps. The second spatial dimension direction pooling processing module is used to perform one-dimensional pooling processing on the intermediate pooling output results of each target original input feature map in parallel along the second spatial dimension direction of each target original input feature map, so as to obtain the output feature map corresponding to each target original input feature map.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the feature data pooling method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the feature data pooling method described in any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the feature data pooling method described in any embodiment of the present invention.

[0010] This invention addresses the issues of memory fragmentation, insufficient computational granularity, and low vector parallelism resource utilization in existing pooling methods. It optimizes pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of the pooling operator. The method also optimizes pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of the pooling operator.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a feature data pooling method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a feature data pooling method provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of a feature data pooling device provided in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," and "target," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1This is a flowchart of a feature data pooling method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where the two-dimensional pooling process of the target original input feature map is decomposed into two one-dimensional pooling steps based on the vector registers of each sub-thread in local memory. This method can be executed by a feature data pooling device, which can be implemented in software and / or hardware, and is generally integrated into an electronic device. This electronic device can be a terminal device or a server device, as long as it can execute the feature data pooling method. The embodiments of the present invention do not limit the specific device type of the electronic device. Accordingly, as... Figure 1 As shown, the method includes the following operations: S110. Transfer the original input feature maps of each target to local memory through the data transformation engine.

[0017] The target original input feature map can be two-dimensional original feature data to be pooled, serving as the original data source for pooling. For example, the target original input feature map can include, but is not limited to, image type feature data, temporal and frequency domain feature data of speech signals, or feature data formed by text embedding vectors. This embodiment of the invention does not limit the specific type of the target original input feature map. The data transformation engine can be an asynchronous DMA (Direct Memory Access) unit. Local memory is a high-speed storage unit with extremely low access latency in the hardware architecture for performing pooling operations, and is a core data temporary storage area in high-performance computing scenarios.

[0018] In hardware architectures that perform pooling processing, while the device's global memory has a large capacity, its bandwidth is relatively limited. Therefore, during pooling, the feature map to be processed can be segmented according to a preset dimension to obtain multiple target original input feature maps, which are then moved to local memory by a data transfer engine for pooling processing. For example, the original input feature map can be segmented according to the batch size, and the resulting single batch of input feature maps can be used as the target original input feature maps.

[0019] S120. In the vector registers of each sub-thread in the local memory, perform one-dimensional pooling processing on each of the original target input feature maps in parallel along the first spatial dimension direction to obtain the intermediate pooling output results of each of the original target input feature maps.

[0020] The first and second spatial dimensions can be two mutually perpendicular directions in the spatial domain of the original target input feature map, representing its spatial distribution. For example, the first spatial dimension can be the height direction of the original target input feature map, and the second spatial dimension can be the width direction. This embodiment of the invention does not limit the specific directions represented by the first and second spatial dimensions. The intermediate pooling output can be the pooling output obtained by performing parallel one-dimensional pooling along the first spatial dimension of the original target input feature map.

[0021] Correspondingly, after the original input feature maps of each target are transferred to local memory through the data transformation engine, pooling operations can be performed in parallel on each original input feature map based on the vector registers allocated to multiple sub-threads corresponding to local memory. For the two-dimensional pooling processing of a single target original input feature map, it can be decomposed into a one-dimensional pooling process executed sequentially along two spatial dimensions. Specifically, a preset first current pooling kernel can be used to perform one-dimensional pooling processing on the target original input feature map along the first spatial dimension. By performing sliding window aggregation operations on the input feature data along the first spatial dimension, intermediate pooling output results of the target original input feature map can be obtained.

[0022] S130. Along the second spatial dimension direction of the intermediate pooling output results of each of the target original input feature maps, perform one-dimensional pooling processing on the intermediate pooling output results of each of the target original input feature maps in parallel to obtain the output feature map corresponding to each of the target original input feature maps.

[0023] The output feature map can be a feature map obtained by performing one-dimensional pooling on the intermediate pooling output of the original target input feature map.

[0024] Accordingly, after performing one-dimensional pooling on each target's original input feature map in parallel along the first spatial dimension to obtain intermediate pooling outputs, one-dimensional pooling can be further performed on each intermediate pooling output along its second spatial dimension. By sequentially completing the step-by-step one-dimensional pooling operations in the above two dimensions, a complete two-dimensional pooling process for the target's original input feature maps can be achieved, ultimately obtaining the output feature maps corresponding to each target's original input feature map after two-dimensional pooling.

[0025] Therefore, the feature data pooling method provided in this embodiment of the invention decomposes the two-dimensional pooling process of the target original input feature map into two one-dimensional pooling steps in the vector registers of each sub-thread in local memory. This eliminates the need to perform joint operations on the row and column dimensions of the feature map simultaneously, effectively simplifying the computation logic. Furthermore, by performing two one-dimensional pooling processes, this method optimizes the pooling computation scheduling and memory access mode, fully releasing the parallel computation potential of the vector registers, reducing data redundancy and waiting latency during the computation process, thereby significantly improving the overall throughput of the pooling process.

[0026] This invention addresses the issues of memory fragmentation, insufficient computational granularity, and low vector parallelism resource utilization in existing pooling methods. It optimizes pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of the pooling operator. The method also optimizes pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of the pooling operator.

[0027] Example 2 Figure 2 This is a flowchart of a feature data pooling method provided in Embodiment 2 of the present invention. This embodiment is based on the above embodiment and is further specified. In this embodiment, specific optional implementation methods are given for performing one-dimensional pooling processing on each target original input feature map in parallel along the first spatial dimension direction of each target original input feature map in the vector registers of each sub-thread in local memory, and for performing one-dimensional pooling processing on the intermediate pooling output results of each target original input feature map in parallel along the second spatial dimension direction of each target original input feature map. Optional implementation operations are also given before moving each target original input feature map to local memory through the data transformation engine. Correspondingly, as... Figure 2 As shown, the method in this embodiment may include: S210. Determine the size of the second current pooling kernel used for one-dimensional pooling processing operation along the second spatial dimension direction.

[0028] The second current pooling kernel can be the pooling kernel used when performing one-dimensional pooling operations along the second spatial dimension.

[0029] In this embodiment of the invention, the target original input feature map adopts the NHWC (Number-Height-Width-Channels) data arrangement format, with the height direction as the first spatial dimension and the width direction as the second spatial dimension. In traditional pooling methods, a single pooling window spans multiple rows of data. Although the channel dimension data within the same row is arranged continuously in memory, the data between different rows is not stored continuously, making it impossible to complete the data vectorization loading and calculation of the entire pooling window with a single vector instruction. Furthermore, during one-dimensional pooling along the second spatial dimension, each data access requires checking whether the width coordinate iw is out of bounds, and frequent out-of-bounds checks cause resource consumption. To avoid these problems, this embodiment of the invention performs a pre-padding operation on the width direction of the target original input feature map based on the size of the second current pooling kernel used for one-dimensional pooling along the second spatial dimension. Therefore, it is necessary to determine the size of the second current pooling kernel corresponding to the one-dimensional pooling operation in this direction.

[0030] S220. Determine the preset number of fill columns of the original input feature map based on the size of the second current pooling kernel.

[0031] The preset number of fill columns can be the number of columns used to fill the original input feature map of the target along the width direction.

[0032] Accordingly, after determining the size of the second current pooling kernel used for one-dimensional pooling processing along the second spatial dimension, the preset number of padding columns required on the left and right sides of the target original input feature map in the width direction can be determined based on the size of the second current pooling kernel. For example, the preset number of padding columns can be set to no more than half the size of the second current pooling kernel. For instance, when the size of the second current pooling kernel is... When filling columns, the default number of columns can be 0 or 1.

[0033] S230. Fill the target original input feature map with the target fill value of the preset fill column number to obtain the target original input feature map.

[0034] Accordingly, after determining the preset number of padding columns for the target original input feature map, the preset number of initial padding columns can be applied to both sides of the target original input feature map once before performing one-dimensional pooling. In a specific example, the type of the initial value can be determined according to the type of pooling operation. For example, if the pooling operation is max pooling, the initial value can be -inf; if the pooling operation is average pooling, the initial value can be 0.

[0035] S240. The original input feature maps of each target are transferred to local memory through the data transformation engine.

[0036] S250. Determine the region type of each output feature data block in the output feature map corresponding to each of the original input feature maps of the target.

[0037] The output feature data block can be a data block in the output feature map. The region type identifies the region type to which the input feature data block corresponding to the output feature data block belongs. Region types can include, but are not limited to, filled regions and original regions; this embodiment of the invention does not limit the specific content of the region type. A filled region can be a region added to both sides of the target original input feature map in the width direction through pre-filling; the data within this region is filled data. An original region can be the region in the target original input feature map that is not filled and originates from the actual input feature data.

[0038] When performing parallel one-dimensional pooling processing on the original input feature maps of each target along the first spatial dimension in the vector registers of each sub-thread in local memory, the region type of each output feature data block in the output feature map corresponding to each original input feature map can first be determined. It should be noted that the region type of each output feature data block can be determined based on the region type to which the corresponding input feature data block belongs.

[0039] S260. Based on the region type of each output feature data block, determine the number of effective pooling kernels for each output feature data block during the one-dimensional pooling process along the first spatial dimension.

[0040] The effective pooling kernel number can refer to the number of pooling kernel windows that actually participate in pooling calculations and cover effective feature data.

[0041] When performing pooling operations, if a padded mode is used, the number of effective elements in each output feature data block at the edge positions is not consistent, requiring dynamic calculation of the number of effective pooling kernels, which further increases the overall computational complexity. Therefore, after determining the region type of each output feature data block in the output feature map corresponding to each target's original input feature map, the number of effective pooling kernels for each output feature data block during one-dimensional pooling along the first spatial dimension can be calculated based on the region type of the output feature data block. In a specific example, when performing one-dimensional pooling operations along the height direction, pre-padding only affects a few rows at the top and bottom of the output feature map. Therefore, when performing one-dimensional pooling operations along the height direction, the number of effective pooling kernels can be determined by the region type of the output feature data block, allowing the unpadded intermediate segments to utilize the complete first current pooling kernel KH to the maximum extent, fully expanding the inner KH loop, avoiding the performance overhead caused by dynamic truncation, and ultimately improving the overall computational efficiency. Here, KH is the dimension of the first current pooling kernel in the height direction.

[0042] In an optional embodiment of the present invention, determining the number of effective pooling kernels for each output feature data block during one-dimensional pooling processing along the first spatial dimension, based on the region type of each output feature data block, may include: when the region type of the target output feature data block is determined to be the original region, determining the number of effective pooling kernels for the target output feature data block based on the size of the first current pooling kernel used in the one-dimensional pooling processing operation along the first spatial dimension; when the region type of the target output feature data block is determined to be the filled region, dynamically determining the number of effective pooling kernels for the target output feature data block based on the size of the first current pooling kernel in the first spatial dimension, the coordinate data of the target input feature data block in the first spatial dimension, and the expansion rate of the first current pooling kernel in the first spatial dimension.

[0043] The target output feature data block can be the feature data block currently to be calculated in the output feature map. The target input feature data block can be the input feature data block at the bottom of the first current pooling kernel in the input feature data block corresponding to the target output feature data block.

[0044] In this embodiment of the invention, when determining the number of effective pooling kernels for each output feature data block during one-dimensional pooling processing along the first spatial dimension based on the region type of each output feature data block, if the region type of the target input feature data block is an original region, it indicates that the data covered by the first current pooling kernel is all located in the original region of the target original input feature map. Therefore, the size of the first current pooling kernel can be used as the number of effective pooling kernels for the target output feature data block. If the region type of the target output feature data block is a filled region, it indicates that the data covered by the first current pooling kernel includes a portion located in the filled region of the target original input feature map. Therefore, the number of effective pooling kernels can be dynamically determined based on the size of the first current pooling kernel in the first spatial dimension, the coordinate data of the target input feature data block in the first spatial dimension, and the expansion rate of the first current pooling kernel in the first spatial dimension. It should be noted that the filled region can be further divided into a starting filled region and an ending filled region. The starting filled region is located at the top of the output feature map, and the ending filled region is located at the tail of the output feature map.

[0045] In an optional embodiment of the present invention, dynamically determining the number of effective pooling kernels for the target output feature data block based on the size of the first current pooling kernel in the first spatial dimension, the coordinate data of the target input feature data block in the first spatial dimension, and the expansion rate of the first current pooling kernel in the first spatial dimension may include: when it is determined that the target output feature data block is located in the initial filling region of the output feature map, calculating the number of effective pooling kernels for the target output feature data block based on the following formula: ; in, The number of effective pooling kernels for the target output feature data block when it is located in the initial filling region of the output feature map. The dimension of the first current pooling kernel in the first spatial dimension direction. The coordinates of the target input feature data block in the first spatial dimension direction are given. The expansion rate of the first current pooling kernel in the first spatial dimension direction.

[0046] In an optional embodiment of the present invention, the step of dynamically determining the number of effective pooling kernels for the target output feature data block based on the size of the first current pooling kernel in the first spatial dimension, the coordinate data of the target input feature data block in the first spatial dimension, and the expansion rate of the first current pooling kernel in the first spatial dimension may further include: when it is determined that the target output feature data block is located in the end-filled region of the output feature map, calculating the number of effective pooling kernels for the target output feature data block based on the following formula: ; in, The number of effective pooling kernels for the target output feature data block when it is located in the end-filled region of the output feature map. The dimension of the original input feature map of the target in the first spatial dimension direction. The coordinates of the target's original input feature data block in the first spatial dimension direction. The expansion rate of the first current pooling kernel in the first spatial dimension direction is given by the expansion rate correction term introduced when calculating the number of effective pooling kernels for the standard output feature data block, which can ensure the accuracy of the calculation when the expansion rate is greater than 1.

[0047] S270. Based on the number of effective pooling kernels for each of the output feature data blocks, perform one-dimensional pooling processing in parallel along the first spatial dimension of each of the target original input feature maps to obtain intermediate pooling output results for each of the target original input feature maps.

[0048] Specifically, after determining the number of effective pooling kernels for each output feature data block, one-dimensional pooling processing can be performed in parallel along the first spatial dimension of each target original input feature map according to the number of effective pooling kernels for each output feature data block. Pooling operation is only performed on effective input feature data blocks that fall into the original region, and invalid data in the filling region is ignored. Thus, the intermediate pooling output results of each target original input feature map after one-dimensional pooling processing can be obtained.

[0049] Optionally, when performing one-dimensional pooling on the original input feature map of the target along the height direction, if one row is processed at a time... For continuous data, the intermediate pooling output can be... The intermediate pooling output result. The width of the original input feature map of the target. The number of channels in the original input feature map of the target. This is the height of the output feature map.

[0050] Optionally, two independent dedicated areas can be set up in local memory for each sub-thread: one for caching the asynchronously transported target raw input feature maps, and the other for storing the intermediate pooling output results generated by the pooling calculation. Furthermore, after completing the one-dimensional pooling processing along the first spatial dimension, the next batch of target raw input feature maps can be asynchronously transported to local memory via the data transformation engine; that is, the data transport operation can be executed in parallel with the one-dimensional pooling processing along the second spatial dimension.

[0051] S280. Determine the initial address for reading data from each of the second current pooling kernels during the parallel one-dimensional pooling process along the second spatial dimension direction based on the original input feature maps of each target.

[0052] The initial address for data reading can be used to indicate the starting position for reading the input feature data block when performing one-dimensional pooling processing along the second spatial dimension.

[0053] Correspondingly, after obtaining the intermediate pooling outputs of each target's original input feature map, one-dimensional pooling processing can be further performed in parallel along the second spatial dimension based on each intermediate pooling output. During the parallel execution of one-dimensional pooling processing along the second spatial dimension on each intermediate pooling output, the initial address for data reading of the second current pooling kernel during one-dimensional pooling processing can first be determined. In a specific example, access can begin from the iw=0 position of the target's original input feature map or the intermediate pooling output, thereby covering the left-side filled region of the target's original input feature map. This ensures that boundary checks in this dimension are not required when performing one-dimensional pooling processing along the second spatial dimension, significantly improving the execution efficiency of the instruction pipeline.

[0054] S290. Starting from the initial address of the data reading of each of the second current pooling kernels, sequentially read the input feature data blocks of each intermediate pooling output result, and perform one-dimensional pooling processing on the input feature data blocks of each intermediate pooling output result in parallel to obtain each output feature map.

[0055] Accordingly, after determining the initial address for reading data of the second current pooling kernel, the input feature data blocks of each intermediate pooling output result can be read sequentially from the initial address for reading data of each second current pooling kernel, and one-dimensional pooling processing can be performed in parallel along the second spatial dimension of each intermediate pooling output result, thereby obtaining the output feature map corresponding to the original input feature map of each target.

[0056] In an optional embodiment of the present invention, the feature data pooling method may further include: determining reusable data blocks of each of the target original input feature maps in the current pooling process based on the size of the current pooling kernel and the pooling step size during the one-dimensional pooling process; and directly calling the reusable data blocks of each of the target original input feature maps to perform one-dimensional pooling process on each of the target original input feature maps when performing the next round of one-dimensional pooling process.

[0057] The reusable data block can be a data block in the original input feature map that does not need to be read and loaded repeatedly. The current pooling kernel includes the first current pooling kernel and the second current pooling kernel.

[0058] Specifically, during the two-stage one-dimensional pooling process on the target original input feature map, the overlapping input feature data blocks in adjacent pooling operations can be used as reusable data blocks based on the size of the current pooling kernel and the pooling step size. When performing the next round of one-dimensional pooling operation, the reusable data blocks of each target original input feature map can be directly called, thereby eliminating the need to repeatedly read overlapping data blocks that have already been read in the target original input feature map, reducing the number of external memory reads and the amount of data transfer.

[0059] In a specific example, suppose the current pooling kernel size is With a pooling step size of 1, the input feature data block corresponding to the output position ow=j is [input[2j], input[2j+1], input[2j+2]], and the input feature data block corresponding to the position ow=j+1 is [input[2j+1], input[2j+2], input[2j+3]]. Then, input[2j+1] and input[2j+2] are reusable data blocks for the two adjacent pooling operations.

[0060] Optionally, the pooling type, pooling kernel type, and boundary type parameters and execution strategy for the pooling method of the aforementioned feature data can be determined during compilation using specific programming language template parameters to eliminate all runtime branches. Simultaneously, vector instructions can be used in all inner loop operations to fully utilize the vector width supported by the hardware.

[0061] This invention, in its embodiments, determines the size of the second current pooling kernel used for one-dimensional pooling processing along the second spatial dimension, and determines the preset number of fill columns for the original input feature map based on the size of the second current pooling kernel. After determining the preset number of fill columns for the original input feature map, the original input feature map is filled with the target fill value of the preset number of fill columns to obtain the target original input feature map. Further, the region type of each output feature data block in the output feature map corresponding to each target original input feature map is determined, and based on the region type of each output feature data block, the number of effective pooling kernels for each output feature data block during one-dimensional pooling processing along the first spatial dimension is determined. Then, based on the number of effective pooling kernels for each output feature data block, one-dimensional pooling processing is performed in parallel along the first spatial dimension for each target original input feature map to obtain the intermediate pooling output results of each target original input feature map. After obtaining the intermediate pooling outputs of the original input feature maps of each target, the initial data read addresses of each second current pooling kernel are determined based on the original input feature maps of each target during the parallel one-dimensional pooling process along the second spatial dimension. Then, starting from the initial data read addresses of each second current pooling kernel, the input feature data blocks of each intermediate pooling output are sequentially read, and the input feature data blocks of each intermediate pooling output are subjected to parallel one-dimensional pooling processing to obtain the output feature maps. This method solves the problems of memory fragmentation, insufficient computational granularity, and low utilization of vector parallel resources in existing pooling processing methods. It optimizes pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of the pooling operator.

[0062] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0063] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions.

[0064] It should be noted that any arrangement or combination of the technical features in the above embodiments also falls within the protection scope of this invention.

[0065] Example 3 Figure 3 This is a schematic diagram of a feature data pooling device provided in Embodiment 3 of the present invention, as shown below. Figure 3As shown, the device includes: a target original input feature map transport module 310, a first spatial dimension directional pooling processing module 320, and a second spatial dimension directional pooling processing module 330, wherein: The target original input feature map transport module 310 is used to transport each target original input feature map to local memory through the data transformation engine.

[0066] The first spatial dimension direction pooling processing module 320 is used to perform one-dimensional pooling processing on each of the target original input feature maps in parallel along the first spatial dimension direction in the vector registers of each sub-thread in the local memory, so as to obtain the intermediate pooling output results of each of the target original input feature maps.

[0067] The second spatial dimension direction pooling processing module 330 is used to perform one-dimensional pooling processing on the intermediate pooling output results of each target original input feature map in parallel along the second spatial dimension direction of the intermediate pooling output results of each target original input feature map, so as to obtain the output feature map corresponding to each target original input feature map.

[0068] This invention addresses the issues of memory fragmentation, insufficient computational granularity, and low vector parallelism resource utilization in existing pooling methods. It optimizes pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of the pooling operator. The method also optimizes pooling computation scheduling and memory access patterns, thereby improving the computational efficiency and throughput of the pooling operator.

[0069] Optionally, the data layout format of the target original input feature map is NHWC. The above device may further include a pre-filling module, used to determine the size of the second current pooling kernel used for one-dimensional pooling processing along the second spatial dimension; determine the preset number of fill columns of the original input feature map according to the size of the second current pooling kernel; fill the original input feature map with the target fill value of the preset number of fill columns to obtain the target original input feature map.

[0070] Optionally, the first spatial dimension direction pooling processing module 320 is specifically used to: determine the region type of each output feature data block in the output feature map corresponding to each of the target original input feature maps; wherein, the region type includes filled regions and original regions; determine the number of effective pooling kernels for each of the output feature data blocks during one-dimensional pooling processing along the first spatial dimension direction according to the region type of each of the output feature data blocks; and perform one-dimensional pooling processing in parallel along the first spatial dimension direction on each of the target original input feature maps according to the number of effective pooling kernels for each of the output feature data blocks to obtain intermediate pooling output results of each of the target original input feature maps.

[0071] Optionally, the first spatial dimension direction pooling processing module 320 is further configured to: when the region type of the target output feature data block is determined to be the original region, determine the number of effective pooling kernels of the target output feature data block based on the size of the first current pooling kernel used for one-dimensional pooling processing operation along the first spatial dimension direction; when the region type of the target output feature data block is determined to be the filled region, dynamically determine the number of effective pooling kernels of the target output feature data block based on the size of the first current pooling kernel in the first spatial dimension direction, the coordinate data of the target input feature data block in the first spatial dimension direction, and the expansion rate of the first current pooling kernel in the first spatial dimension direction.

[0072] Optionally, the first spatial dimension directional pooling processing module 320 is further configured to: when it is determined that the target output feature data block is located in the initial filling region of the output feature map, calculate the number of effective pooling kernels for the target output feature data block based on the following formula: ; in, The number of effective pooling kernels for the target output feature data block when it is located in the initial filling region of the output feature map. The dimension of the first current pooling kernel in the first spatial dimension direction. The coordinates of the target input feature data block in the first spatial dimension direction are given. The expansion rate of the first current pooling kernel in the first spatial dimension direction.

[0073] Optionally, the first spatial dimension directional pooling processing module 320 is further configured to: when it is determined that the target output feature data block is located in the end-filled region of the output feature map, calculate the number of effective pooling kernels for the target output feature data block based on the following formula: ; in, The number of effective pooling kernels for the target output feature data block when it is located in the end-filled region of the output feature map. The dimension of the original input feature map of the target in the first spatial dimension direction. The coordinates of the target's original input feature data block in the first spatial dimension direction. The expansion rate of the first current pooling kernel in the first spatial dimension direction.

[0074] Optionally, the second spatial dimension direction pooling processing module 330 is specifically used to: determine the initial address for reading data of each of the second current pooling kernels during the parallel one-dimensional pooling processing along the second spatial dimension direction based on the original input feature maps of each target; starting from the initial address for reading data of each of the second current pooling kernels, sequentially read the input feature data blocks of each of the intermediate pooling output results, and perform one-dimensional pooling processing on the input feature data blocks of each of the intermediate pooling output results in parallel to obtain each of the output feature maps.

[0075] Optionally, the above-mentioned device may further include a data block reuse module, which is used to determine the reusable data blocks of each of the target original input feature maps in the current pooling process based on the size of the current pooling kernel and the pooling step size during the one-dimensional pooling process; and to directly call the reusable data blocks of each of the target original input feature maps to perform one-dimensional pooling process on each of the target original input feature maps when performing the next round of one-dimensional pooling process.

[0076] The aforementioned feature data pooling device can execute the feature data pooling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the feature data pooling method provided in any embodiment of the present invention.

[0077] Since the feature data pooling device described above is an apparatus capable of executing the feature data pooling method in the embodiments of the present invention, those skilled in the art can understand the specific implementation and various variations of the feature data pooling device in this embodiment based on the feature data pooling method described in the embodiments of the present invention. Therefore, how the feature data pooling device implements the feature data pooling method in the embodiments of the present invention will not be described in detail here. Any apparatus used by those skilled in the art to implement the feature data pooling method in the embodiments of the present invention falls within the scope of protection of this application.

[0078] Example 4 Figure 4A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0079] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0080] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0081] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as pooling methods for feature data.

[0082] In some embodiments, the feature data pooling method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the feature data pooling method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the feature data pooling method by any other suitable means (e.g., by means of firmware).

[0083] Optionally, the feature data pooling method may include: transferring each target original input feature map to local memory through a data transformation engine; performing one-dimensional pooling processing on each target original input feature map in parallel along the first spatial dimension direction in the vector register of each sub-thread in the local memory to obtain intermediate pooling output results of each target original input feature map; and performing one-dimensional pooling processing on the intermediate pooling output results of each target original input feature map in parallel along the second spatial dimension direction to obtain output feature maps corresponding to each target original input feature map.

[0084] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0086] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0087] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0088] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0089] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0090] This application also discloses a computer program product, which includes a computer program that, when executed by a processor, implements the feature data pooling method provided in any embodiment of this application. This program product and the feature data pooling methods disclosed in the embodiments of this application belong to the same inventive concept, and therefore will not be described in detail here.

[0091] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0092] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A pooling method for feature data, characterized in that, include: The original input feature maps of each target are moved to local memory through the data transformation engine; In the vector registers of each sub-thread in the local memory, one-dimensional pooling is performed in parallel on each of the original target input feature maps along the first spatial dimension direction to obtain intermediate pooling output results of each of the original target input feature maps; Along the second spatial dimension of the intermediate pooling output of each of the original target input feature maps, one-dimensional pooling is performed in parallel on the intermediate pooling output of each of the original target input feature maps to obtain the output feature map corresponding to each of the original target input feature maps; The step of performing parallel one-dimensional pooling processing on each of the target original input feature maps along the first spatial dimension direction in the vector registers of each sub-thread in the local memory includes: Determine the region type of each output feature data block in the output feature map corresponding to each of the original input feature maps of the target; wherein, the region type includes filled regions and original regions; Based on the region type of each output feature data block, determine the number of effective pooling kernels for each output feature data block during the one-dimensional pooling process along the first spatial dimension. Based on the number of effective pooling kernels for each of the output feature data blocks, one-dimensional pooling is performed in parallel along the first spatial dimension of each target original input feature map to obtain intermediate pooling output results for each target original input feature map; The step of determining the number of effective pooling kernels for each output feature data block during one-dimensional pooling processing along the first spatial dimension, based on the region type of each output feature data block, includes: When the region type of the target output feature data block is determined to be the original region, the number of effective pooling kernels of the target output feature data block is determined according to the size of the first current pooling kernel used for one-dimensional pooling processing operation along the first spatial dimension direction. When the region type of the target output feature data block is determined to be the filled region, the effective number of pooling kernels of the target output feature data block is dynamically determined based on the size of the first current pooling kernel in the first spatial dimension direction, the coordinate data of the target input feature data block in the first spatial dimension direction, and the expansion rate of the first current pooling kernel in the first spatial dimension direction.

2. The method according to claim 1, characterized in that, The data layout format of the target original input feature map is batch number-height-width-channel number NHWC. Before the target original input feature map is moved to local memory through the data transformation engine, the following steps are also included: Determine the size of the second current pooling kernel used for one-dimensional pooling operations along the second spatial dimension; The preset number of padding columns for the original input feature map is determined based on the size of the second current pooling kernel; The original input feature map is filled with the target fill value of the preset number of fill columns to obtain the target original input feature map.

3. The method according to claim 1, characterized in that, The step of dynamically determining the number of effective pooling kernels for the target output feature data block based on the size of the first current pooling kernel in the first spatial dimension, the coordinate data of the target input feature data block in the first spatial dimension, and the expansion rate of the first current pooling kernel in the first spatial dimension includes: Given that the target output feature data block is located in the initial filling region of the output feature map, the effective pooling kernel number of the target output feature data block is calculated based on the following formula: ； in, The number of effective pooling kernels for the target output feature data block when it is located in the initial filling region of the output feature map. The dimension of the first current pooling kernel in the first spatial dimension direction. The coordinates of the target input feature data block in the first spatial dimension direction are given. The expansion rate of the first current pooling kernel in the first spatial dimension direction.

4. The method according to claim 3, characterized in that, The step of dynamically determining the number of effective pooling kernels for the target output feature data block based on the size of the first current pooling kernel in the first spatial dimension, the coordinate data of the target input feature data block in the first spatial dimension, and the expansion rate of the first current pooling kernel in the first spatial dimension further includes: Given that the target output feature data block is located in the end-filled region of the output feature map, the effective pooling kernel number of the target output feature data block is calculated based on the following formula: ； in, The number of effective pooling kernels for the target output feature data block when it is located in the end-filled region of the output feature map. The dimension of the original input feature map of the target in the first spatial dimension direction. The coordinates of the target's original input feature data block in the first spatial dimension direction. The expansion rate of the first current pooling kernel in the first spatial dimension direction.

5. The method according to claim 2, characterized in that, The process of performing one-dimensional pooling on the intermediate pooling outputs of each of the original target input feature maps in parallel along the second spatial dimension direction includes: Based on the original input feature maps of each target, determine the initial address for data reading of each of the second current pooling kernels during the parallel one-dimensional pooling process along the second spatial dimension; Starting from the initial address of the data read of each of the second current pooling kernels, the input feature data blocks of each intermediate pooling output result are read sequentially, and one-dimensional pooling processing is performed in parallel on the input feature data blocks of each intermediate pooling output result to obtain each output feature map.

6. The method according to claim 1, characterized in that, The method further includes: Based on the size of the current pooling kernel and the pooling step size during the one-dimensional pooling process, determine the reusable data blocks of each target original input feature map in the current pooling process; When performing the next round of one-dimensional pooling processing, the reusable data blocks of each of the target original input feature maps are directly called to perform one-dimensional pooling processing on each of the target original input feature maps.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that is executed by the at least one processor to enable the at least one processor to perform the feature data pooling method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the pooling method for feature data as described in any one of claims 1-6.

9. A computer program product, characterized in that, Includes a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the pooling method for feature data according to any one of claims 1-6.

Images