Pixel-parallel depth calculation implementation method, device, medium, equipment and product

Abstract

The present invention discloses a method, device, medium, equipment and product for implementing pixel-parallel depth operation, the method comprising: dividing the input tensor into m channel groups and assigning them to N channel groups respectively. T The thread bundles of each thread are processed independently; the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting point coordinates of each element initially covered are obtained; a register group equal to the number of elements is allocated to each thread; each register group is used to store a pixel group of the corresponding element; in the channel group, starting from each row and column starting point coordinate, according to the sliding step size, scan N rows by row T The present invention allocates independent computational tasks to threads through pixel parallelism, maximizing the utilization of allocated hardware resources.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, computer-readable storage medium, electronic device, and computer program product for implementing pixel-parallel depth operations. Background Art

[0002] When performing deep operations on artificial intelligence processors, such as depth-wise convolution, pooling, and grouped convolution, existing methods use a channel parallel strategy between threads. This reduces the computing power utilization of the artificial intelligence processor due to the mismatch between thread scheduling granularity and task allocation.

[0003] Taking depth-wise convolution as an example, the existing method assigns a single channel convolution calculation task to each thread. T threads, when the number of channels C of the input tensor cannot be N T When divided evenly, some threads in a thread bundle will be idle, that is, the enabled threads cannot be fully utilized, and the computing power utilization of the AI processor will be reduced to . Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device, computer-readable storage medium, electronic device and computer program product for implementing pixel-parallel depth calculations, which realize independent computing task allocation at the thread level through a pixel parallel strategy between threads, and allocate corresponding local sliding window field of view computing tasks to each thread, thereby maximizing the utilization of allocated hardware resources and improving computing efficiency.

[0005] A first embodiment of the present invention provides a pixel-parallel depth calculation implementation method, including:

[0006] The input tensor is divided into m channel groups and assigned to m thread warps for independent processing; each thread warp contains N T threads; each channel group contains N C Pixels of channels; N T ≥1; N C ≥1;m≥1;

[0007] Get the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting coordinates of each element initially covered;

[0008] Allocating register groups equal to the number of elements to each thread; wherein each register group is used to store a pixel group corresponding to the element; each pixel group is composed of all pixels at the same row and column position in the corresponding channel group;

[0009] In the channel group, starting from each row and column starting coordinate, with the sliding step length as the span, scan N rows T row and column positions, and scan the N T The pixel groups at the row and column positions are sequentially loaded into the corresponding register groups of each thread in the warp;

[0010] Each thread calculates the loaded pixel group to obtain an output result within the corresponding local sliding window field of view.

[0011] Optionally, the input tensor consists of a feature data block, or is obtained by splicing multiple feature data blocks in the vertical direction of the original space, or is composed of any sub-data block split from a feature data block.

[0012] Optionally, the sliding window operator is a pooling operator;

[0013] The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes:

[0014] In any of the thread warps, each thread performs a channel-isolated pooling operation on the loaded pixel group to obtain N pixels within the corresponding local sliding window field of view. C The pooling result of each channel.

[0015] Optionally, the sliding window operator is a single-channel convolution kernel;

[0016] The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes:

[0017] Broadcasting the convolution weight corresponding to each register group to all threads in the corresponding warp in scalar form;

[0018] In any of the thread bundles, each thread performs channel-isolated multiplication and addition operations on the loaded pixel groups and convolution weights to obtain the corresponding N local sliding window field of view. C The convolution result of channels.

[0019] Optionally, the sliding window operator is a multi-channel convolution kernel;

[0020] The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes:

[0021] Broadcasting the convolution weight corresponding to each register group to all threads in the corresponding warp in scalar form;

[0022] In any of the thread bundles, each thread performs a channel fusion multiplication and addition operation on the loaded pixel group and the convolution weight to obtain an output feature value within the corresponding local sliding window field of view; wherein each of the channel groups is used to represent an input group in the grouped convolution operation.

[0023] Optionally, the method further includes:

[0024] Before the thread performs the multiplication and addition operation, an out-of-bounds judgment is performed on the coordinates of the pixel values in the pixel group, and the pixel values corresponding to the out-of-bounds coordinates are masked to specified values to obtain new pixel values.

[0025] A second embodiment of the present invention provides a device for implementing pixel-parallel depth calculation, including:

[0026] The thread warp configuration module is used to divide the input tensor into m channel groups and assign them to m thread warps for independent processing; each thread warp contains N T threads; each channel group contains N C Pixels of channels; N T ≥1; N C ≥1;m≥1;

[0027] The sliding window parameter acquisition module is used to obtain the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting coordinates of each element initially covered;

[0028] A thread configuration module is configured to allocate register groups equal to the number of elements to each of the threads; wherein each register group is used to store a pixel group corresponding to the element; and each pixel group is composed of all pixels at the same row and column position in the corresponding channel group;

[0029] The pixel loading module is used to scan N pixels in each channel group starting from the starting coordinates of each row and column, with the sliding step length as the span. T row and column positions, and scan the N T The pixel groups at the row and column positions are sequentially loaded into the corresponding register groups of each thread in the warp;

[0030] The thread calculation module is used to calculate the loaded pixel group through each thread to obtain the output result within the corresponding local sliding window field of view.

[0031] An embodiment of the third aspect of the present invention provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the pixel-parallel depth operation implementation method described in any one of the first aspects above.

[0032] An embodiment of a fourth aspect of the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the pixel-parallel depth operation implementation method described in any one of the first aspects above.

[0033] An embodiment of the fifth aspect of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the pixel-parallel depth operation implementation method described in any one of the first aspects above when executing the computer program.

[0034] Compared with the prior art, the embodiments of the present invention provide a pixel-parallel depth operation implementation method, device, computer-readable storage medium, electronic device and computer program product, which have the following beneficial effects: the embodiments of the present invention first divide the input tensor into multiple channel groups and assign them to different thread warps; then, obtain the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting point coordinates of each element initially covered; then, allocate a corresponding number of register groups to each thread, each register group stores a pixel group of the corresponding element; then, starting from each row and column starting point coordinate of each channel group, with the sliding step size as the span, scan N rows by row T The scanned pixel groups are sequentially loaded into the register groups of the corresponding threads, allowing each thread to process the computational tasks for the corresponding local sliding window field of view. Therefore, the embodiments of the present invention implement independent computational task allocation at the thread level through a pixel parallelization strategy between threads. By assigning each thread the computational tasks for the corresponding local sliding window field of view, the allocated hardware resources can be maximized, thereby improving computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of an embodiment of the pixel-parallel depth calculation implementation provided by the present invention.

[0036] Figure 2 1 is a schematic diagram of an embodiment of a sliding window operator provided by the present invention sliding in an input tensor;

[0037] Figure 3 is a schematic diagram of an embodiment of the projection of an input tensor on a two-dimensional plane provided by the present invention;

[0038] Figure 4 is a schematic diagram of an embodiment of the mapping relationship between thread warps and pixel groups provided by the present invention;

[0039] Figure 5 is a schematic diagram of an embodiment of loading a pixel group into a thread warp provided by the present invention;

[0040] Figure 6 is a schematic diagram of another embodiment of loading a pixel group into a warp provided by the present invention;

[0041] Figure 7 This is an example diagram of channel parallelism when threads perform depth-wise convolution operations provided by the prior art;

[0042] Figure 8 1 is a schematic structural diagram of an embodiment of a device for implementing pixel-parallel depth calculation provided by the present invention;

[0043] Figure 9 It is a structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this technical field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0045] The artificial intelligence processor involved in the present invention can be any one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), an NPU (Neural Network Processing Unit), a DPU (Deep Learning Processing Unit), an APU (Accelerated Processing Unit), and a GPGPU (General-Purpose computing on Graphics Processing Unit), and is determined when the embodiment of the present invention is applied to a specific product or technology.

[0046] Furthermore, in the embodiments of the present invention, the so-called "pixel" is not an image pixel value in the traditional sense, but an abstract unit used to represent any type of data. Specifically, a "pixel" corresponds to a data unit at a certain position in the input tensor.

[0047] The following uses a GPU as an example to illustrate the pixel-parallel depth calculation implementation method, device, computer-readable storage medium, electronic device, and computer program product provided by the embodiments of the present invention.

[0048] See also Figure 1 , is a flow chart of an embodiment of pixel-parallel depth calculation implementation provided by the present invention.

[0049] The first embodiment of the present invention provides a pixel-parallel depth calculation implementation method, including steps S1 to S5, as follows:

[0050] Step S1: Divide the input tensor into m channel groups and assign them to m thread warps for independent processing; each thread warp contains N T threads; each channel group contains N C Pixels of channels; N T ≥1; N C ≥1;m≥1;

[0051] Step S2: Obtain the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting point coordinates of each element initially covered;

[0052] Step S3: allocating register groups equal to the number of elements to each thread; wherein each register group is used to store a pixel group corresponding to the element; each pixel group is composed of all pixels at the same row and column position in the corresponding channel group;

[0053] Step S4: In the channel group, starting from the starting coordinates of each row and column, with the sliding step length as the span, scan N rows T row and column positions, and scan the N T The pixel groups at the row and column positions are sequentially loaded into the corresponding register groups of each thread in the warp;

[0054] Step S5: Calculate the loaded pixel group through each thread to obtain the output result within the corresponding local sliding window field of view.

[0055] It should be noted that the sliding window operator in the embodiment of the present invention includes: a pooling operator, a single-channel convolution kernel (which can be used for depth-wise convolution operations) or a multi-channel convolution kernel (which can be used for grouped convolution operations).

[0056] In step S1, the input tensor of dimension H×W×C is grouped along the channel dimension to generate m channel groups, each containing N C Pixels of consecutive channels; where H is the height of the input tensor, W is the width of the input tensor, and C is the total number of channels of the input tensor; the dimension of each channel group is H×W×N C Furthermore, an independent thread warp is assigned to each channel group; wherein each thread warp contains N Tthreads (such as 32 threads).

[0057] In step S2, the attribute parameters of the sliding window operator on the two-dimensional plane are obtained, including: the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the row and column starting coordinates of each element initially covered, and the sliding step size; wherein, the two-dimensional plane is a plane determined by the vertical and horizontal directions of the sliding window operator or the input tensor, and has nothing to do with the depth / channel dimension. Figure 2 , which is a schematic diagram of an embodiment of a sliding window operator provided by the present invention sliding in an input tensor. Figure 2 Medium pixel data The coordinates are , that is, data block The Rank columns, which is also independent of the channel dimension; , , ,and and Co-locating pixel data The actual row position on the two-dimensional plane (ie row position = + -1). Figure 3 FIG. 1 is a schematic diagram of an embodiment of the projection of an input tensor on a two-dimensional plane provided by the present invention. Figure 3 middle, is the row and column position coordinates of the pixel data; the sliding window operator is a 3×3 single-channel convolution kernel, the number of elements involved in the coverage operation in the two-dimensional plane is 9, and the element is the convolution weight ;in, is the vertical offset between the convolution weight and the center of the convolution kernel, is the horizontal offset between the convolution weight and the center of the convolution kernel. Therefore, the 9 elements are , , , , , , , and .

[0058] like Figure 2 and Figure 3 As shown, the center point is The gray area ① is used as the local sliding window field of view (initial coverage area) of the 3×3 convolution kernel in the current calculation batch. The starting coordinates of the rows and columns are , weight The starting coordinates of the rows and columns are , weight The starting coordinates of the rows and columns are , ..., weight The starting coordinates of the rows and columns are , that is, the row and column starting coordinates of the sliding window operator on the two-dimensional plane are { , , , , , , , , }; The sliding step is 1, that is, from the center point Gray area ①, slide 1 step by row, and get the center The dotted line encloses the area ②. Since the above coordinates are independent of the depth (channel) dimension, the 3×3 single-channel convolution kernel is expanded to a multi-channel convolution kernel (such as 3×3×N C ), its attribute parameters on the two-dimensional plane remain consistent with the above content.

[0059] In step S3, K is allocated to each thread in each warp. H ×K W A dedicated register group forms a sliding window data buffer area; among them, K H is the height of the sliding window operator, K W is the width of the sliding window operator. For example, for a 3×3 convolution kernel, each thread will be allocated 9 dedicated register groups. Each register group is used to store N at the same row and column position. C In the case where the register can store 32-bit floating point numbers and each pixel value is a 16-bit floating point number, if N C =2, each register group (labeled Rx) consists of only one register (labeled rx); if N C >2, each register group consists of registers. Figure 4 FIG. 1 is a schematic diagram of an embodiment of the mapping relationship between thread warps and pixel groups provided by the present invention. Figure 4 In the example, the input tensor contains 8 channels (i.e., C0, C1, ..., C7), according to N C = 2, the input tensor is divided into 4 channel groups, so 4 thread warps are enabled, and any pixel group processed by each thread in the thread warp is the 2-channel pixel at the same row and column position in the corresponding channel group.

[0060] In addition, there is a mapping relationship between each register group in the thread and the corresponding element in the sliding window operator. Figure 5FIG. 1 is a schematic diagram of an embodiment of the present invention for loading pixel groups into thread warps. Figure 5 In the example, the sliding window operator is a 3×3 convolution kernel, and warp 0 processes a channel group containing channels C0 and C1. Each thread has 9 dedicated register groups (i.e., R0, R1, …, R8); each register group corresponds to a group ( , ) of the convolution weights. For example, register group R0 (in N C =2 can be represented by register r1) used to store a set of convolution weights with an offset of (-1, -1) { , }Corresponding pixel group (The coordinates are The corresponding two channel pixels); similarly, register group R1 is used to store a set of offsets The convolution weights { , }Corresponding pixel group From this we can see that a thread in any warp can hold N corresponding to a local sliding window view of the sliding window operator. C channel pixels.

[0061] In step S4, in the i-th channel group, the j-th row and column starting point coordinate in the row and column starting point coordinate set of the sliding window operator is used as the starting point of the j-th row scanning route, and the sliding step length is used as the span to scan N T The starting coordinates of the rows and columns, and then these N T The pixel group at the row and column position is loaded into the jth register group of each thread in the i-th thread warp; where 1≤i≤m; 1≤j≤K H ×K W .

[0062] In order to more clearly describe the technical solutions provided by the embodiments of the present invention, some specific embodiments are provided below for reference:

[0063] Example 1: Figure 2 、 3 As shown in Figure 5, the pixel group of the first channel group (corresponding to channels C0 and C1) is loaded into the 32 threads of thread warp 0. For the first register group R0 (or register r0) exclusive to each of the 32 threads, the first row and column starting point coordinate in the row and column starting point coordinate set of the sliding window operator is obtained. Start by sliding the span of 1 and continuously read 32 row and column coordinates (the last row and column coordinate read is ), and the pixel groups corresponding to these 32 row and column coordinates to Fill in the register group R0 of each of the 32 threads in sequence; for the second register group R1 (or register r1) of each of the 32 threads, the second row and column starting point coordinates in the row and column starting point coordinate set of the sliding window operator are Start by sliding the span of 1 and continuously read 32 row and column coordinates (the last row and column coordinate read is ), and the pixel groups corresponding to these 32 row and column coordinates to Fill the registers R1 of each of the 32 threads in sequence; and continue to load the data of the registers R2 to R8 of each of the 32 threads of warp 0 in a similar manner. Therefore, the local sliding window field of view corresponding to thread T0 is The gray area ① is centered, and the local sliding window field of view corresponding to thread T1 is The dotted line encloses the area ② as the center, ..., the local sliding window field of view corresponding to thread T31 is The dotted line encloses the area ③ as the center.

[0064] Example 2: Figure 6 FIG. 1 is a schematic diagram of another embodiment of the present invention for loading pixel groups into thread warps. Figure 6 In , the input tensor has been stored in the shared memory, each memory unit stores two channel pixels corresponding to a row and column position, and the pixels are marked with the sequence number and channel number according to the order of loading row by row; for example, (p -1 C0, p -1 C1) is equivalent to the above pixel group , and p -1 C0 is the pixel value of channel C0 in the pixel group. In the current calculation batch, the input tensor contains 8 channels (i.e. C0, C1, ..., C7), according to N C =2 to divide the channel group, then enable 4 thread warps as execution units (ExecutiveUnit, EU). Each thread is assigned K H ×K W Dedicated register groups. If the sliding window operator is a 3×3 convolution kernel, each thread is allocated 9 dedicated register groups (i.e., R0, R1, …, R8). In the channel group corresponding to each execution unit, starting from the 9 row and column starting coordinates of the sliding window operator, a 9-way data load operation is performed, continuously loading 32 sets of pixel data in a row-by-row scanning manner, and filling the data into the corresponding numbered register groups.

[0065] In step S5, each thread processes K independently H ×K W ×N C pixel values and supports multiple operation modes, such as convolution and pooling.

[0066] Existing methods use a channel parallel strategy between threads. For example, if the input tensor has 90 channels, two thread warps (128 threads) need to be enabled. However, these two thread warps can only process 90 local sliding window fields of view (corresponding to the computational tasks of 90 channels) when performing a convolution calculation, failing to fully utilize the computing power of hardware resources. In contrast, the embodiment of the present invention uses a pixel parallel strategy between threads, that is, a local sliding window field of view parallel strategy. The above two thread warps can process N corresponding to 128 local sliding window fields of view when performing a convolution calculation. C Therefore, the embodiments of the present invention implement independent computational task allocation at the thread level through a pixel parallel strategy between threads, assigning each thread a computational task corresponding to a local sliding window field of view, thereby maximizing the utilization of allocated hardware resources and improving computational efficiency.

[0067] In an optional embodiment, the input tensor is composed of a feature data block, or is obtained by splicing multiple feature data blocks in the vertical direction of the original space, or is composed of any sub-data block split from a feature data block.

[0068] It should be noted that the input tensors in the embodiments of the present invention can be obtained through the following three main data organization methods, as long as the conditions of warp parallel processing of data and reasonable resource allocation are met:

[0069] (1) Single feature data block method: If the dimension of the feature data block is [H, W, C], and the number of positions that the center point of the local sliding window field of view can cover on the two-dimensional plane formed by the vertical and horizontal directions is N center ≤N T , and if the thread warp resources are sufficient to cover the total number of channels C of the entire feature data block, the input tensor is directly composed of the complete feature data block; this method is suitable for scenarios where the feature data block size is small and the computing resources are sufficient.

[0070] (2) Vertical splicing: If q feature data blocks are vertically spliced along the height dimension, a composite tensor with dimensions [qH, W, C] is formed, and the N corresponding to the composite tensor is center ≤N T , and if the thread warp resources are sufficient to cover the total number of channels C of the entire feature data block, the composite tensor is used as the input tensor; this method is suitable for scenarios where multiple data blocks need to be integrated for unified processing.

[0071] (3) Dynamic splitting method: ① Splitting by row and column position, when the N corresponding to the feature data block center >N T When N center =N T② Split the feature data block perpendicular to the 2D plane into multiple sub-data blocks. If the warp resources cannot cover the total number of channels C in the feature data block, the feature data block is split into multiple sub-data blocks along the channel dimension, based on the warp resource limitations. These sub-data blocks are processed in batches, with one sub-data block used as the input tensor for each batch. This approach is suitable for scenarios with large feature data blocks or limited computing resources, as it allows for efficient utilization of computing resources through splitting.

[0072] It is worth noting that the method for obtaining the input tensor is not limited to the use of the above three methods alone, and can also be any combination of multiple methods, which is not limited in the embodiment of the present invention. For example, multiple feature data blocks can be spliced into a new feature data block along the vertical direction of the original space (such as Figure 2 ), and then satisfy N center =N T The new feature data block is split perpendicular to the two-dimensional plane to obtain multiple input tensors.

[0073] The embodiments of the present invention have a high degree of flexibility in constructing input tensors and can adapt to different feature data scales and computing resource conditions.

[0074] In an optional embodiment, the sliding window operator is a pooling operator;

[0075] The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes:

[0076] In any of the thread warps, each thread performs a channel-isolated pooling operation on the loaded pixel group to obtain N pixels within the corresponding local sliding window field of view. C The pooling result of each channel.

[0077] It should be noted that for each thread, the total dimension of the data composed of all pixel groups loaded is K H ×K W ×N C During the calculation process, the thread will perform independent pooling operations for each channel. Common pooling operations include maximum pooling, average pooling, etc. Taking maximum pooling as an example, any thread will operate on each channel separately within the local sliding window field of view it is responsible for, and select the maximum pixel value in the channel as the pooling result of the channel. In this way, each thread will eventually obtain N pixels within the corresponding local sliding window field of view. C The pooling result of each channel.

[0078] In an optional embodiment, the sliding window operator is a single-channel convolution kernel;

[0079] The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes:

[0080] Broadcasting the convolution weight corresponding to each register group to all threads in the corresponding warp in scalar form;

[0081] In any of the thread bundles, each thread performs channel-isolated multiplication and addition operations on the loaded pixel groups and convolution weights to obtain the corresponding N local sliding window field of view. C The convolution result of channels.

[0082] It should be noted that if Figure 7 The figure shows an example of channel parallelism when threads perform depth-wise convolution operations provided by the prior art. Figure 7 In the example, the total number of channels of the input tensor is 12, so only 12 threads (T0 to T11) participate in the convolution operation task, and the remaining threads (T12 to T31) are idle. Each thread loads the pixel data of the local sliding window field of view in the corresponding channel, and also loads the convolution kernel corresponding to the channel. For example, thread T0 loads the 3×3 convolution kernel W0 corresponding to channel C0, and also loads the coverage pixels of the current convolution kernel W0 in channel C0; similarly, thread T10 loads the pixel data of channel C0. 10 The corresponding 3×3 convolution kernel W10 also loads the current convolution kernel W10 in channel C 10 Obviously, the pixel data and convolution kernels loaded by different threads are different. Therefore, when performing a convolution operation, each thread needs to process three input vector operands: the first is the vector of input feature data, the second is the vector of convolution weights, and the third is the vector of accumulated input. However, the current GPU architecture is more suitable for processing instructions with no more than two vector operands, so the third operand usually needs to wait, resulting in a decrease in instruction execution efficiency. Based on this, the existing technology has high requirements for the read bandwidth of vector registers, which further limits the improvement of computing performance.

[0083] To address this issue, an embodiment of the present invention replaces the input convolution weights (the second vector operand) with scalar operands, reducing the number of input operands for the multiply-add instruction to two vector operands. This improvement not only better adapts to the characteristics of the GPU architecture, enabling the parallel execution of two instructions, but also significantly reduces the read bandwidth requirements of the vector registers. Furthermore, the scalar operands and the two vector operands can be executed synchronously and in parallel, reducing instruction latency and further improving computational efficiency.

[0084] Specifically, in an embodiment of the present invention, each thread in any thread warp holds the data within the local sliding window field of view in its corresponding channel group. Since these sliding window data are in the same channel group, all threads in the thread warp need to access the same convolution weights. Based on this feature, the present invention broadcasts the convolution weights to all threads in the thread warp in scalar form, so that all threads in the same thread warp share the same convolution weights. In other words, the convolution weights are shared within the thread warp in scalar form. Figure 5 As shown, T0 to T31 of warp 0 share K H ×K W ×N C Finally, each thread independently processes the pixels on each channel and the shared convolution weights, performs element-by-element multiplication, and accumulates the results to obtain the convolution result of the local sliding window field of view in the corresponding channel.

[0085] In an optional embodiment, the sliding window operator is a multi-channel convolution kernel;

[0086] The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes:

[0087] Broadcasting the convolution weight corresponding to each register group to all threads in the corresponding warp in scalar form;

[0088] In any of the thread bundles, each thread performs a channel fusion multiplication and addition operation on the loaded pixel group and the convolution weight to obtain an output feature value within the corresponding local sliding window field of view; wherein each of the channel groups is used to represent an input group in the grouped convolution operation.

[0089] It should be noted that in the grouped convolution operation, each channel group is used to represent an input group in the grouped convolution operation. For each input group, the corresponding thread warp will independently complete the calculation of one output channel in the group calculation through a shared multi-channel convolution kernel. For example, assuming that the first channel group is used as the first input group (with dimensions H×W×N C ), the number of output channels of a single group is 5, then the first thread warp needs to complete the calculation of these 5 output channels respectively and share 5 multi-channel convolution kernels (dimension is K H ×K W ×N C ), that is, when performing the calculation of an output channel, the 32 threads of the first thread bundle jointly access a multi-channel convolution kernel.

[0090] During the calculation process, each thread is responsible for processing an output feature value within the local sliding window field of view. H ×K W ×NC The pixel values are multiplied element by element with the corresponding convolution weights, and then all the product values are accumulated, so that the thread outputs a convolution result value as an element in the corresponding output feature layer.

[0091] In an optional embodiment, the method further includes:

[0092] Before the thread performs the multiplication and addition operation, an out-of-bounds judgment is performed on the coordinates of the pixel values in the pixel group, and the pixel values corresponding to the out-of-bounds coordinates are masked to specified values to obtain new pixel values.

[0093] It should be noted that the operation of filling a specified value in the embodiment of the present invention (taking filling 0 as an example) includes but is not limited to the following two methods:

[0094] (1) After the thread register group obtains the pixel group, the coordinates of the center point in the local sliding window field of view and the offset of each convolution weight are used to determine whether the coordinates of the corresponding convolution weight at the current coverage position are out of bounds; if out of bounds, the pixel value corresponding to the out-of-bounds coordinate is masked to a specified value (such as a mask of 0).

[0095] (2) During the scanning operation, while loading the pixel group, it is actively determined whether the pixel coordinates are out of bounds; if so, the pixel value corresponding to the out-of-bounds coordinate is masked to a specified value, and the specified value is written into the corresponding register group; otherwise, the original pixel value is still written into the register group.

[0096] In addition to the above two methods, other suitable masking strategies may be adopted according to actual requirements and hardware architecture, which is not limited in the embodiment of the present invention.

[0097] See also Figure 8 , is a structural diagram of an embodiment of a pixel-parallel depth calculation implementation device provided by the present invention.

[0098] A second embodiment of the present invention provides a device for implementing pixel-parallel depth calculation, including:

[0099] The thread warp configuration module 11 is used to divide the input tensor into m channel groups and assign them to m thread warps for independent processing; wherein each thread warp contains N T threads; each channel group contains N C Pixels of channels; N T ≥1; N C ≥1;m≥1;

[0100] The sliding window parameter acquisition module 12 is used to obtain the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting point coordinates of each element initially covered;

[0101] The thread configuration module 13 is configured to allocate register groups equal to the number of elements to each thread; wherein each register group is used to store a pixel group corresponding to the element; each pixel group is composed of all pixels at the same row and column position in the corresponding channel group;

[0102] The pixel loading module 14 is used to scan N pixels in each channel group starting from the starting coordinates of each row and column, with the sliding step length as the span. T row and column positions, and scan the N T The pixel groups at the row and column positions are sequentially loaded into the corresponding register groups of each thread in the warp;

[0103] The thread calculation module 15 is configured to calculate the loaded pixel group through each thread to obtain an output result within a corresponding local sliding window field of view.

[0104] It should be noted that the pixel-parallel depth operation implementation device provided in the embodiment of the second aspect of the present invention can implement all the processes of the pixel-parallel depth operation implementation method described in any embodiment of the first aspect above. The functions of each module and unit in the device and the technical effects achieved are respectively the same as the functions and technical effects achieved by the pixel-parallel depth operation implementation method described in any embodiment of the first aspect above, and will not be repeated here.

[0105] An embodiment of the third aspect of the present invention provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the pixel-parallel depth operation implementation method described in any embodiment of the first aspect above.

[0106] A fourth aspect of the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the pixel-parallel depth calculation implementation method described in any one of the embodiments of the first aspect.

[0107] See also Figure 9 , is a structural diagram of an embodiment of an electronic device provided by the present invention.

[0108] An embodiment of the fifth aspect of the present invention provides an electronic device, comprising a processor 21, a memory 22, and a computer program stored in the memory 22 and configured to be executed by the processor 21, wherein when executing the computer program, the processor implements the pixel-parallel depth operation implementation method described in any embodiment of the first aspect above.

[0109] Preferably, the computer program can be divided into one or more modules / units (e.g., computer program 1, computer program 2, ...). These one or more modules / units are stored in the memory 22 and executed by the processor 21 to implement the present invention. The one or more modules / units can be a series of computer program instruction segments capable of implementing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device.

[0110] The processor 21 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor, or the processor 21 can be any conventional processor. The processor 21 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the electronic device.

[0111] The memory 22 primarily includes a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function, and the data storage area can store related data. Furthermore, the memory 22 can be a high-speed random access memory or a non-volatile memory, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, or a flash card. Alternatively, the memory 22 can be another volatile solid-state memory device.

[0112] It should be noted that the above electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 9 The structural block diagram shown is only an example of the structure of the above-mentioned electronic device and does not constitute a structural limitation of the above-mentioned electronic device. The above-mentioned electronic device may include more or fewer components than shown in the figure, or combine certain components, or different components.

[0113] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A pixel-parallel depth calculation implementation method, characterized in that: include: The input tensor is divided into m channel groups and assigned to m thread warps for independent processing; each thread warp contains N T threads; each channel group contains N C Pixels of channels; N T ≥1; N C ≥1;m≥1; Get the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting coordinates of each element initially covered; Allocating register groups equal to the number of elements to each thread; wherein each register group is used to store a pixel group corresponding to the element; each pixel group is composed of all pixels at the same row and column position in the corresponding channel group; In the channel group, starting from each row and column starting coordinate, with the sliding step length as the span, scan N rows T row and column positions, and scan the N T The pixel groups at the row and column positions are sequentially loaded into the corresponding register groups of each thread in the warp; Each thread calculates the loaded pixel group to obtain an output result within the corresponding local sliding window field of view.

2. The pixel-parallel depth calculation implementation method according to claim 1, characterized in that: The input tensor is composed of a feature data block, or is obtained by splicing multiple feature data blocks in the vertical direction of the original space, or is composed of any sub-data block split from a feature data block.

3. The pixel-parallel depth calculation implementation method according to claim 1, wherein: The sliding window operator is a pooling operator; The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes: In any of the thread warps, each thread performs a channel-isolated pooling operation on the loaded pixel group to obtain the corresponding N pixels in the local sliding window field of view. C The pooling result of channels.

4. The pixel-parallel depth calculation implementation method according to claim 1, wherein: The sliding window operator is a single-channel convolution kernel; The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes: Broadcasting the convolution weight corresponding to each register group to all threads in the corresponding warp in scalar form; In any of the thread bundles, each thread performs channel-isolated multiplication and addition operations on the loaded pixel groups and convolution weights to obtain the corresponding N local sliding window field of view. C The convolution result of channels.

5. The pixel-parallel depth calculation implementation method according to claim 1, wherein: The sliding window operator is a multi-channel convolution kernel; The step of calculating the loaded pixel group by each thread to obtain an output result within the corresponding local sliding window field of view includes: Broadcasting the convolution weight corresponding to each register group to all threads in the corresponding warp in scalar form; In any of the thread bundles, each thread performs a channel fusion multiplication and addition operation on the loaded pixel group and the convolution weight to obtain an output feature value within the corresponding local sliding window field of view; wherein each of the channel groups is used to represent an input group in the grouped convolution operation.

6. The pixel-parallel depth calculation implementation method according to claim 4 or 5, characterized in that: The method further comprises: Before the thread performs the multiplication and addition operation, an out-of-bounds judgment is performed on the coordinates of the pixel values in the pixel group, and the pixel values corresponding to the out-of-bounds coordinates are masked to specified values to obtain new pixel values.

7. A pixel-parallel depth calculation implementation device, characterized in that: include: The thread warp configuration module is used to divide the input tensor into m channel groups and assign them to m thread warps for independent processing; each thread warp contains N T threads; each channel group contains N C Pixels of channels; N T ≥1; N C ≥1;m≥1; The sliding window parameter acquisition module is used to obtain the number of elements involved in the covering operation of the sliding window operator in the two-dimensional plane, the sliding step size, and the row and column starting coordinates of each element initially covered; A thread configuration module is configured to allocate register groups equal to the number of elements to each of the threads; wherein each register group is used to store a pixel group corresponding to the element; and each pixel group is composed of all pixels at the same row and column position in the corresponding channel group; The pixel loading module is used to scan N pixels in each channel group starting from the starting coordinates of each row and column, with the sliding step length as the span. T row and column positions, and scan the N T The pixel groups at the row and column positions are sequentially loaded into the corresponding register groups of each thread in the warp; The thread calculation module is used to calculate the loaded pixel group through each thread to obtain the output result within the corresponding local sliding window field of view.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program; wherein, when the computer program is run, it controls the device where the computer-readable storage medium is located to execute the pixel-parallel depth operation implementation method according to any one of claims 1 to 6.

9. A computer program product, characterized in that The method comprises a computer program, which, when executed by a processor, implements the pixel-parallel depth calculation implementation method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the pixel-parallel depth operation implementation method according to any one of claims 1 to 6 when executing the computer program.

Images