Asymmetric filling and convolution operator fusion optimization method and system based on mask mechanism

Abstract

The invention discloses an asymmetric filling and convolution operator fusion optimization method and system based on a mask mechanism, and belongs to the technical field of performance optimization of a universal graphics processor. The objective of the invention is to solve the problem of deep combination of an operator fusion technology in a high-level computational graph and a mask mechanism in a bottom kernel function. The method comprises the following steps: defining a Pytorch original model containing filling and convolution calculation modes to obtain a calculation graph; triggering a Pytorch-based graph optimization operator fusion method, namely traversing the computational graph to perform pattern matching, performing graph rewriting on the computational graph with a successfully matched pattern, and replacing original operator nodes in the computational graph with fusion operator nodes to obtain an optimized computational graph; and calling a fusion operator based on the asymmetric filling convolution of the mask mechanism, updating fusion nodes in the optimized calculation graph, obtaining the calculation graph fused and optimized by the asymmetric filling convolution operator based on the mask mechanism, and optimizing the reasoning performance under partial scenes of the GPGPU.

Description

Technical Field

[0001] This invention belongs to the field of general-purpose graphics processing unit (GPGPU) performance optimization technology, specifically involving an operator fusion optimization method and system based on masking mechanism of asymmetric padding and convolution. Background Technology

[0002] In complex image processing tasks, conventional symmetrical padding often fails to meet the requirements of refined network design, thus giving rise to asymmetric padding. The necessity of asymmetric padding is mainly reflected in the following two aspects:

[0003] Ensuring precise alignment of feature maps: In many advanced network architectures, such as the encoder-decoder architecture for medical image segmentation like U-Net, the network needs to concatenate feature maps from different depths and resolutions. When the stride of a convolution is greater than 1, asymmetric padding is often required to ensure that the downsampled feature map is perfectly aligned spatially with the feature map of the "skip connection" during subsequent upsampling. If the total padding amount is odd in any dimension, symmetric padding will not achieve this pixel-level precise alignment, thus affecting the segmentation accuracy of the model.

[0004] Constructing Task-Oriented Asymmetric Receptive Fields: In some applications, networks need to pay more attention to specific directions in an image. For example, in road scene segmentation tasks for autonomous driving, the model needs a wider "field of view" to understand the road conditions ahead, while paying less attention to the bottom areas of the image, such as the vehicle's hood. By adding more padding to one side of the feature map (such as the top), asymmetric padding can construct a receptive field biased towards a specific direction, guiding the model to learn and utilize directional spatial contextual information more effectively.

[0005] However, while asymmetric padding is crucial in algorithm design, its implementation at the GPGPU level often becomes a performance bottleneck. In current mainstream deep learning frameworks, asymmetric padding and convolution are typically treated as two independent computational steps. From a performance perspective, a separate padding operation is a typical memory-bound operation, requiring the allocation of new memory space and the copying of massive amounts of original data, introducing significant memory allocation and data transfer overhead. This "pad first, then convolution" sequential approach requires launching two independent computational kernels on the GPGPU, generating a temporary, padded intermediate tensor between the two. This intermediate tensor not only consumes valuable GPU memory resources but also causes additional data read / write latency, severely limiting the model, especially in applications processing high-resolution images or requiring real-time video streaming, where this bottleneck effect becomes increasingly prominent. Summary of the Invention

[0006] The problem this invention aims to solve is to deeply integrate the operator fusion technology in the high-level computation graph with the masking mechanism in the low-level kernel function to achieve end-to-end collaborative optimization. It proposes an operator fusion optimization method and system based on masking mechanism, which combines asymmetric padding and convolution.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for optimizing the fusion of asymmetric padding and convolution operators based on a masking mechanism includes the following steps:

[0009] S1. Define the original PyTorch model containing padding and convolution computation patterns to obtain the computation graph;

[0010] S2. Call torch.compile and specify a custom backend to trigger the graph optimization operator fusion method based on PyTorch, which includes traversing the computation graph for pattern matching, rewriting the computation graph for successfully matched patterns, and replacing the original operator nodes in the computation graph with fusion operator nodes to obtain the optimized computation graph.

[0011] S3. The asymmetric padding convolution based on the masking mechanism calls the fusion operator to update the fusion nodes in the optimized computation graph obtained in step S2, resulting in a computation graph optimized by fusion of the asymmetric padding and convolution operators based on the masking mechanism.

[0012] Furthermore, the specific implementation method of step S2 includes the following steps:

[0013] S2.1. Traverse the computation graph to perform pattern matching;

[0014] First, the input computation graph is traversed, and a specific computation sequence composed of torch.nn.functional.pad and torch.nn.functional.conv2d operators is matched using a pattern matching mechanism.

[0015] During pattern matching, check and ensure that the padding parameter of the convolution operator itself is 0 and the group number parameter is 1;

[0016] S2.2. Rewrite the computation graph for the successful pattern matching in step S2.1;

[0017] Extract the original input tensor and asymmetric padding values ​​from the padding node;

[0018] Extract the weight tensor, optional bias tensor, stride, and dilation from the convolution node conv2d;

[0019] S2.3. Replace the original operator nodes in the computation graph with fused operator nodes.

[0020] Create a new fused operator node fused_node in the computation graph, replace the original nodes pad and conv2d in the computation graph, and use the extracted original input tensor and asymmetric padding values ​​as the input of the new fused operator node. Redirect all subsequent users of the original nodes pad and conv2d in the computation graph to use the output of the new fused operator node.

[0021] Remove existing fill nodes and convolution nodes from the computation graph and perform dead code elimination to clean up redundant parts of the computation graph that are no longer used.

[0022] S2.4. Return the optimized computation graph. The pointer to the final output tensor of the optimized computation graph is Y_ptr, which is logically equivalent to the original computation graph.

[0023] Furthermore, the original input tensor mentioned in step S2.2 includes batch size N, number of input channels C, height H, and width W; the asymmetric padding value includes top padding. t padding below b Left padding l and right fill pad r ;

[0024] The pointer to the weight tensor is W_ptr, and its dimensions are the number of output channels K, the number of input channels C, the kernel height R, and the kernel width S.

[0025] The pointer to the optional bias tensor bias is B_ptr, and the optional bias tensor is a one-dimensional tensor of length K".

[0026] The convolution stride is divided into the stride of the convolution operation in height. h and the stride of the convolution operation on the width w ;

[0027] The void ratio dilation is divided into void ratio dilation of the convolution operation at the height. h dilation of the dilation rate across the width of the convolution operation w .

[0028] Furthermore, the specific implementation method of obtaining the fusion operator based on the asymmetric padding convolution of the masking mechanism in step S3 includes the following steps:

[0029] S3.1. Reverse coordinate calculation determines the coordinates of the data that the calculation depends on;

[0030] First, the spatial dimensions of the output tensor Y are calculated based on the original input tensor and the asymmetric padding values, including the height H of the output tensor. out Width of the output tensor The calculation formula is as follows:

[0031]

[0032] ;

[0033] Then, the height coordinates of the input tensor corresponding to the output tensor are calculated using the reverse calculation formula. The calculation expression is:

[0034]

[0035]

[0036] in, , These represent the ordinate and abscissa of any point on the output tensor, respectively. and These are the relative coordinates on R and S when traversing the convolution kernel;

[0037] S3.2. Using a masking mechanism for boundary processing and simulated filling:

[0038] For the coordinates of the input tensors calculated in step S3.1, the kernel dynamically generates a boolean input mask, input_mask, to determine whether the coordinates of each input tensor fall within the valid boundaries H and W of the original input tensor. Using the conditional load instruction provided by Triton, when the mask value is true, data is loaded normally from the video memory; when the mask value is false, that is, the coordinates of the input tensor correspond to the filling area, the preset filling value 0 is directly returned for boundary processing and simulated filling.

[0039] S3.3. Rearrange the convolution operation to optimize convolution speed by multiplying the matrix:

[0040] S3.3.1. The core computation process of convolution is rearranged. By rearranging the input data and weight tensor, and performing dimensional rearrangement and shape transformation on the weight tensor, the original format is converted into a two-dimensional matrix. The number of rows of this matrix is ​​determined by the product of C, R, and S, and the number of columns is K.

[0041] S3.3.2. Based on the two-dimensional matrix obtained in step S3.3.1, the convolution operation is transformed into the general matrix multiplication (GEMM) operation, and the intermediate results are stored in the accumulator.

[0042] S3.3.3. Combining Triton's autotuning capability, based on the specific input size and hardware characteristics, it automatically explores and selects the optimal execution configuration parameters to obtain an asymmetric padding convolution fusion operator based on a masking mechanism.

[0043] A system for optimizing the fusion of operators of asymmetric padding and convolution based on a masking mechanism includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for optimizing the fusion of operators of asymmetric padding and convolution based on a masking mechanism.

[0044] The beneficial effects of this invention are:

[0045] The present invention provides an operator fusion optimization method based on a mask mechanism for asymmetric padding and convolution, which can significantly reduce memory bandwidth overhead. By performing padding logic within a single kernel, the present invention completely eliminates the need to create and read / write intermediate padding tensors, thereby greatly reducing the memory access traffic of global video memory on GPGPU and solving the memory bandwidth bottleneck problem.

[0046] The present invention provides an optimization method for fusion of asymmetric padding and convolution operators based on a masking mechanism, which can reduce kernel startup and CPU overhead. The present invention merges two independent operators (padding and convolution) into one, so that when processing an "asymmetric padding-convolution" sequence, the number of GPGPU kernel startups required is reduced from two to one, thereby effectively reducing the scheduling overhead and synchronization latency on the CPU side.

[0047] The present invention provides an asymmetric padding and convolution operator fusion optimization method based on a masking mechanism, which can automatically adapt to performance. With the help of Triton's automatic tuning capabilities, the fusion operator of the present invention can automatically find the optimal execution strategy for different hardware and input parameters, and maximize performance in diverse GPGPU deployment environments without manual intervention.

[0048] The present invention provides an operator fusion optimization method based on asymmetric padding and convolution using a masking mechanism, which can improve end-to-end performance. Through the above optimization, the present invention can significantly improve the inference throughput of the model and reduce latency in traditional performance bottleneck scenarios such as large stride and large input tensor size.

[0049] The present invention provides a non-intrusive operator fusion optimization method based on asymmetric padding and convolution using a masking mechanism. As a pluggable module in the compiler backend, the present invention can automatically optimize existing models without modifying the user-level model code, and has good practicality and scalability. Attached Figure Description

[0050] Figure 1 This is an architecture diagram of an operator fusion optimization method based on masking mechanism for asymmetric padding and convolution as described in this invention.

[0051] Figure 2 This is a comparison chart of the performance test results after the optimization of this invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0053] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0054] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 and attached Figure 2 Detailed explanation is as follows:

[0055] Example 1:

[0056] A method for optimizing the fusion of asymmetric padding and convolution operators based on a masking mechanism includes the following steps:

[0057] S1. Define the original PyTorch model containing padding and convolution computation patterns to obtain the computation graph;

[0058] Furthermore, in this embodiment, the top layer is a graph optimization pass (PASS) based on the PyTorch framework, used to automatically identify and replace optimizable computational patterns during the neural network compilation stage; the bottom layer is an implementation of an asymmetric padding convolution fusion operator execution unit (Kernel) based on a masking mechanism for replacement.

[0059] S2. Call torch.compile and specify a custom backend to trigger the graph optimization operator fusion method based on PyTorch, which includes traversing the computation graph for pattern matching, rewriting the computation graph for successfully matched patterns, and replacing the original operator nodes in the computation graph with fusion operator nodes to obtain the optimized computation graph.

[0060] Furthermore, this stage is executed in the PyTorch compiler frontend and is designed to automatically replace inefficient combinations of operators in the original computation graph with our custom, efficient fusion operators.

[0061] Furthermore, the specific implementation method of step S2 includes the following steps:

[0062] S2.1. Traverse the computation graph to perform pattern matching;

[0063] First, the input computation graph is traversed, and a specific computation sequence composed of torch.nn.functional.pad and torch.nn.functional.conv2d operators is matched using a pattern matching mechanism.

[0064] During pattern matching, check and ensure that the padding parameter of the convolution operator itself is 0 and the group number parameter is 1;

[0065] S2.2. Rewrite the computation graph for the successful pattern matching in step S2.1;

[0066] Extract the original input tensor and asymmetric padding values ​​from the padding node;

[0067] Extract the weight tensor, optional bias tensor, stride, and dilation from the convolution node conv2d;

[0068] Furthermore, the original input tensor mentioned in step S2.2 includes batch size N, number of input channels C, height H, and width W; the asymmetric padding value includes top padding. t padding below b Left padding l and right fill pad r ;

[0069] The pointer to the weight tensor is W_ptr, and its dimensions are the number of output channels K, the number of input channels C, the kernel height R, and the kernel width S.

[0070] The pointer to the optional bias tensor bias is B_ptr, and the optional bias tensor is a one-dimensional tensor of length K".

[0071] The convolution stride is divided into the stride of the convolution operation in height. h and the stride of the convolution operation on the width w ;

[0072] The void ratio dilation is divided into void ratio dilation of the convolution operation at the height. h dilation of the dilation rate across the width of the convolution operation w .

[0073] S2.3. Replace the original operator nodes in the computation graph with fused operator nodes.

[0074] Create a new fused operator node fused_node in the computation graph, replace the original nodes pad and conv2d in the computation graph, and use the extracted original input tensor and asymmetric padding values ​​as the input of the new fused operator node. Redirect all subsequent users of the original nodes pad and conv2d in the computation graph to use the output of the new fused operator node.

[0075] Remove existing fill nodes and convolution nodes from the computation graph and perform dead code elimination to clean up redundant parts of the computation graph that are no longer used.

[0076] S2.4. Return the optimized computation graph. The pointer to the final output tensor of the optimized computation graph is Y_ptr, which is logically equivalent to the original computation graph.

[0077] S3. The asymmetric padding convolution based on the masking mechanism calls the fusion operator to update the fusion nodes in the optimized computation graph obtained in step S2, resulting in a computation graph optimized by fusion of the asymmetric padding and convolution operators based on the masking mechanism.

[0078] Furthermore, this phase describes the internal workings of the high-performance Triton GPGPU core invoked by the fusion node in the previous phase. This core seamlessly performs asymmetric padding and convolution operations within a single computational unit.

[0079] Furthermore, the specific implementation method of obtaining the fusion operator based on the asymmetric padding convolution of the masking mechanism in step S3 includes the following steps:

[0080] S3.1. Reverse coordinate calculation determines the coordinates of the data that the calculation depends on;

[0081] First, the spatial dimensions of the output tensor Y are calculated based on the original input tensor and the asymmetric padding values, including the height H of the output tensor. out Width of the output tensor The calculation formula is as follows:

[0082]

[0083] ;

[0084] Then, the height coordinates of the input tensor corresponding to the output tensor are calculated using the reverse calculation formula. The calculation expression is:

[0085]

[0086]

[0087] in, , These represent the ordinate and abscissa of any point on the output tensor, respectively. and These are the relative coordinates on R and S when traversing the convolution kernel;

[0088] S3.2. Using a masking mechanism for boundary processing and simulated filling:

[0089] For the coordinates of the input tensors calculated in step S3.1, the kernel dynamically generates a boolean input mask, input_mask, to determine whether the coordinates of each input tensor fall within the valid boundaries H and W of the original input tensor. Using the conditional load instruction provided by Triton, when the mask value is true, data is loaded normally from the video memory; when the mask value is false, that is, the coordinates of the input tensor correspond to the filling area, the preset filling value 0 is directly returned for boundary processing and simulated filling.

[0090] Furthermore, the core mechanism of this mask is shown in the following formula:

[0091]

[0092] in, This represents the value obtained through a mask loading operation at the calculated input coordinates (input, input). This value may come from the actual data of the original tensor or it may be a padding value.

[0093] Represents the original input tensor;

[0094] This indicates accessing the data element at the position of row 'inputth' and column 'inputw' in the original input tensor; this operation is performed only if the condition is true.

[0095] The if statement is a conditional statement that defines the boundary conditions under which an operation should be performed.

[0096] These are the height (row) coordinates of the input tensor upon which the current calculation depends, derived through reverse calculation.

[0097] The width (column) coordinates of the input tensor upon which the current calculation depends, derived through reverse calculation.

[0098] ∧ is the logical AND operator. It requires that both conditions on its left and right sides be true for the entire condition to be true.

[0099] (0 < inputh < H) Determines whether the calculated row coordinates are within the valid row boundary [0, H-1] of the tensor.

[0100] (0 < inputw < W) determines whether the calculated column coordinates are within the valid column boundaries of the tensor [0, W-1].

[0101] 0: The specified value returned by the system when the if condition is not met (i.e., the otherwise case). In this scenario, it is the padding value used for filling.

[0102] Furthermore, for any calculated access coordinate (inputw, inputw), the conditional load instruction provided by Triton is used. When the mask value is true, data is loaded normally from video memory; when the mask value is false (i.e., the coordinates correspond to a filled area), the preset fill value of 0 is returned directly. This method efficiently simulates the filling operation while avoiding the physical memory allocation and data read / write overhead of any general-purpose graphics processor.

[0103] S3.3. Rearrange the convolution operation to optimize convolution speed by multiplying the matrix:

[0104] S3.3.1. The core computation process of convolution is rearranged. By rearranging the input data and weight tensor, and performing dimensional rearrangement and shape transformation on the weight tensor, the original format is converted into a two-dimensional matrix. The number of rows of this matrix is ​​determined by the product of C, R, and S, and the number of columns is K.

[0105] S3.3.2. Based on the two-dimensional matrix obtained in step S3.3.1, the convolution operation is transformed into the general matrix multiplication (GEMM) operation, and the intermediate results are stored in the accumulator.

[0106] S3.3.3. Combining Triton's autotuning capability, based on the specific input size and hardware characteristics, it automatically explores and selects the optimal execution configuration parameters to obtain an asymmetric padding convolution fusion operator based on a masking mechanism.

[0107] For example, the thread block size (BLOCK_SIZE_M, BLOCK_SIZE_N, BLOCK_SIZE_K) determines the size of the data block computed in a single operation, the number of software pipeline stages (num_stages) used to hide memory latency, and the number of thread bundles (num_warps) that affect parallelism and resource consumption, thereby maximizing the computation speed of convolution.

[0108] Specifically, we conducted an optimization performance analysis of a novel convolution module with asymmetric padding and convolution calculation mode—the windmill convolution module. Under different input image sizes and strides, we compared the inference speeds of PyTorch's original Eager mode, the default torch.compile backend, and the optimized torch.compile custom backend used in this embodiment. We compared the average inference time after 100 warm-ups and 1000 inference runs. Our experimental tests were conducted on an x86 server equipped with an Nvidia 3090GPGPU. The server configuration is shown in Table 1.

[0109] Table 1

[0110]

[0111] To comprehensively evaluate the performance and correctness impact of the custom Triton fusion kernel and graph optimization pass, three execution modes are compared: native PyTorch (Eager Mode), the default torch.compile backend, and our custom backend integrated with the Triton fusion kernel. Tests will cover different convolution strides and input tensor sizes to ensure the generality and robustness of the evaluation results. The tests will consistently use a pinwheel convolution module with 32 input channels, 64 output channels, and a kernel size of 4. Convolution stride: traversing [1, 2, 3] to simulate dense, medium, and sparse sampling scenarios. Input size (H / W): traversing [320, 640] to cover two common resolutions (low and medium). Runtime: Tests will be executed on CUDA-compatible GPGPU devices to fully utilize the performance advantages of the Triton kernel. To eliminate initialization overhead and jitter, each model's performance test includes 100 warm-up runs. The model will then execute 1000 times consecutively, recording the total execution time. `torch.cuda.synchronize()` ensures precise timing of GPGPU operations. Performance analysis will calculate and display the end-to-end speedup, which is the percentage performance improvement achieved by the custom backend compared to native PyTorch. Test results are as follows: Figure 2 As shown, after optimization by the asymmetric padding of the masking mechanism and the operator fusion optimization technique of convolution proposed in this embodiment, the optimized custom backend has a good acceleration effect under various convolution strides and input size scenarios, which can significantly improve the inference speed of the model.

[0112] Example 2:

[0113] A system for optimizing the fusion of operators of asymmetric padding and convolution based on a masking mechanism includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for optimizing the fusion of operators of asymmetric padding and convolution based on a masking mechanism as described in Example 1.

[0114] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for optimizing the fusion of operators of asymmetric padding and convolution based on a masking mechanism, characterized in that, Includes the following steps: S1. Define the original PyTorch model containing padding and convolution computation patterns to obtain the computation graph; S2. Call torch.compile and specify a custom backend to trigger the graph optimization operator fusion method based on PyTorch, which includes traversing the computation graph for pattern matching, rewriting the computation graph for successfully matched patterns, and replacing the original operator nodes in the computation graph with fusion operator nodes to obtain the optimized computation graph. S3. The asymmetric padding convolution based on the masking mechanism calls the fusion operator to update the fusion nodes in the optimized computation graph obtained in step S2, resulting in a computation graph optimized by fusion of the asymmetric padding and convolution operators based on the masking mechanism.

2. The operator fusion optimization method based on masking mechanism for asymmetric padding and convolution according to claim 1, characterized in that, The specific implementation method of step S2 includes the following steps: S2.

1. Traverse the computation graph to perform pattern matching; First, the input computation graph is traversed, and a specific computation sequence composed of torch.nn.functional.pad and torch.nn.functional.conv2d operators is matched using a pattern matching mechanism. During pattern matching, check and ensure that the padding parameter of the convolution operator itself is 0 and the group number parameter is 1; S2.

2. Rewrite the computation graph for the successful pattern matching in step S2.1; Extract the original input tensor and asymmetric padding values ​​from the padding node; Extract the weight tensor, optional bias tensor, stride, and dilation from the convolution node conv2d; S2.

3. Replace the original operator nodes in the computation graph with fused operator nodes. Create a new fused operator node fused_node in the computation graph, replace the original nodes pad and conv2d in the computation graph, and use the extracted original input tensor and asymmetric padding values ​​as the input of the new fused operator node. Redirect all subsequent users of the original nodes pad and conv2d in the computation graph to use the output of the new fused operator node. Remove existing fill nodes and convolution nodes from the computation graph and perform dead code elimination to clean up redundant parts of the computation graph that are no longer used. S2.

4. Return the optimized computation graph. The pointer to the final output tensor of the optimized computation graph is Y_ptr, which is logically equivalent to the original computation graph.

3. The operator fusion optimization method based on masking mechanism for asymmetric padding and convolution according to claim 1 or 2, characterized in that, The original input tensor in step S2.2 includes batch size N, number of input channels C, height H, and width W; the asymmetric padding value includes top padding. t padding below b Left padding l and right fill pad r ; The pointer to the weight tensor is W_ptr, and its dimensions are the number of output channels K, the number of input channels C, the kernel height R, and the kernel width S. The pointer to the optional bias tensor bias is B_ptr, and the optional bias tensor is a one-dimensional tensor of length K". The convolution stride is divided into the stride of the convolution operation in height. h and the stride of the convolution operation on the width w ; The void ratio dilation is divided into void ratio dilation of the convolution operation at the height. h dilation of the dilation rate across the width of the convolution operation w .

4. The operator fusion optimization method based on masking mechanism for asymmetric padding and convolution according to claim 3, characterized in that, Step S3, which uses asymmetric padding convolution based on a masking mechanism to obtain the fusion operator, includes the following steps: S3.

1. Reverse coordinate calculation determines the coordinates of the data that the calculation depends on; First, the spatial dimensions of the output tensor Y are calculated based on the original input tensor and the asymmetric padding values, including the height H of the output tensor. out Width of the output tensor The calculation formula is as follows: ； Then, the height coordinates of the input tensor corresponding to the output tensor are calculated using the reverse calculation formula. The calculation expression is: in, , These represent the ordinate and abscissa of any point on the output tensor, respectively. and These are the relative coordinates on R and S when traversing the convolution kernel; S3.

2. Using a masking mechanism for boundary processing and simulated filling: For the coordinates of the input tensors calculated in step S3.1, the kernel dynamically generates a boolean input mask, input_mask, to determine whether the coordinates of each input tensor fall within the valid boundaries H and W of the original input tensor. Using the conditional load instruction provided by Triton, when the mask value is true, data is loaded normally from the video memory; when the mask value is false, that is, the coordinates of the input tensor correspond to the filling area, the preset filling value 0 is directly returned for boundary processing and simulated filling. S3.

3. Rearrange the convolution operation to optimize convolution speed by multiplying the matrix: S3.3.

1. The core computation process of convolution is rearranged. By rearranging the input data and weight tensor, and performing dimensional rearrangement and shape transformation on the weight tensor, the original format is converted into a two-dimensional matrix. The number of rows of this matrix is ​​determined by the product of C, R, and S, and the number of columns is K. S3.3.

2. Based on the two-dimensional matrix obtained in step S3.3.1, the convolution operation is transformed into the general matrix multiplication (GEMM) operation, and the intermediate results are stored in the accumulator. S3.3.

3. Combining Triton's autotuning capability, based on the specific input size and hardware characteristics, it automatically explores and selects the optimal execution configuration parameters to obtain an asymmetric padding convolution fusion operator based on a masking mechanism.

5. A system for optimizing the fusion of operators of asymmetric padding and convolution based on a masking mechanism, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the operator fusion optimization method based on a mask mechanism of any one of claims 1-4.

Images