An isometric linear layer computation acceleration method and system based on group fourier transform
By using group Fourier transform and Fourier convolution theorem, the problem of high computational complexity of equivariant linear layers is solved, achieving improved computational efficiency and preservation of accuracy, thus resolving the issues of high computational complexity and accuracy loss in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XI AN JIAOTONG UNIV
- Filing Date
- 2026-06-04
- Publication Date
- 2026-07-21
AI Technical Summary
Existing isovariant linear layers have high computational complexity, and general acceleration methods cannot utilize their structural redundancy, resulting in low computational efficiency and accuracy loss.
By obtaining the input features and weight matrix of the equivariant linear layer, projecting them into the frequency domain using the group Fourier transform, performing matrix multiplication of group components based on the group Fourier convolution theorem, and using the conjugate symmetry of the real-valued Fourier transform to complete the frequency domain output features, and finally restoring them to the group space through the inverse group Fourier transform, the computation is accelerated.
It significantly reduces computational complexity from quadratic to linear, saving approximately 50% of computational and storage costs, while maintaining numerical consistency and achieving efficient computational acceleration.
Smart Images
Figure CN122432455A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural network computation and processing technology, and in particular to a method and system for accelerating computation of equivariant linear layers based on group Fourier transform. Background Technology
[0002] Currently, while equivariant networks, based on group theory, can effectively utilize the geometric symmetry of data, their core module—the equivariant linear layer—generally involves cyclic matrix operations in the group dimension, resulting in high computational complexity. This fails to leverage the structural redundancy inherent in equivariance, leading to low computational efficiency and severely hindering its engineering applications. Existing mainstream deep learning acceleration methods (such as pruning, quantization, and knowledge distillation) are all general model compression strategies. These not only fail to utilize the inherent symmetric redundancy of equivariant networks but also often sacrifice model accuracy, resulting in very limited acceleration effects. Summary of the Invention
[0003] This invention provides a computation acceleration method and system for equivariant linear layers based on group Fourier transform, which addresses the problems of high computational complexity of existing equivariant linear layers, the inability of general acceleration methods to utilize their structural redundancy and the resulting loss of accuracy, and the lack of dedicated efficient algorithms for equivariant linear layer structures, leading to low computational efficiency.
[0004] The objective of this invention can be achieved through the following technical solutions:
[0005] The first aspect of this invention is to provide a method for accelerating the computation of equivariant linear layers based on group Fourier transform, comprising: Obtain the input features and weight matrix of the equivariant linear layer; wherein the input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension; The input features and the weight matrix are projected from the group space to the frequency domain using the group Fourier transform to obtain the frequency domain input features and the frequency domain weight features. Based on the group Fourier convolution theorem, the group-by-group component matrix multiplication is performed on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features. By utilizing the conjugate symmetry of the real-valued Fourier transform, the second half of the frequency domain component of the frequency domain output feature is obtained through the first half of the frequency domain component; the first half of the frequency domain component and the second half of the frequency domain component constitute the complete frequency domain output feature. The complete frequency domain output features are restored to the group space using the inverse group Fourier transform, resulting in the final group space output features; the final group space output features are the result of accelerated computation.
[0006] Furthermore, the weight matrix is a learnable weight matrix; the learnable weight matrix is generated during model initialization and updated during training through backpropagation and gradient descent algorithms.
[0007] Furthermore, the general calculation form of the isovariant linear layer is as follows:
[0008] In the formula, It is a finite cyclic rotation group; This represents the learnable weight matrix. and for The elements in and These are the input and output features with group dimension, respectively; Indicates the input features in the group elements The component at the location; Indicates the output feature in the group element The component at that location.
[0009] Further, the step of projecting the input features and the weight matrix from the group space to the frequency domain using the group Fourier transform to obtain frequency domain input features and frequency domain weight features includes: Perform a Discrete Fourier Transform along the group dimension on the input feature x to obtain the frequency domain input feature. Simultaneously, a discrete Fourier transform is performed on the weight matrix W along the group dimension to obtain the frequency domain weight features. ; in, It represents the discrete Fourier transform in the group dimension.
[0010] Furthermore, based on the group Fourier convolution theorem, the first half of the frequency domain component of the frequency domain output feature is obtained by performing a group-component matrix multiplication on the frequency domain input feature and the frequency domain weight feature in the frequency domain, based on the group Fourier convolution theorem. This includes: According to the group Fourier convolution theorem , to the front Given a frequency component k, perform frequency domain input feature analysis. Frequency domain weighting features Product of feature dimension matrices:
[0011] In the formula, This represents the frequency domain output characteristics corresponding to frequency component k. This represents the frequency domain input feature corresponding to frequency component k. This represents the frequency domain weighting feature corresponding to frequency component k. Indicates to Perform the floor function; It is a finite cyclic rotation group; Before The frequency domain output features corresponding to each frequency component k constitute a complete frequency domain output feature. The first half of the frequency domain components.
[0012] Furthermore, the step of utilizing the conjugate symmetry of the real-valued Fourier transform to obtain the second half of the frequency domain components of the frequency domain output features through the first half of the frequency domain components includes: The formula for the conjugate symmetry of the real-valued Fourier transform is specifically expressed as follows:
[0013] In the formula, This indicates that the index in the frequency domain is taken first. The components, and then the index is The complex result of the component is taken as the complex conjugate.
[0014] Furthermore, the step of using the inverse group Fourier transform to recover the complete frequency domain output features to the group space, thereby obtaining the final group space output features, includes: Complete frequency domain output characteristics Performing the inverse discrete Fourier transform along the group dimension yields the final group space output feature y; the inverse discrete Fourier transform is specifically expressed by the formula:
[0015] In the formula, This represents the complete frequency domain output characteristics. This represents the final group space output feature. Represents the discrete inverse Fourier transform matrix. It is a finite cyclic rotation group.
[0016] A second aspect of the present invention is to provide a computation acceleration system for equivariant linear layers based on group Fourier transform, comprising: Feature acquisition module: used to acquire the input features and weight matrix of the equivariant linear layer; wherein, the input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension; Forward Transform Module: Used to project the input features and the weight matrix from the group space to the frequency domain using the group Fourier transform, respectively, to obtain frequency domain input features and frequency domain weight features; Frequency domain multiplication module: Based on the group Fourier convolution theorem, it performs group-by-group component matrix multiplication on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features. Symmetric completion module: Used to obtain the second half of the frequency domain component of the frequency domain output feature by utilizing the conjugate symmetry of the real-valued Fourier transform through the first half of the frequency domain component; the first half of the frequency domain component and the second half of the frequency domain component form a complete frequency domain output feature. Inverse Transform Module: Used to recover the complete frequency domain output features to the group space using the inverse group Fourier transform, to obtain the final group space output features; the final group space output features are the result of accelerated calculation.
[0017] A third aspect of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the aforementioned method for accelerating computation of equivariant linear layers based on group Fourier transform.
[0018] A fourth aspect of the present invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for accelerating computation of equivariant linear layers based on group Fourier transform.
[0019] Compared with existing technologies, the beneficial effects of this invention are: obtaining the input features and weight matrix of an equivariant linear layer; wherein the input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension; clarifying the group-dimensional structure and cyclically shifted sharing characteristics of the input features and weight matrix provides a mathematical premise for subsequent acceleration using group Fourier transform; projecting the input features and weight matrix from the group space to the frequency domain using group Fourier transform to obtain frequency domain input features and frequency domain weight features; converting the cyclic convolution structure in the group space into an independent dot product structure in the frequency domain through group Fourier transform, laying the foundation for reducing the computational complexity from quadratic to linear; based on the group Fourier convolution theorem, performing group-component-wise matrix multiplication on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features; using the group Fourier convolution theorem, only the first half of the frequency components need to be multiplied, significantly reducing computational complexity. The number of multiplication operations is reduced; utilizing the conjugate symmetry of the real-valued Fourier transform, the second half of the frequency domain component of the frequency domain output feature is obtained through the first half of the frequency domain component; the first half and the second half of the frequency domain components constitute the complete frequency domain output feature; utilizing the conjugate symmetry of the real-valued Fourier transform, the second half of the frequency domain component can be obtained without additional calculation, saving about 50% of the computation and storage overhead; the complete frequency domain output feature is restored to the group space using the inverse group Fourier transform, resulting in the final group space output feature; the final group space output feature is the accelerated computation result; the frequency domain result is accurately restored to the group space output through the inverse group Fourier transform, ensuring that the accelerated computation is mathematically equivalent to the original equivariant linear layer and maintaining numerical consistency in finite precision implementation; this solves the problems of high computational complexity of existing equivariant linear layers, the inability of general acceleration methods to utilize their structural redundancy and the existence of precision loss, and the lack of dedicated efficient algorithms for equivariant linear layer structures, leading to low computational efficiency. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This invention provides a flowchart illustrating the steps of an accelerated computation method for equivariant linear layers based on group Fourier transform. Figure 2 This invention provides a schematic diagram of the module flow of an equivariant linear layer computation acceleration system based on group Fourier transform. Detailed Implementation
[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0024] To address the problems existing in the background technology, a computation acceleration method and system for equivariant linear layers based on group Fourier transform is designed, which has important practical significance.
[0025] like Figure 1 As shown, the first aspect of this invention is to provide a method for accelerating the computation of equivariant linear layers based on group Fourier transform, comprising the following steps: Step S1: Obtain the input features and weight matrix of the equivariant linear layer; wherein the input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension.
[0026] It should be noted that because the input feature x of the equivariant linear layer has structured symmetry in the group dimension, and the weight matrix W has a cyclically shifted parameter-sharing structure in the group dimension, the operation between the two is essentially equivalent to a cyclic convolution in the group dimension. In order to explicitly extract this core mathematical feature and lay the foundation for the subsequent use of group Fourier transform to convert the cyclic convolution into a pointwise product in the frequency domain, it is necessary to first obtain and clarify the input features and weight parameters.
[0027] It should be further noted that the equivariant linear layer is the core basic module of the group equivariant network. Its input features x and output features y both have structured symmetry induced by the group equivariant constraints in the group dimension, and the weight matrix W has a cyclically shifted parameter-sharing structure in the group dimension.
[0028] Specifically, the general calculation form of the isovariant linear layer is as follows:
[0029] In the formula, It is a finite cyclic rotation group, or a finite transformation group that can be represented as a cyclic convolution structure along the group dimension; This represents the learnable weight matrix. and for The elements in the group (also used as indexes for the group dimension). and These are the input and output features with group dimension, respectively; Indicates the input features in the group elements The component at that point, that is, the input feature corresponding to the group dimension. The feature value at this location (i.e., it is directly provided by the output of the previous layer during the forward computation of the model, or obtained from the input data after preprocessing); Indicates the output feature in the group element The components at the specified location. The input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension.
[0030] Among them, both the input features and the output features are composed of groups. The feature matrix consists of the element categories and the number of channels. Each basic learnable weight matrix is a C×C matrix, where C is the number of feature channels. The learnable weight matrices are generated during model initialization and updated during training using backpropagation and gradient descent algorithms.
[0031] In equivariant linear layers, due to group equivariance constraints (requiring the network output to change in the same way as the input), the weight matrix must exhibit a cyclically shifted parameter-sharing structure in the group dimension. This means that from any input group element... To output group elements weight matrix Only depend on As a result, there are only |G| independently learnable weight matrices (each with a shape of C×C), and the total number of parameters is |G|×C². However, during computation, these |G| weight matrices need to be expanded into |G|×|G| matrices through cyclic shifting to participate in the calculation, resulting in an actual computational cost of |G|²C². This phenomenon of "few parameters but repeated use during computation" is called structural redundancy. In contrast, ordinary linear layers do not have equivariant constraints and need to perform calculations for each pair of ( , Learning independent C×C weight matrices, the total number of parameters and computational cost is |G|²C². Therefore, the equivariant linear layer has inherent structural redundancy, which is the fundamental premise for this scheme to reduce its computational complexity from O(|G|²C²) to O(|G|C²) using group Fourier transform. Here, |G| is a finite cyclic rotation group. The order of the finite cyclic rotation group (i.e., the order of the finite cyclic rotation group) (Number of elements in the text).
[0032] In this embodiment, the finite cyclic rotation group (i.e., rotations of 0°, 90°, 180°, and 270°, |G|=4) is discussed as a typical example. The weight matrix of its equivariant linear layer forms the following cyclic block matrix structure in the group dimension:
[0033] In this matrix, each element is a C×C learnable weight matrix, used to map the C-dimensional vector of the input feature at a certain group position to the C-dimensional vector of the output feature at a certain group position. Each row is the result of a rightward cyclic shift of the previous row, meaning that there is a sharing and rotation pattern among the block matrices in the weight matrix.
[0034] Step S2: Project the input features and the weight matrix from the group space to the frequency domain using the group Fourier transform to obtain the frequency domain input features and the frequency domain weight features.
[0035] It should be noted that, since the operation of the equivariant linear layer is essentially equivalent to the circular convolution in the group dimension, and the circular convolution has a high complexity of O(|G|²C²) when directly computed in the group space, in order to use the group Fourier convolution theorem to transform the circular convolution into a simple pointwise multiplication in the frequency domain, thereby reducing the computational complexity to O(|G|C²), it is necessary to perform group Fourier transform on the input features and weight matrix respectively, projecting them from the original group space to the frequency domain.
[0036] It should be further noted that, in the embodiments of this application, a Group Fourier Transform (GFT) is introduced to eliminate the computational redundancy caused by circular convolution, taking into account the nature of circular convolution in the group dimension of equivariant linear layers. In the circular rotation group scenario involved in this application, the Group Fourier Transform is equivalent to the Discrete Fourier Transform (DFT) in the group dimension.
[0037] Specifically, a discrete Fourier transform is performed on the input feature x along the group dimension to obtain the frequency domain input feature. Simultaneously, a discrete Fourier transform is performed on the weight matrix W along the group dimension to obtain the frequency domain weight features. .in, It represents the discrete Fourier transform (also the transform matrix) in the group dimension.
[0038] in, The (n, k)th term in the transformation matrix is... In the formula, n is the group space sample index, k is the frequency component index, and j is the imaginary unit; It is the rotation factor (complex number).
[0039] In this embodiment, for the equivariant group with 4-direction rotation (|G|=4), the group Fourier transform matrix (simplified to real form) is:
[0040] This utilizes the twitch factor in the 4th-order DFT (Discrete Fourier Transform). The simplification of the real and imaginary parts allows the transformation matrix to be directly represented by real numbers, thus avoiding the overhead of complex number operations in actual GPU (Graphics Processing Unit) implementations.
[0041] Through the above transformation, the input feature tensor of dimension N×C×|G| (where N is the sequence length, C is the number of feature channels, and |G| is the order of the group) in the original group space is projected into the frequency domain, and the frequency domain features are transformed into the frequency domain features. With frequency domain weights The dimensions are the same as the original features, but the circular convolution structure in the group dimension has been transformed into a dot product structure in the frequency domain where each component is independent.
[0042] Step S3: Based on the group Fourier convolution theorem, perform group-by-group component matrix multiplication on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features.
[0043] It should be noted that after the group Fourier transform in step S2, the input features and weight parameters have been projected from the group space to the frequency domain. According to the group Fourier convolution theorem, the circular convolution operation in the original group space is equivalent to the independent product of each frequency component in the frequency domain. Therefore, in order to fully utilize the independence and parallelism of the frequency domain, the original O(|G|²C²) group space matrix operation is transformed into an O(|G|C²) matrix product of each component in the frequency domain. Therefore, it is necessary to perform a multiplication of the frequency domain features... With frequency domain weights Performing group-by-group component matrix multiplication yields the frequency domain output features. The first half of the component.
[0044] It should be further explained that the group Fourier convolution theorem transforms the circular convolution in the group space (which requires traversing all g and g', a total of |G|² matrix multiplications) into pointwise multiplication in the frequency domain (which only requires one matrix multiplication for each frequency component k, a total of |G| matrix multiplications).
[0045] Specifically, according to the group Fourier convolution theorem... , to the front Given a frequency component k, perform frequency domain input feature analysis. Frequency domain weighting features Product of feature dimension matrices:
[0046] In the formula, This represents the frequency domain output characteristics corresponding to frequency component k. This represents the frequency domain input feature corresponding to frequency component k. This represents the frequency domain weighting feature corresponding to frequency component k. Indicates to Perform the floor function.
[0047] in, Representing input features With weight Convolution operation on group G.
[0048] Before The frequency domain output features corresponding to each frequency component k constitute a complete frequency domain output feature. The first half of the frequency domain components.
[0049] In this embodiment, for the equivariant group with 4-direction rotation (|G|=4), theoretically only the matrix product of the three frequency domain components k=0,1,2 needs to be calculated (replacing the original 4×4=16 matrix product operations).
[0050] Step S4: Utilizing the conjugate symmetry of the real-valued Fourier transform, the second half of the frequency domain component of the frequency domain output feature is obtained through the first half of the frequency domain component; the first half of the frequency domain component and the second half of the frequency domain component constitute the complete frequency domain output feature.
[0051] It should be noted that since the input features and weight parameters in the equivariant network are all real numbers, the real-valued discrete Fourier transform result naturally satisfies conjugate symmetry (i.e., the second half component is the complex conjugate of the first half component). Therefore, in order to avoid repeatedly performing matrix multiplication calculations on the second half frequency component (accounting for about 50% of the frequency domain multiplication), and thus further reduce the computational overhead and improve the acceleration effect, the second half component of the frequency domain output feature is directly supplemented by the calculated first half component using conjugate symmetry.
[0052] Specifically, based on the first half of the components By utilizing the conjugate symmetry formula of the real-valued Fourier transform, the second half of the frequency domain components is directly derived through conjugate operations, thus completing the full frequency domain output characteristics. No additional matrix multiplication is required. For the real-valued components k=0 and k=|G| / 2 (when |G| is even), they are symmetric on their own and do not require padding.
[0053] The conjugate symmetry formula for the real-valued Fourier transform is specifically expressed as follows:
[0054] In the formula, This indicates that the index in the frequency domain is taken first. The components, and then the index is The complex result of the component is taken as the complex conjugate (i.e., the real part remains unchanged and the imaginary part is inverted). This equation shows that in the Fourier transform result of a real signal, the first half and the second half of the component are complex conjugates of each other.
[0055] Utilizing the aforementioned conjugate symmetry, for equivariant groups with 4-direction rotations, the total number of frequency domain matrix multiplications is reduced from 4 to 3. Compared to dimensional matrix operations in naive groups, the matrix multiplication scale is reduced from the computation corresponding to 16 cyclic blocks to the computation corresponding to 3 frequency components, thereby significantly reducing computational load and storage access overhead.
[0056] At this point, the second half of the frequency domain components has been obtained.
[0057] The first half of the frequency domain components and the second half of the frequency domain components are combined to form a complete frequency domain output feature. .
[0058] Step S5: Use the inverse group Fourier transform to restore the complete frequency domain output features to the group space, and obtain the final group space output features; the final group space output features are the result of accelerated calculation.
[0059] It should be noted that steps S3 and S4 yield frequency domain output characteristics. The final output of the equivariant linear layer must reside in the original group space so that subsequent network modules (such as equivariant convolutional layers and activation functions) can process it correctly. Therefore, in order to restore the frequency domain calculation result to the group space output result that is completely equivalent to the original equivariant linear layer, the complete frequency domain output features are... Perform the inverse group Fourier transform to obtain the final group space output feature y.
[0060] Specifically, for the complete frequency domain output features Performing the inverse discrete Fourier transform along the group dimension yields the final group space output feature y (i.e., the accelerated computation result); the inverse discrete Fourier transform is specifically expressed by the formula:
[0061] In the formula, This represents the complete frequency domain output characteristics. This represents the final group space output feature. This represents the discrete inverse Fourier transform matrix.
[0062] in, The (n, k)th term in the discrete Fourier inverse transform matrix is... This transformation restores the frequency domain output features to the group space, resulting in the accelerated calculation result y.
[0063] In this embodiment, for the equivariant group with 4-direction rotation, the inverse transformation matrix (in real form) is:
[0064] To verify the accuracy and speed of the acceleration algorithm in this application, systematic tests were conducted on a standard GPU platform (NVIDIA A100) to compare the accelerated equal-variable linear layer (Flash EQ-Linear) and the original equal-variable linear layer (Naive EQ-Linear). As shown in Table 1, in the numerical accuracy test, the maximum absolute error of the output characteristics of Flash EQ-Linear and Naive EQ-Linear was only 1.31 × 10⁻⁶. -6 The mean absolute error is approximately 1.12 × 10⁻⁶. -7 This verifies the numerical equivalence between the accelerated calculation results and the original calculation results.
[0065] Table 1. Accuracy comparison of acceleration algorithms for isovariant linear layers (fp32, forward / backward)
[0066] As shown in Table 2, in terms of speed testing, under different input feature dimensions (|G|=4, with varying sequence lengths and number of channels), Flash EQ-Linear achieved a significant speedup of 1.3× (1.3 times) to 4.4× (4.4 times) compared to Naive EQ-Linear during the inference phase, and a speedup of 1.1× (1.1 times) to 5.7× (5.7 times) during the training phase, fully validating the effectiveness of this scheme. The speedup was more significant with small-scale features (4.4× inference speedup), while with large-scale features, the speedup converged somewhat due to memory bandwidth limitations, but remained above 1.1× (1.1 times), demonstrating the universality of this scheme across different scales.
[0067] Table 2. Performance Comparison of Equivariant Linear Layers (Different Input Dimensions)
[0068] As shown in Table 3, compared to the ordinary linear layer in the official PyTorch CUDA implementation, Flash EQ-Linear achieves a significant speedup of 1.3× to 1.7× in the inference stage and 1.1× to 1.2× in the training stage under different input feature dimensions (|G|=4, varying sequence lengths and number of channels), fully validating the effectiveness of this scheme. Furthermore, the speedup effect of Flash EQ-Linear compared to PyTorch Linear is more pronounced with a higher number of channels, demonstrating the universality of this scheme.
[0069] Table 3 Performance Comparison of Flash Isovariant Linear Layers and PyTorch Ordinary Linear Layers (Different Input Dimensions)
[0070] like Figure 2 As shown, a second aspect of the present invention is to provide an equivariant linear layer computation acceleration system based on group Fourier transform, comprising: Feature acquisition module 101: used to acquire the input features and weight matrix of the equivariant linear layer; wherein, the input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension; Forward transformation module 102: used to project the input features and the weight matrix from the group space to the frequency domain using the group Fourier transform, respectively, to obtain frequency domain input features and frequency domain weight features; Frequency domain multiplication module 103: Based on the group Fourier convolution theorem, it performs a group-by-group component matrix multiplication on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features. Symmetric completion module 104: Used to obtain the second half of the frequency domain component of the frequency domain output feature by utilizing the conjugate symmetry of the real-valued Fourier transform through the first half of the frequency domain component; the first half of the frequency domain component and the second half of the frequency domain component form a complete frequency domain output feature. Inverse transform module 105: used to restore the complete frequency domain output features to the group space using the inverse group Fourier transform, to obtain the final group space output features; the final group space output features are the result of accelerated calculation.
[0071] A third aspect of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements a method for accelerating computation of equivariant linear layers based on group Fourier transform.
[0072] A fourth aspect of the present invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for accelerating computation of equivariant linear layers based on group Fourier transform.
[0073] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.
[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.
Claims
1. A method for accelerating computation of equivariant linear layers based on group Fourier transform, characterized in that, include: Obtain the input features and weight matrix of the equivariant linear layer; wherein the input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension; The input features and the weight matrix are projected from the group space to the frequency domain using the group Fourier transform to obtain the frequency domain input features and the frequency domain weight features. Based on the group Fourier convolution theorem, the group-by-group component matrix multiplication is performed on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features. By utilizing the conjugate symmetry of the real-valued Fourier transform, the second half of the frequency domain component of the frequency domain output feature is obtained through the first half of the frequency domain component; the first half of the frequency domain component and the second half of the frequency domain component constitute the complete frequency domain output feature. The complete frequency domain output features are restored to the group space using the inverse group Fourier transform, resulting in the final group space output features; the final group space output features are the result of accelerated computation.
2. The method for accelerating computation of equivariant linear layers based on group Fourier transform according to claim 1, characterized in that, The weight matrix is a learnable weight matrix; the learnable weight matrix is generated during model initialization and updated during training through backpropagation and gradient descent algorithms.
3. The method for accelerating computation of equivariant linear layers based on group Fourier transform according to claim 1, characterized in that, The general calculation form of the isotropic linear layer is as follows: In the formula, It is a finite cyclic rotation group; This represents the learnable weight matrix. and for The elements in and These are the input and output features with group dimension, respectively; Indicates the input features in the group elements The component at the location; Indicates the output feature in the group element The component at that location.
4. The method for accelerating computation of equivariant linear layers based on group Fourier transform according to claim 1, characterized in that, The step of projecting the input features and the weight matrix from the group space to the frequency domain using the group Fourier transform to obtain frequency domain input features and frequency domain weight features includes: Perform a Discrete Fourier Transform along the group dimension on the input feature x to obtain the frequency domain input feature. Simultaneously, a discrete Fourier transform is performed on the weight matrix W along the group dimension to obtain the frequency domain weight features. ; in, It represents the discrete Fourier transform in the group dimension.
5. The method for accelerating computation of equivariant linear layers based on group Fourier transform according to claim 4, characterized in that, The method, based on the group Fourier convolution theorem, performs a group-component-wise matrix multiplication on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features, including: According to the group Fourier convolution theorem , to the front Given a frequency component k, perform frequency domain input feature analysis. Frequency domain weighting features Product of feature dimension matrices: In the formula, This represents the frequency domain output characteristics corresponding to frequency component k. This represents the frequency domain input feature corresponding to frequency component k. This represents the frequency domain weighting feature corresponding to frequency component k. Indicates to Perform the floor function; It is a finite cyclic rotation group; Before The frequency domain output features corresponding to each frequency component k constitute a complete frequency domain output feature. The first half of the frequency domain components.
6. The method for accelerating computation of equivariant linear layers based on group Fourier transform according to claim 5, characterized in that, The method of utilizing the conjugate symmetry of the real-valued Fourier transform to obtain the second half of the frequency domain components of the frequency domain output characteristics through the first half of the frequency domain components includes: The formula for the conjugate symmetry of the real-valued Fourier transform is specifically expressed as follows: In the formula, This indicates that the index in the frequency domain is taken first. The components, and then the index is The complex result of the component is taken as the complex conjugate.
7. The method for accelerating computation of equivariant linear layers based on group Fourier transform according to claim 1, characterized in that, The process of using the inverse group Fourier transform to restore the complete frequency domain output features to the group space, resulting in the final group space output features, includes: Complete frequency domain output characteristics Performing the inverse discrete Fourier transform along the group dimension yields the final group space output feature y; the inverse discrete Fourier transform is specifically expressed by the formula: In the formula, This represents the complete frequency domain output characteristics. This represents the final group space output feature. Represents the discrete inverse Fourier transform matrix. It is a finite cyclic rotation group.
8. A computation acceleration system for equivariant linear layers based on group Fourier transform, used to implement the computation acceleration method for equivariant linear layers based on group Fourier transform as described in any one of claims 1-7, characterized in that, include: Feature acquisition module: used to acquire the input features and weight matrix of the equivariant linear layer; wherein, the input features have a group-dimensional structure, and the weight matrix has a cyclically shifted parameter-sharing structure in the group dimension; Forward Transform Module: Used to project the input features and the weight matrix from the group space to the frequency domain using the group Fourier transform, respectively, to obtain frequency domain input features and frequency domain weight features; Frequency domain multiplication module: Based on the group Fourier convolution theorem, it performs group-by-group component matrix multiplication on the frequency domain input features and the frequency domain weight features in the frequency domain to obtain the first half of the frequency domain output features. Symmetric completion module: Used to obtain the second half of the frequency domain component of the frequency domain output feature by utilizing the conjugate symmetry of the real-valued Fourier transform through the first half of the frequency domain component; the first half of the frequency domain component and the second half of the frequency domain component form a complete frequency domain output feature. Inverse Transform Module: Used to recover the complete frequency domain output features to the group space using the inverse group Fourier transform, to obtain the final group space output features; the final group space output features are the result of accelerated calculation.
9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method for accelerating computation of equivariant linear layers based on any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for accelerating computation of equivariant linear layers based on group Fourier transform as described in any one of claims 1-7.