System and method for constructing neural network isovariance for any matrix group by using group representation and isovariant tensor fusion

Abstract

A robust machine learning system for robust machine learning is provided. The robust machine learning system includes: a processor; and a memory storing a robust machine learning program and a group representation network (GRepsNet), where the robust machine learning program runs on the processor. In this case, the processor is configured to: receive, via an input layer connected to the GrepsNet, an input data set represented by a first group representation; the input dataset is communicated / transmitted by a GRepsNet configured to generate an output dataset represented by an output group representation from the input dataset, where the output group is equivariant with the first group, and the output dataset is generated from an output layer connected to the GRepsNet.

Description

Technical Field

[0001] This invention generally relates to systems and methods for robust machine learning systems, and more specifically to swarm-equal neural networks for applications such as image classification and predicting dynamics. Background Technology

[0002] Equivariant neural networks are neural networks that are completely equivariant to certain input transformations (often group transformations). Equivariant neural networks often have the advantage of requiring fewer training data samples and are guaranteed to be robust to transformations to which they are designed to be equivariant. For example, convolutional neural networks (CNNs) are designed to be equivariant to translations; that is, if the input of a CNN is translated, all hidden feature maps are also translated by the same or similar amount. The set of translations, together with the vector additions used to constitute the two translations, forms an algebraic structure called a "group." This idea of ​​equivariance can be generalized to other groups, such as rotations, to form equivariant neural networks.

[0003] However, constructing efficient equivariant networks for general groups and domains is challenging. While existing methods directly address equivariant constraints for arbitrary matrix groups to obtain equivariant multilayer perceptrons (EMLPs), these methods do not scale well.

[0004] Therefore, there is a need to develop a system and method that can construct and provide neural networks that are equivariant to arbitrary matrix groups and easily extend to larger networks and various architectures such as multilayer perceptrons (MLPs), CNNs, and MLP-mixers, given a correct group representation for the input data. Summary of the Invention

[0005] Some embodiments of the present invention are based on the understanding that group equivariance is a strong inductive bias applicable to a wide range of domains, including images, point clouds, dynamical systems, and partial differential equations (PDEs). Scalability is crucial for deep learning. This necessitates the design of group equivariant networks for simple and scalable general domains and groups. To this end, this disclosure describes a robust machine learning system using a group representation network (GRepsNet) as a simple equivariant network for arbitrary matrix groups. In some embodiments of the invention, a key intuition behind the design is that the use of tensor representations in the hidden layers of neural networks, along with appropriate mixtures of various representations, can yield expressive equivariant networks, a fact we have empirically confirmed. The robust machine learning system using GRepsNet in this invention competes with EMLP on several tasks with group symmetry, such as O(5), O(1,3), and O(3) with scalar, vector, and second-order tensor data types. Some embodiments of the invention are used for image classification using an MLP-mixer, predicting N-body dynamics using a message-passing neural network (MPNN), and solving PDEs using a Fourier neural operator (FNO). Furthermore, in some embodiments, the higher-order tensor representations in this invention can be used for group equivariant fine-tuning, which is superior to existing equivariant fine-tuning methods.

[0006] One of the objectives of this invention is to provide a system and method that can construct and provide neural networks that are equivariant to arbitrary matrix groups and easily extend to larger networks and various architectures such as multilayer perceptrons (MLPs), CNNs, and MLP-mixers, given a correct group representation of the input data.

[0007] For example, a group representation can be in the form of a dataset, such as a tensor, scalar, matrix, vector, or pixel dataset of an image. In addition, a “group” can be a type of data processing, data transformation, data conversion, and data manipulation, including algebraic structures, such as translation of a dataset (T(n)), rotation of a dataset (SO(n)), rotation and reflection (O(n)), and Lorentz group (O(1,n)).

[0008] Different existing solutions have different drawbacks. Parameter-sharing methods are limited to discrete groups. Manipulable networks are generally difficult to construct and computationally expensive. Vector neurons also use simple group representations as input and use scalar weights to combine these representations, employing nonlinearities that respect their equivariance. However, they are designed only for SO(3) groups and use only first-order tensor representations, i.e., vectors. Frame averaging methods involve averaging over any frame or using auxiliary equivariant networks, which increases computational complexity.

[0009] To overcome the shortcomings of earlier methods, this disclosure describes a robust machine learning system with a group representation network (GRepsNet) as its core component. GRepsNet replaces the scalar representation of classical neural networks with tensor representations of different orders to obtain expressive equivariant networks. GRepsNet is applicable to arbitrary matrix groups (data transformation groups) and can also utilize high-order tensor representations, unlike vector neurons.

[0010] This disclosure provides a simple, provable equivariant architecture called GRepsNet, which is equivariant to several groups that compete with EMLP on arbitrary matrix groups using scalar, vector, and second-order tensor representations such as O(5), O(3), and SO(1,3).

[0011] In a preferred embodiment, GRepsNet, with its simple representation, works well when combined with several architectures used in different fields, such as image classification, PDE, and N-body dynamics prediction, respectively, using MLP-mixers, FNO, and MPNN.

[0012] Some implementations use CNNs to provide second-order tensor features for isotropic image classification. When used for fine-tuning, second-order tensor features can outperform first-order representations such as those used for isotropic representations.

[0013] Some embodiments of this disclosure provide a robust machine learning system with a group representation neural network (GRepsNet) as its core deep neural network architecture, which can be an easily constructed and scalable equivariant architecture for arbitrary matrix groups.

[0014] According to some embodiments of this disclosure, a robust machine learning system is provided. The robust machine learning system may include: a processor; and a memory storing a robust machine learning program and a group representation network (GRepsNet), wherein the robust machine learning program runs on the processor, wherein the processor is configured to: receive an input dataset represented by an input group representation via an input layer connected to the GrepsNet; and transmit the input dataset through the GrepsNet, configured to generate an output represented by an output group representation from the input dataset, wherein the GRepsNet is designed to be equivariant to the transform group.

[0015] Furthermore, some embodiments of the present invention provide a data transformation system. This system may include: a memory configured to store a group representation network (GRepsNet) and an instruction set for performing data transformation using the GRepsNet; and a processor coupled to the memory and configured to execute the instruction set, the instruction set including the steps of: providing an input dataset represented by an input group representation to an input layer connected to the group representation network (GRepsNet); transforming the input dataset using the GRepsNet configured to generate an output dataset represented by an output group representation from the input dataset, wherein the GRepsNet is designed to be equivariant to a predetermined transformation group; and outputting the output dataset from an output layer connected to the GRepsNet.

[0016] Furthermore, some embodiments provide a computer-implemented method for robust machine learning implemented in a computing system, the computing system including: a processor; and a memory storing a robust machine learning program and a group representation network (GRepsNet), wherein the robust machine learning program runs on the processor, and wherein the computer-implemented method includes the steps of: receiving an input dataset represented by an input group representation via an input layer connected to the GrepsNet; and transmitting the input dataset through the GRepsNet, the GRepsNet being configured to generate an output represented by an output group representation from the input dataset, wherein the GRepsNet is designed to be equivariant to the transform group. Attached Figure Description

[0017] The present disclosure is further described in the following detailed description with reference to several accompanying drawings, which are illustrated by way of non-limiting examples of exemplary embodiments of the present disclosure, wherein the same reference numerals denote similar parts in various views of the drawings. The drawings shown are not necessarily drawn to scale, but generally focus on illustrating the principles of the embodiments of the present disclosure.

[0018] While the accompanying drawings illustrate embodiments currently disclosed, other embodiments are contemplated as indicated in the discussion. This disclosure presents illustrative embodiments by way of presentation, not limitation. Many other modifications and embodiments can be devised by those skilled in the art, falling within the scope and spirit of the principles of the embodiments currently disclosed.

[0019] [ Figure 1A ]

[0020] Figure 1A This is a block diagram illustrating an implementation of the data transformation system; [ Figure 1B ] Figure 1B This is a block diagram illustrating an implementation of the data transformation engine in a data transformation system; [ Figure 2A ] Figure 2A An example of an invariant function that is equivariant to translation and rotation is shown; [ Figure 2B ] Figure 2B An example of an invariant function that is equivariant to translation and rotation is shown; [ Figure 3A ] Figure 3A The overall architecture of the equivariant neural network is shown, and how equivariance is achieved at its output. [ Figure 3B ] Figure 3B The overall architecture of the equivariant neural network is shown, and how invariance is achieved at its output. [ Figure 4 ] Figure 4 An example GRepsNet layer with inputs of types T0, T1, and T2 and outputs of the same type according to an embodiment of the present invention is shown; [ Figure 5 ] Figure 5 An example internal configuration of a GRepsNet layer according to an embodiment of the present invention is shown, which has one T0 (scalar), three T1 (vector) inputs (each with four elements), and two T1 outputs; [ Figure 6 ] Figure 6 An example method for constructing an input group representation for image classification and obtaining rotation-invariant features from a convolutional neural network, according to an embodiment of the present invention, is shown. [ Figure 7A ] Figure 7A The results and comparisons of GRepsNet with MLP and EMLP for three tasks according to an embodiment of the present invention are shown—(a) O(5)-invariant regression, O(3)-isovariant regression and (c) SO(1,3)-invariant regression for different types of input and output representations; [ Figure 7B ] Figure 7BThe results and comparisons of GRepsNet with MLP and EMLP for three tasks according to an embodiment of the present invention are shown—(a) O(5)-invariant regression, O(3)-isovariant regression and (c) SO(1,3)-invariant regression for different types of input and output representations; [ Figure 7C ] Figure 7C The results and comparisons of GRepsNet with MLP and EMLP for three tasks according to an embodiment of the present invention are shown—(a) O(5)-invariant regression, O(3)-isovariant regression and (c) SO(1,3)-invariant regression for different types of input and output representations; [ Figure 8 ] Figure 8 A comparison of training times for GRepsNet, MLP, and EMLP for different tasks is shown according to embodiments of the present invention; [ Figure 9 ] Figure 9 The image classification results of MLP-Mixer, GRepsMLP-Mixer-1, and GRepsMLP-Mixer-2 on three datasets according to embodiments of the present invention are shown. [ Figure 10 ] Figure 10 The results of solving the PDE according to an embodiment of the present invention are shown, and the performance of FNO, GFNO, GRepsFNO-1 and GRepsFNO-2 are compared. [ Figure 11 ] Figure 11 An experimental performance comparison of GRepsGNN and EGNN for predicting N-body dynamics according to an embodiment of the present invention is shown; [ Figure 12A ] Figure 12A This illustrates the effect of using T2 representation at different depths in the network for rot90-CIFAR10 image classification experiments according to an embodiment of the present invention; [ Figure 12B ] Figure 12B This illustrates the effect of using T2 representation at different depths in the network for rot90-CIFAR10 image classification experiments according to an embodiment of the present invention; [ Figure 13 ] Figure 13 A comparison of the isotone results of T1 representation and T2 representation according to an embodiment of the present invention is shown; [ Figure 14A ] Figure 14A An example of an implementation of GrepsNet according to an embodiment of the present invention is shown, wherein (a) residual connections are added to the GRepsNet layers, and (b) the first k layers consist of T1 layers, and the extracted features are then converted into T2 tensors and processed by the T2 layers; [ Figure 14B ] Figure 14B An example of an implementation of GRepsNet according to an embodiment of the present invention is shown, wherein (a) residual connections are added to the GRepsNet layers, (b) the first k layers consist of T1 layers, and then the extracted features are converted into T2 tensors and processed by the T2 layers; and [ Figure 15 ] Figure 15 An example of a robust machine learning system according to an embodiment of the present invention is shown. Detailed Implementation

[0021] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with a description of the feasibility of implementing one or more exemplary embodiments. Various changes may be made to the function and arrangement of the elements without departing from the spirit and scope of the subject matter disclosed in the appended claims.

[0022] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that embodiments may be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form so as not to obscure the embodiments with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments. Furthermore, the same reference numerals and names in the various figures may indicate the same elements.

[0023] Furthermore, the various implementations can be described as processes depicted as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. Although flowcharts can describe operations as sequential processes, many operations can be executed in parallel or concurrently. Additionally, the order of operations can be rearranged. A process may terminate upon completion of its operations, but may have additional steps not discussed or included in the diagram. Furthermore, not all operations within any specifically described process will occur in all implementations. A process can correspond to a method, function, procedure, subroutine, subroutine, etc. When a process corresponds to a function, the termination of the function can correspond to the function returning to the calling function or the main function.

[0024] Furthermore, implementations of the disclosed subject matter can be carried out, at least partially, manually or automatically. Manual or automatic implementation can be performed, or at least assisted in, using machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments for performing the necessary tasks can be stored in a machine-readable medium. The processor can then perform the necessary tasks.

[0025] Group sum representation theory

[0026] A group is a set And a binary operator " Thus, the axioms of the group are satisfied: a) Closure: for all , b) Associative law: applicable to all , c) Unit element: for any ,exist , making d) Inverse: for each All exist Make .

[0027] For a given set via mapping Define group exist Group actions on the top make it possible to target all , And for all and , ,in, yes The unit element. For clarity of context, we will Simply written as .

[0028] Given function If targeting all and , Then we call it a function. for Equivalent.

[0029] set up express The group of all invertible matrices of dimension. Then, for the group... , The linear group representation is defined as a mapping , making It is an identity matrix. Dimension The group representation of is a vector space Linear group actions on.

[0030] Given a certain vector space The above is aimed at Some basic linear group representations In this case, we apply the Kronecker sum. (Kronecker product) Kronecker product tensor duality To construct tensor representations. Each of these tensor operations on the vector space yields a corresponding new group action. The corresponding group action becomes Suppose that for of and They are vector spaces and Group actions on top. So, for Group actions by Give, and for Group actions by Provided.

[0031] Corresponding to the base representation The tensor is represented as A tensor, specifically a first-order tensor, and Represents a scalar. Typically, express Tensor of order. Furthermore, tensor and The Kronecker product gives tensor of order We use symbols to indicate tensor Becronek product. Two types are... and The tensor of Kronecker and gives a type of Tensors of the same type. Finally, tensors of the same type. of The tensors Kronecker and are written as .

[0032] These are four types of provable isovariant networks, and how our work differs from them: Parameter sharing method: Existing techniques for constructing group equivariant architectures involve sharing learnable parameters within the network to guarantee equivariance, such as CNNs, GCNNs, and Deepsets. However, all of these methods are limited to discrete groups, unlike this disclosure, which can handle equivariance for arbitrary matrix groups.

[0033] Controllable network: Another popular approach for constructing group equivariant networks is to first compute the basis of the equivariant function space and then linearly combine these basis vectors to construct the equivariant network. This approach can also handle continuous groups. Several popular architectures employ this approach, such as manipulable CNNs, E(2)-CNNs, and tensor field networks. However, these methods are computationally expensive and limited to specific groups. Equivariant MLP (EMLP) offers a concise solution to equivariant constraints on linear layers of arbitrary matrix groups, for computed basis vectors and constructing equivariant networks based on the computed basis. Due to the simplicity of the approach in this disclosure, it can be used on several large datasets, unlike EMLP, whose application is mainly limited to simple test datasets.

[0034] Equivariant MLP (EMLP)

[0035] Given the input and output types for a matrix group, the corresponding tensor representation can be derived from the given basis group representation. Using these tensor representations, the space of linear equivariant functions can be solved directly from the equivariant constraints corresponding to the tensor representations. Recent work has proposed a solution to these constraints by computing the basis of the linear equivariant space and constructing an equivariant MLP (EMLP) based on the computed basis. Our work comes closest to this work because we use the same data representations, but we propose a simpler architecture for equivariance to arbitrary matrix groups. Due to the simplicity of our method, our novel architecture can be applied to several larger datasets, in stark contrast to their work, where experiments are mostly limited to synthetic experiments. Furthermore, it is well known that using these basis representations is computationally expensive.

[0036] Representation-based approach: For continuous networks, a simple alternative to manipulable networks is to simply represent the data using group representations, combine these representations using only scalar weights, and construct equivariant networks using nonlinearities that respect their equivariance. This simple approach has been used for vector neurons, providing fairly good performance for SO(3) groups. This disclosure provides a system and computer implementation method that is more general and applicable to arbitrary matrix groups, and uses higher-order tensor representations, whereas vector neurons use first-order tensor representations, i.e., vectors.

[0037] Universal scalar

[0038] Another recent work proposes a method to circumvent the need for explicitly using these equivariant bases. The first fundamental theorem of the invariant theory of Euclidean groups O(d) states that "a function of vector inputs returns an invariant scalar if and only if it can be written as a function of the invariant scalar product of only the input vectors." Inspired by this theorem and related theorems for equivariant vector functions, this work characterizes equivariant functions for various Euclidean and non-Euclidean groups. These further advance the construction of neural networks that take the invariant scalar product of a given tensor data as input. However, the number of invariant scalars for N tensors in the data points increases with N... 2 However, this approach is impractical for most real-world machine learning datasets. Therefore, as in EMLP, their application is largely limited to synthetic datasets.

[0039] Furthermore, they also show that even if the number of scalars produced is similar to N 2 Proportional growth occurs when the dimension of the data is d, and the number of these scalars is approximately N×(d+1), sufficient to construct an invariant function. However, finding this set of scalar quantum numbers may not be easy for real-world datasets such as images. Therefore, this disclosure uses a deep network with equivariant characteristics, directly taking N tensors as input, instead of N... 2 This also avoids the need to use scalar inputs.

[0040] Another approach to obtaining group isotropic properties is to use frame averaging, where averaging is performed on isotropic frames corresponding to each input to obtain an isotropic output. This method works for both discrete and continuous groups, but requires constructing these frames, either by designing them to be fixed or by learning them using an auxiliary isotropic neural network. This disclosure differs substantially from this approach because our method does not involve averaging on any frame or using an auxiliary isotropic network. For the special case of discrete groups, the concept of frame averaging is closely related to both parameter sharing and representation methods. Therefore, in the context of “isotropic” properties, we show how to directly incorporate higher-order tensor representations into their frame averaging methods.

[0041] Figure 1A This is a block diagram illustrating an implementation of a data transformation system. In the diagram, data transformation system 12 receives source data 11, such as a set of data files, one or more source databases, and / or other data sources such as a set of data images. In some cases, data transformation system 12 transforms / converts the data and provides the transformed data to a target data system 13 for storage in a target database 15. In various implementations, data transformation is performed by passing the data through a group representation network (GRepsNet).

[0042] Figure 1B This is a block diagram illustrating an implementation of a data transformation engine in a data transformation system. In the diagram, a data transformation engine 112 based on the GRepsNet model is included. Figure 1A In the data transformation system 12, it is configured to transform / convert from the source database 111 and output to the target database 113.

[0043] In some implementations, the data transformation engine 112 may be stored on a non-transitory computer-readable medium. In this case, a set of instructions for data transformation is stored on the non-transitory computer-readable medium. When executed by one or more processors, the set of instructions causes one or more processors to provide at least an input dataset represented by an input group representation to an input layer connected to a group representation network (GRepsNet), transforming the input dataset by using GRepsNet configured to generate an output dataset represented by an output group representation from the input dataset, wherein GRepsNet is designed to be equivalent to a predetermined transformation group. The one or more processors then output the output dataset from the output layer connected to GRepsNet to a target database.

[0044] Figure 2A An example of an invariant function with equivariance to translation and rotation according to an embodiment of the invention is shown. The figure illustrates group equivariance by way of an example showing equivariance to translation and rotation. Here, the function... f It exhibits isovariance to translation and rotation. Translations and rotations applied at the input are reflected at the output. The function for calculating edges in an image is expressed as... f So, it is expected that when the input rotates, from f The output edge map is also rotated by the same amount. This means the function f It should be isovariant with respect to rotation. In general, if for all g in group G, we obtain... f ( g (x)) = g ( f(x)), then the function that receives input x belonging to set X. f It is equivariant for group G.

[0045] Figure 2B Examples of invariant functions with equivariance to translation and rotation according to some embodiments of the present invention are shown. The figure illustrates the concept of group invariance and shows examples of translation and rotation invariance. Here, the function... h It identifies objects in an image and is invariant to translation and rotation. Regardless of the translation and rotation applied to the input, h The outputs are always the same. As an example, consider an image recognition application. Regardless of the rotation and / or translation applied to the input, the objects in the image remain the same. That is, the image classification function... h It should be invariant to rotations and translations of the input. In general, for an input set X with element-valued x and a group G with element-valued G, if for all x belonging to X, for all g belonging to G, ... h (g(x)) = h (x), then the function h Actions on G are invariant.

[0046] Group equivariance and invariance are important and desirable properties for designing robust machine learning systems that use neural networks. Group equivariance plays a key role in the success of several popular architectures, such as translational equivariance in convolutional neural networks (CNNs) for image processing, 3D rotational equivariance of point clouds, and equivariance of arbitrary groups in group convolutional neural networks (GCNNs).

[0047] Figure 3A and Figure 3B A general architecture for deep isovariant neural networks is shown. Figure 3A The general architecture of a deep equivariant neural network with an equivariant output of 330A is described. Each layer of the equivariant neural network is equivariant with respect to the transformation group. Figure 3B This paper describes the general architecture of a deep equivariant neural network with an invariant output of 330B. Each layer of an equivariant neural network is equivariant with respect to the transformation group. Invariance at the output is achieved by pooling in the group dimension of the output. This type of neural network consists of multiple layers, each of which is equivariant with respect to the group of interest. As equivariant layers are stacked one after another, the output of the stack remains equivariant with respect to the group. If invariance at the output is required, an additional layer (often a pooling layer) is added to pool the output in the group dimension to create an invariant output.

[0048] Some embodiments of the present invention are motivated to overcome the following technical problems.

[0049] Designing efficient equivariant networks can be challenging, both because they require domain-specific knowledge and because they are computationally inefficient. For example, several works have focused on designing architectures for different groups such as the special Euclidean group SE(3), the special Lorentz group O(1,3), and the discrete Euclidean group. Furthermore, some of these networks can be computationally inefficient, prompting the design of simpler, lighter-weight equivariant networks, such as E(n) equivariant graph neural networks for graphs and vector neurons for point cloud processing.

[0050] Recent work proposed an algorithm that constructs an equivariant MLP (EMLP) for arbitrary matrix groups when data is provided using tensor polynomial representations. This method directly computes the basis of the equivariant MLP and requires minimal domain knowledge. While EMLPs are concise, they are limited to MLPs or can be used as sub-components in larger networks, offering little benefit for making more general architectures equivariant as a whole. Furthermore, using equivariant basis functions is often computationally expensive, leading to several group-specific, efficient architectures. However, recent research has shown that scaling asymmetric models can yield results comparable to equivariant models.

[0051] Architecture

[0052] First, the construction of GRepsNet layers is described. Then, the GRepsNet model is constructed by stacking several GRepsNet layers. Let the input of the GRepsNet layer be... Type, among which, s is a scalar, indicating that the input has Type Tensor.

[0053] For simplicity, each GRepsNet layer also has several T... i Layer. Each The layer performs two operations: a) converting the input tensor to the appropriate tensor type, and b) processing the converted tensor using a neural network layer (such as an MLP or CNN). If the representation used is not a regular representation, it is assumed that the input to the GRepsNet model is always a representation with... The representation consists of some tensors, which is not a strong assumption that helps keep the construction simple.

[0054] Because the system and computer implementation method of this disclosure provide a simpler and more manageable network compared to known architectures, the computational efficiency of the computer / processor used in the system is significantly improved, and the computational performance can be significantly enhanced, thereby reducing the power consumption of the computer (processor and memory).

[0055] Convert tensor type The input to the layer is categorized before being passed through the appropriate neural network layer (such as an MLP or CNN). ( )and (If available) Convert all inputs together to type .

[0056] Input type Only or Time is converted to Otherwise, not Used in layers .when hour, Written as ,in, and . Tensors are used ,and As Obtained.

[0057] when At that time, by using appropriate invariant operators, such as the Euclidean norm of the Euclidean group, the type can be... Each input is converted to a type These design choices result in designs that are both lightweight and expressive, as proven by experience. The following section will describe in detail how these input data were processed.

[0058] When processing transformed tensors without using regular representation, firstly, Layers enable all types Tensors or scalars are processed by neural networks such as MLPs or CNNs. Since the input is an invariant scalar, the output is always invariant, therefore for tensors or scalars used in neural networks... There are no restrictions on the number of layers in a neural network.

[0059] The output of the layer is called .for of For the layer, we simply need to pass it through a linear neural network without pointwise nonlinearities or bias terms to ensure that the output is equivariant. Let's call this output... Then, in order to mix better tensor and Tensors, we will Updated to , where inv( These are simply invariant functions such as the Euclidean norm for the Euclidean group. Finally, we make Through another linear layer, without any bias or point-by-point nonlinearity, to obtain .

[0060] When the size is When a discrete group is represented using a regular representation, the neural network used also contains pointwise nonlinearities and biases, because they do not affect the equivariance of the regular representation.

[0061] Overview of steps in the layer: a) Convert all input representations to , b) Scalar mixture of all Tensors and nonlinearities are only used for regular representations, i.e., in the case of images.

[0062] c) Layers are always composed of pointwise nonlinearities (such as ReLU).

[0063] This architecture is naturally equivariant, because for The mixture of tensors and nonlinear scalars preserves equivariance. For canonical representations, nonlinearity preserves equivariance.

[0064] Figure 4 An example of a GRepsNet layer is shown, where the input representation is... The representation, and the output representation can be determined by... express.

[0065] Figure 5 An example of a GRepsNet layer is shown, consisting of one T0 (scalar) 510, three T1 (vector) inputs 511 each with four elements, one T0 output 530, and two T1 outputs 531. The tensors are processed internally to ensure the outputs remain equivariant. In this case, the parameters a, b, c, d, e, f 512 are scalars and are trainable parameters (weights) within the layer. The input tensors are linearly combined to form an intermediate representation 515. The "invariant" computation 520 is non-linear (e.g., Euclidean or Minkowski norm, depending on the transformation group). The invariance then interacts with the intermediate outputs from the linear combination of the T1 tensors and the T0 tensors 525.

[0066] Proof of the equivariance of the GRepsNet architecture

[0067] Since stacking equivariant layers preserves the equivariance of the resulting model, the equivariance of the GRepsNet model is directly inherited.

[0068] First, consider the regular representation. Note that in the regular representation for discrete groups, the group dimension is treated as the batch dimension. Therefore, any permutation in the input will naturally appear in the output, resulting in an equivariant output.

[0069] Now consider the non-regular representations considered in this disclosure, such as matrix-based representations. Figure 4 Example GRepsNet layers with inputs of types T0, T1, and T2 and outputs of the same type are shown. i The layer first transforms all inputs into T. i The data is then processed by neural network layers such as MLPs. Figure 4 This explains the assumption that the input to the GRepsNet layer is of type Please note the composition of the tensors. The output of the layer remains unchanged, then Layer output is equivalent Tensor.

[0070] Since all inputs to the network are of an unchanging type ,therefore The output of the layer is obviously unchanged.

[0071] Now, pay attention Layers. Think back, The layers consist only of linear networks, without any bias terms or pointwise nonlinearities. Assume the linear networks are given by stacks of linear matrices. Any such linear combination performed by matrices preserves equivariance; therefore, stacking these matrices will still preserve the equivariance of the output. Let the type be... The input tensor is That is, the existence of types of There are tensors, and the size of the representation of each tensor is equal to 1 / 2. Consider a matrix. multiplied by get ,in, yes A linear combination of input tensors, where each tensor is... Type. Let tensor group transformations on Given. Then, the input after group transformation is... Provided. pass The output of the layer is Given, where the penultimate equation comes from the associative property of matrix multiplication. Therefore, each All layers are isomorphic.

[0072] Synthesis experiments related to EMLP

[0073] Three regression tasks were used for comparison with EMLP and MLP: O(5)-invariant regression, O(3)-isovariant regression, and O(1,3)-invariant regression. The corresponding (input type, output type) for these tasks are ( , ), ( , )and( , Following the design of GRepsNet, we use an architecture with a given input and output type. As mentioned above, once the tensor type is converted, we use some neural network to process the tensors. Here, we use a simple MLP for this processing task.

[0074] -Invariant Model

[0075] Input by via Layers and The first layer consists of layers It consists of two tensors of the same type. All All layers are composed of MLPs. In some implementations, the number of output tensors equals the channel size. The following is from... Layers and Three similar layers are composed of layers, all of which are based on Tensors are used as inputs, and tensors of the same type are output. Furthermore, these layers use residual connections. Finally, the norm is taken... tensor conversion Tensor, and make it pass through the final The layer provides the output.

[0076] - Equivariant Model

[0077] The input consists of 5 tensors, each of which is of type 1. and The first layer of the model converts them into types. Tensors. In some implementations, the number of tensors obtained is equal to the channel size. Next are two layers of input and output types. These layers also use residual connections. Then, the resulting output is... Tensor through another The layer that provides the final output.

[0078] -Invariant Model

[0079] Aside from using the Minkowski norm instead of the Euclidean norm to obtain invariant tensors, this design strictly follows the above principles. - Design of invariant networks.

[0080] Image classification using MLP-mixer

[0081] We will now describe a rotationally equivariant neural network architecture within the GRepsNet framework.

[0082] Figure 6 An example of constructing a group representation for an image classification experiment with the C4 group (a group with discrete rotations of 0, 90, 180, and 270 degrees) is shown. The figure illustrates how to achieve equivariance using the correct input representation and how to obtain rotation invariance through features from four sub-images in the average input representation. In this case, randomization is used. Rotated dataset CIFAR10 (which we call rot90-CIFAR10) and its natural... The network was tested using the symmetric Galaxy10 and EuroSAT datasets.

[0083] The image was effectively converted Rotating group of Tensors, which make designing more efficient than traditional regular representations, for example, by repeating the transformed image in the input in isotone. Once the regularity is obtained... This indicates that, in addition to batch, channel, and spatial dimensions, an additional group dimension will be obtained from the data. The group dimension is processed in the same way as the batch dimension.

[0084] The MLP mixer consists of eight layers, each containing two smaller layers: a spatial MLP mixer layer with a hidden dimension of 64 and a channel MLP mixer layer with a hidden dimension of 512. Two GRepsMLP mixer designs are provided: GRepsMLP-mixer-1 and GRepsMLP-mixer-2. GRepsMLP-mixer-1 always treats the group dimension as the batch dimension, while GRepsMLP-mixer-2 additionally uses nonparametric fusion among the features in the group dimension. Here, layer normalization along the group dimension, which has no learnable parameters, is used as the fusion layer.

[0085] Now describe the construction based on the input image. The process of tensors. Given an image. It can be written as Among them, for , .set up express A rotating group. In Upper definition Group operations, that is, .

[0086] structure The following express: Aside from when using an intra-frame mixer, matrix Each of the four entries is considered a separate channel with no data flow between them. Furthermore, all channels share the same parameters, such as... .So The output will be .

[0087] We can now verify the isotropy of the four rotations obtained using this representation. First, let's point out... Therefore, the output is: Obviously, yes The permutation is performed. To obtain invariance, the four channels are averaged. This method is computationally more efficient than using the image after four transformations as input, as was the case with earlier work on isotonality. This efficient representation still retains the benefits of group isovariance.

[0088] Solving using PDEs with Fourier neural operators

[0089] It has now been demonstrated that rotationally equivariant Fourier neural operators (FNOs) can be constructed using the GRepsNet framework and used to predict the future states of systems that follow partial differential equations.

[0090] Two versions of the incompressible Navier-Stokes equations (relative to) This application can be demonstrated by showing both symmetrical and asymmetrical rotations.

[0091] Similar to image classification, the group dimension is handled in a manner analogous to the batch dimension. The data is then passed sequentially through four FNO layers. The ability to directly utilize various models (such as FNO) while maintaining equivariance underscores the simplicity of our approach.

[0092] Two GRepsFNO designs similar to the GRepsMLP mixer are described: GRepsFNO-1 and GRepsFNO-2. GRepsFNO-1 always treats the group dimension as the batch dimension, while GRepsFNO-2 additionally uses nonparametric fusion between features in the group dimension. Here, features are partitioned along the group dimension by the standard deviation across that dimension. Layer normalization is avoided here because it requires implementation in the complex Fourier domain. Instead, division by the standard deviation is a simpler alternative for the same purpose.

[0093] For FNO, the traditional group representation is used, i.e., The reason for this choice is that, for FNO, all frequency modes are preserved by using the converted input.

[0094] Predicting N-body dynamics using graph neural networks

[0095] It has now been demonstrated that group isovariant graph neural networks (GNNs) can be constructed using the GRepsNet framework, called GRepsGNN, and used to predict N-body dynamics.

[0096] Considering the given The problem of predicting the dynamics of a charged particle given its charge and initial position. Each particle is placed in the figure... At the node, where, and It is a set of vertices and edges. Let... The edge properties are And set For layers of a message-passing neural network (MPNN) The node at the location Node features. MPNN has edge update. and node updates , ,in, and These are the MLPs corresponding to edge updates and node updates, respectively.

[0097] GRepsGNN is designed by modifying the MPNN architecture. In GRepsGNN, specific methods are used for... and The two edge updates of the tensor, and the updates for... One node update is performed. Two edge updates are... , ,in, From the perspective of the Euclidean group Obtained from tensor tensor, yes Layer MLP, and It is composed of MLP The layer has no point-by-point nonlinearity or bias. The final edge update obtained is... Finally, node updates are performed by... Given, among which, It is an MLP without any pointwise nonlinearity or bias. Therefore, the final node update is Tensor.

[0098] use Indicates the need for equal-variable fine-tuning

[0099] Here, the effectiveness of features extracted by CNN is demonstrated. Indicates the use of ratios and equal tones This indicates better equivariant characteristics. This is based on the following intuition: Represents the mixing ratio between features in the group dimension. This is better. This is because, It indicates that it originates from two The outer product is represented. This is similar to how bilinear CNNs use outer products on features to efficiently process features for fine-grained classification. The GrepsNet approach proposed in this disclosure differs from bilinear CNNs in that our outer product is in the group dimension that preserves equivariance, while the work of bilinear CNNs is independent of group equivariance.

[0100] rot90-CIFAR10 was first used for testing Indicates ratio This represents the assumption that better performance is provided. A CNN with 3 convolutional layers followed by 5 linear layers is used. The network is initially... Layer usage This indicates that all subsequent layers will use... This indicates that, as verified, when the last few layers use... When represented, optimal performance can be obtained.

[0101] Based on this observation, a construction was made isotones are extracted from the pre-trained model. Features, but convert them to their original form before providing invariant output. The pre-trained ResNet18 was fine-tuned on the rot90-CIFAR10 and Galaxy10 datasets. This was achieved by using the extracted features... Indicate, prove An isotone is better than an isotone.

[0102] Additional network design

[0103] Figure 14A An example implementation of GRepsNet is shown, in which residual connections are added to the GRepsNet layers. Figure 14B An example implementation of the GRepsNet architecture with T2 representation is shown. k The layer consists of a T1 layer, and the extracted features are then converted into a T2 tensor, which is processed by the T2 layer. Finally, a T0 tensor, or scalar, is obtained as the final output. In this case, a diagram is shown for... CNN and isotropic architectures, in which, the front Layer by Layers are composed to extract features, and then the extracted features are transformed Tensor, then by The layers are processed. Finally, the result is... Tensors, or scalars, are the final output.

[0104] Datasets and Experiments

[0105] Comparison with EMLP

[0106] Dataset: Three regression tasks are considered here: O(5)-invariant task, O(3)-isovariant task, and O(1,3)-invariant task. In the O(5)-invariant regression task, we have type Input and type Output Then, for O(3)-equivariant tasks, the input type of This corresponds to 5 masses and their positions. The output is of type [type missing]. inertia matrix Finally, for tasks with the same variation of O(1,3), we consider electron- Sub-scattering ( Task. Here, the input is the type. Corresponding to input and output electrons and The four momentum of the child, and the output is of type [type]. Matrix elements.

[0107] Experimental setup: MLP, EMLP, and GRepsNet were trained on the datasets discussed above, which have different sizes over 100 epochs. For each task and model, a model size between small (100 channels) and large (384 channels) was chosen. The learning rate ranged from... Choose from the options. Generally, MLP and EMLP provide the best results for large model sizes, while GRepsNet produces better results for small model sizes.

[0108] In the O(5)-invariant regression task, for MLP and EMLP, using The learning rate and channel size are 384. For GRepsNet, the following is used: The learning rate and channel size are 100. For O(3)-equivariant tasks, the learning rate is used for all models. And channel size 384. For an O(1,3)-invariant regression task, The optimal learning rate yields the best results for all models. Furthermore, a channel size of 384 is chosen for MLP and EMLP, while a channel size of 100 is selected for GRepsNet to achieve the best results.

[0109] Observations and results: Figure 7A , Figure 7B and Figure 7C A comparison of GRepsNet with EMLP and MLP is shown. Figure 7A This illustrates an O(5)-invariant synthetic regression task with input type 2T1 and output type T0. Figure 7B The example illustrates an O(3)-isovariant regression, with inputs being the mass and position of 5 points represented using type 5T0 + 5T1, and outputting a type T2 inertia matrix. Figure 7C The SO(1, 3)-invariant regression is shown, which calculates the matrix elements in electron-mu particle scattering, with an input type of 4T1 and an output type of T0.

[0110] from Figure 7A , Figure 7B and Figure 7C As can be seen, GRepsNet performs comparably to EMLP across all tasks and significantly outperforms non-equivariant MLPs. These figures also demonstrate that GRepsNet requires far less data to achieve the same performance as MLPs. Figure 8 A comparison of training times for GRepsNet, MLP, and EMLP on different tasks according to embodiments of the present invention is shown; the training time per epoch (in seconds) for a model with the same channel size of 384 on a dataset of size 1000. GRepsNet provides the same equivariance as EMLP, but at a significantly lower computational cost than MLP. GRepsNet is computationally much more efficient than EMLP, but its cost is only slightly higher than that of simple MLP. This demonstrates that GRepsNet can provide performance comparable to EMLP on equivariant tasks. Furthermore, GRepsNet's lightweight design facilitates its use on larger datasets.

[0111] MLP and GrepsNet have comparable training times per epoch, while EMLP is significantly more time-consuming. Therefore, although EMLP performs well on equivariant tasks, it cannot scale to larger datasets of real importance; the training time per epoch (in seconds) for a model with the same channel size of 384 on a 1000-size dataset is considerably longer. GRepsNet provides the same equivariance as EMLP but at a much lower computational cost.

[0112] Image classification using MLP-mixer

[0113] Given an input image, the system needs to classify it into a predetermined set of categories.

[0114] Datasets and experimental setup: The rot90-CIFAR10 (CIFAR10 with randomized rot90 transformation), Galaxy10, and EuroSAT image datasets were used for image classification experiments. Note that for these datasets, the input images have no preferred orientation, and orientation is a confounding variable; rotation-invariant features are required for classification. The non-equivariant MLP-mixer used in this disclosure has… Two preferred implementations of the rot90-equivariant MLP mixer (GRepsMLP-Mixer-1 and GRepsMLP-Mixer-2) are compared. GRepsMLP-Mixer-2 simply adds a nonparametric early fusion operation in the group dimension to the GRepsMLP-Mixer-1 architecture. Each model has 8 MLP mixer layers, with a block size of 16 for the Galaxy10 dataset and 4 otherwise. Each model is configured with a learning rate of... The training consists of 100 rounds, including 5 warm-up rounds.

[0115] In each of the models, each layer is further composed of two smaller layers: a) one that applies layer normalization in the channel dimension, followed by two MLP layers, and b) another that applies layer normalization in the spatial dimension, followed by two MLP layers, as is done in a conventional MLP mixer. For the GRepsMLP-Mixer-1 of this invention, the conventional scalar representation of the MLP mixer is... This indicates replacement. For GRepsMLP-Mixer-2, we added an early fusion layer without any additional parameters. The early fusion layer is constructed as follows: for the rot90 group... The four distinct components are represented using a length of 4. The group dimension is represented by a simple layer normalization for fusion.

[0116] For training non-equivariant and equivariant MLP-mixer models, each model uses a batch size of 128 and a learning rate of... The training was performed for 100 epochs, using 5 minimum learning rates. The preheating rounds use a cosine scheduler and an Adam optimizer, where The weight decays to .

[0117] Results and Observations: Table 2 shows that the GRepsMLP-Mixer model significantly outperforms the non-equivariant MLP-Mixer across all datasets. Furthermore, the benefit of group equivariance was observed. Note that the early fusion layers in GRepsMLP-Mixer-2 contribute to its superior performance compared to GRepsMLP-Mixer-1 on both datasets, and are generally comparable to GRepsMLMP-Mixer-1. Therefore, fusing these features early is generally helpful.

[0118] Solving PDEs using FNO

[0119] The model described in this invention uses regularization. It is stated that its architecture is almost identical to the FNO model used. In addition to the late fusion layer present in GRepsFNO-1, GRepsFNO-2 also uses an additional early fusion layer.

[0120] Datasets and Experimental Setup: Two versions of the incompressible Navier-Stokes equations were considered. The first version is the Navier-Stokes equation with no symmetries in the data (NS dataset), and the second version has 90... Rotational symmetry (NS-SYM dataset). The general Navier-Stokes equations under consideration are written as follows: i) (1) ii)

[0121] in, Point vorticity at that location It is the initial vorticity. for The speed at that location, and is the viscosity coefficient. This represents the external forces that affect fluid dynamics. The task here is to determine the forces that affect fluid dynamics given a previous... Predicting the vorticity values ​​at all points in the domain over a given step for a specific step. domain The vorticity at all points. When Compared to When rotation remains constant, the solution is identical; otherwise, it is not. For non-invariant forces, And as a constant force, We use Each previous step serves as input to the NS dataset, using As input to the NS-SYM dataset, and predict The model uses a batch size of 20 and a learning rate of... Train 100 times.

[0122] Results and observations: Figure 9 The image classification results of MLP-Mixer, GRepsMLP-Mixer-1, and GRepsMLP-Mixer-2 on three datasets according to embodiments of the present invention are shown. The figure shows the mean (standard) of the relative mean squared error as a percentage on the three seeds. In this case, the mean (standard deviation) of the test accuracy for three runs with different network weights randomly initialized is used for the image classification experiment. Both GRepsMLP-Mixer-1 and GRepsMLP-Mixer-2 use a regularized T1 representation, and their architecture is almost identical to the MLP-mixer used. GRepsMLP-Mixer-2 uses a simple early fusion module, except for the late fusion present in GRepsMLP-Mixer-1. The GRepsMLP-mixer significantly outperforms the MLP-mixer. Both GRepsFNO-1 and GRepsFNO-2 (with early fusion) significantly outperform the traditional FNO on both the NS and NS-SYM datasets. Note that the NS dataset does not possess rot90 symmetry, yet GRepsFNO outperforms FNO, suggesting that using equivariant representations may be more expressive for tasks without any apparent symmetry, as noted in previous work. Furthermore, we find that the GRepsFNO model performs comparably to the more complex G-FNO models proposed recently. Therefore, we achieve the benefits of equivariance by using equivariant representations and making minimal changes to the architecture.

[0123] Figure 10 The results of solving the PDE are shown. According to an embodiment of the invention, the performance of FNO, GFNO, GRepsFNO-1, and GRepsFNO-2 is compared; for the PDE solving experiments, the average (standard deviation) of the relative mean squared error (expressed as a percentage) over three runs with different random initializations of network weights is used. GRepsFNO uses T1 notation, and its architecture remains almost identical to the FNO model. GRepsFNO-2 uses an early fusion layer in addition to the late fusion layer present in GRepsFNO-1. GRepsFNO outperforms the non-equivariant FNO and is comparable to the more complex G-FNO using group convolutions.

[0124] Modeling dynamic N-body systems using GNNs

[0125] Dataset and experimental setup: Using an N-body dynamics dataset, where the task is to perform experiments given their initial positions. ,speed and charge In the case of prediction A charged particle in The position after each step. 3000 trajectories were used for training, 2000 for validation, and 2000 for testing. Both the EGNN and GRepsGNN models have 4 layers and were trained for 10,000 epochs. Recall that GRepsGNN is achieved by replacing the scalar representation in MPNN with... It is designed to represent and appropriately combine various elements.

[0126] Results and observations: Figure 11 A performance comparison of GRepsGNN and EGNN for predicting N-body dynamics according to an embodiment of the present invention is shown. In this case, GRepsGNN provides comparable test loss and runtime complexity to EGNN in predicting N-body dynamics. GRepsGNN is constructed by replacing the representations in the GNN architecture with T1 representations and tensor mixing, while EGNN is a dedicated GNN designed for tasks with E(n) equivariance. As can be seen from the figure, although EGNN is an architecture specifically designed for this task, GRepsGNN performs comparably to EGNN on the N-body problem. Furthermore, its computational complexity is comparable to EGNN, therefore, it is computationally superior. Isovariance utilizes many specialized group equivariant architectures that employ spherical harmonics, making them far more efficient. Previously reported results are also shown, demonstrating that EGNN is significantly faster than other equivariant networks such as the SE(3) transformer and outperforms other equivariant networks in terms of test loss performance.

[0127] Second-order isotone

[0128] In some embodiments of the present invention, we conducted two sets of experiments to understand the impact of T2 representation on the performance of the isotropic image classifier.

[0129] Dataset and Experiment Setup

[0130] As an example of a network with T2 representation, a rot90 equivariant CNN with 3 convolutional layers followed by 5 fully connected layers is constructed as the first... Layer usage This indicates that the remaining layers use... express.

[0131] use The isotonic algorithm is extended to be used in the final layer. This indicates that, as a comparison baseline, a pre-trained ResNet18 was used as a non-equivariant model, and... and This indicates non-equivariant and equivariant fine-tuning. Experiments were performed on the rot90-CIFAR10 and Galaxy10 datasets.

[0132] The CNN used for training consists of three convolutional layers, each with a kernel size of 5 and output channel sizes of 6, 16, and 120 respectively. Following the convolutional layers are five fully connected layers, each consisting of 120-dimensional features. For training from scratch, we use a learning rate of 10. -3 Each model was trained for 10 epochs using stochastic gradient descent with a momentum of 0.9. Additionally, we used... 0.1 A stepLR learning rate scheduler with a step size of 7, which reduces the learning rate after each number of rounds equal to the step size. The T2 layer is computed by simply taking the outer product of the T1 features at the desired layer where the T2 representation is introduced, and then we simply use the same architecture as the T1 representation. Both the T1 and T2 regular representations remain equivariant.

[0133] The pre-trained ResNet18 was used with a learning rate of 10. -3 Five rounds of stochastic gradient descent fine-tuning with a momentum of 0.9 were performed. For isovariant fine-tuning using T2 representations, the operation was the same as for isovariant tuning: T1 features were first extracted from the pre-trained model, and then converted to T2 representations using a simple outer product. Once the desired features were obtained, they were passed through two fully connected layers (with a ReLU activation function in between) to obtain the final classification output.

[0134] Results and Observations

[0135] Figure 12A The performance of a rot90 equivariant CNN with 3 convolutional layers and 5 fully connected layers on rot90-CIFAR10 is shown. Here, T2 representations are introduced in layers i ∈ [1, ..., 8]. Using T2 representations in the final layer of the CNN outperforms both the non-equivariant CNN and the traditional equivariant representation using T1 representations; this demonstrates the effect of using T2 representations for rot90-CIFAR10 classification, easily outperforming both the non-equivariant CNN and the equivariant model using only T1 representations in the final layer of the CNN. Figure 12B The T1 and T2 features obtained from one channel of a pre-trained ResNet, corresponding to T1 isotone and T2 isotone, respectively, are shown. Figure 13A comparison of isotone results for the T1 representation and the T2 representation according to an embodiment of the invention is shown; the mean (standard deviation) of isotone test accuracy using a pre-trained ResNet utilizing the Rot90-CIFAR10 dataset and the Galaxy10 dataset is presented. The extended isotone using the T2 representation outperforms the earlier version using only the T1 representation.

[0136] from Figure 13 As can be seen, in the later layers of the same network, This indicates that it significantly outperforms both non-equivariant CNNs and equivariant CNNs. Both CNNs. On both the Rot90-CIFAR10 and Galaxy10, The equal-tone properties are all superior to the equal-tone properties, which confirms the characteristics in... This indicates the generation of a powerful equivariant network.

[0137] Example of a robust machine learning system

[0138] Figure 15 An example schematic diagram of a robust machine learning system configured with a processor, memory, and interface according to some implementation methods is shown. Specifically, Figure 15 This is a block diagram illustrating an example of the system. System 1500 includes a machine learning device 1501, at least one processor 1520, a memory (or a collection of memory groups) 1530, and a storage device 1540, wherein the machine learning device 1501 has an interface set and a data link 1505 configured to receive and transmit signals (digital datasets). The processor 1520, in conjunction with the memory 1530, executes computer-executable programs and algorithms (instructions) stored in the memory 1530 and storage device 1540. The interface set and data link 1505 include a human-machine interface (HMI) 1510 and a network interface controller 1550. The processor 1520 can execute computer-executable programs and algorithms in conjunction with the memory 1530, which uploads computational instructions, computer-executable software, and algorithms from the storage device 1540. The instructions, computer-executable programs, and algorithms stored in the storage device 1540 utilize a deep neural network in the form of GRepsNet 1541, an input dataset 1542, and an output dataset 1543.

[0139] Processor 1520 is configured to combine an interface and memory group 1530 to generate signals and dataset 1595 via network 1590 and output them to GRepsNet block 1541 to train machine learning device 1501 using dataset 1595.

[0140] System 1500 receives signals (data sets) from a set of sensors 1511 (which may be image sensors, microphones) via network 1590, interface set and data link 1505, and other interface modules such as pointing device / medium 2112.

[0141] The embodiments of the present invention described above can be implemented in any of a variety of ways. For example, the embodiments can be implemented using hardware, software, or a combination thereof. When implemented in software, the software code can execute on any suitable processor or set of processors, whether it is located in a single computer or distributed among multiple computers. Such a processor can be implemented as an integrated circuit having one or more processors in an integrated circuit assembly. However, the processor can be implemented using circuitry of any suitable format.

[0142] Furthermore, embodiments of the present invention can be implemented as methods, examples of which have been provided. Actions performed as part of a method can be ordered in any suitable manner. Therefore, embodiments in which actions are performed in a different order than those shown can be constructed, which may include the simultaneous execution of several actions, although these actions are shown as sequential actions in the illustrative embodiments.

[0143] Furthermore, the use of ordinal numbers such as "first" or "second" in the claims to modify the claimed elements does not imply any priority, order of precedence or sequence of actions of one claimed element relative to another claimed element, or the temporal order of the actions of the method. Rather, it serves merely as a label to distinguish one claimed element with a certain name from another element with the same name (except for the use of ordinal numbers), in order to differentiate the claimed elements.

[0144] While the invention has been described by way of example of preferred embodiments, it should be understood that various other adjustments and modifications can be made within the spirit and scope of the invention. Therefore, the appended claims are intended to cover all such variations and modifications that fall within the true spirit and scope of the invention.

Claims

1. A non-transitory computer-readable medium storing a set of instructions for data transformation, the set of instructions, when executed by one or more processors, causing the one or more processors to perform at least the following operations: The input dataset, represented by the input group representation, is provided to the input layer connected to the group representation network GRepsNet; The input dataset is transformed using GRepsNet, which is configured to generate an output dataset represented by an output ensemble representation from the input dataset, wherein... The GRepsNet is designed to be equivariant to a predetermined transform group; as well as The output dataset is output from the output layer connected to the GRepsNet.

2. The non-transitory computer-readable medium according to claim 1, wherein, When the input group is represented as a first-order or higher-order tensor, the GRepsNet is formed by one or more linear neural layers without pointwise nonlinearity or bias terms, wherein the linear neural layers linearly combine the input group representation with the output group representation to maintain equivariance with the predetermined transform group.

3. The non-transitory computer-readable medium according to claim 2, wherein, The GRepsNet is configured to create additional higher-order group representations.

4. The non-transitory computer-readable medium according to claim 2, wherein, The GRepsNet is formed by one or more invariant nonlinearities applied to the output of a linear neural layer, wherein the output of the nonlinearity is configured to interact with the output of the linear neural layer and maintain the equivariance to the predetermined transform group, wherein the invariant nonlinearity is represented by the Euclidean norm.

5. The non-transitory computer-readable medium according to claim 1, wherein, The GRepsNet consists of multiple layers, each of which is equivariant to the predetermined transform group.

6. The non-transitory computer-readable medium according to claim 1, wherein, When invariance is required at the output dataset, an additional pooling invariant layer is arranged at the output layer to pool the output data along the group dimension.

7. A data transformation system, the data transformation system comprising: The memory is configured as a storage cluster representation network GRepsNet and an instruction set for data transformation using the GRepsNet; as well as A processor, which is connected to the memory and configured to execute the instruction set, the instruction set Includes the following steps: The input dataset, represented by the input group representation, is provided to the input layer connected to the group representation network GRepsNet; The input dataset is transformed by using GRepsNet, which is configured to generate an output dataset represented by an output group representation from the input dataset, wherein the GRepsNet is designed to be equivariant to a predetermined transformation group; as well as The output dataset is output from the output layer connected to the GRepsNet.

8. The data transformation system according to claim 7, wherein, When the input group is represented as a first-order or higher-order tensor, the GRepsNet is formed by one or more linear neural layers without pointwise nonlinearity or bias terms, wherein the linear neural layers linearly combine the input group representation with the output group representation.

9. The data transformation system according to claim 8, wherein, The GRepsNet is configured to create additional higher-order group representations.

10. The data transformation system according to claim 8, wherein, The GRepsNet is formed by one or more invariant nonlinearities applied to the output of a linear neural layer, wherein the output of the nonlinearity is configured to interact with the output of the linear neural layer and maintain equivariance to a predetermined transform group, wherein the invariant nonlinearity is represented by a Euclidean norm.

11. The data transformation system according to claim 7, wherein, The GRepsNet consists of multiple layers, each of which is equivariant to the predetermined transform group.

12. The data transformation system according to claim 7, wherein, When invariance is required at the output dataset, an additional pooling invariant layer is arranged at the output layer to pool the output data along the group dimension.

Images