Training method and device of modular network autoencoder based on isometric constraint

Abstract

This invention discloses a training method and apparatus for a modular network autoencoder based on equivariant constraints, belonging to the field of artificial intelligence technology. The method includes: acquiring training data, wherein the training data includes image I and image I′ obtained by transforming image I; and training a convolutional network autoencoder using equivariant constraints based on the training data to obtain a trained autoencoder. This invention enhances the correlation within autoencoder modules and the independence between modules by constructing equivariant constraints and using these constraints to train the modular network autoencoder on training data of homologous image pairs, thus promoting the specialization of network module functions. The functionally differentiated modular network can be applied to downstream tasks such as classification, detection, segmentation, and associative generation, providing a better representation space for downstream tasks and improving their robustness and generalization.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a training method and apparatus for a modular network autoencoder based on equivariant constraints. Background Technology

[0002] Currently, artificial neural networks have experienced rapid development and widespread application. Modular design of artificial neural networks offers numerous advantages, such as reducing the number of network parameters, thereby decreasing model storage space and computational cost; making networks more easily parallelized, thus improving computational speed; more effectively utilizing computing resources, such as GPU memory and bandwidth; making network structures clearer and easier to understand the model's internal workings; allowing different models to share the same modules, thus improving generalization ability; and combining different modules into new models to achieve more complex network structures and functions. In summary, modular network design can effectively improve model efficiency, interpretability, and generalization ability, and can adapt to various tasks, making it an important method for building complex, efficient, and interpretable deep learning models.

[0003] Existing network modularization methods mainly fall into two categories:

[0004] Modular structure and manually defined functions: This type of method achieves functional differences between modules by manually designing different modules and training them with different modalities of data or different tasks. For example, a module trained on image data may be better at recognizing visual features, while a module trained on text data may be better at understanding semantic information. The disadvantage of this approach is that it relies heavily on manual design, limiting the scalability of the network's modularity. When encountering new tasks or data types, modules need to be redesigned, resulting in a large workload and a lack of flexibility.

[0005] Modular structure and automatic feature learning: These methods divide the network structure into different modules, and then allow the network to automatically learn the functional specializations of each module during training. For example, grouped convolutions divide the convolutional kernels into different groups, each responsible for extracting a specific type of feature. Expert ensembles (MoE) combine multiple expert networks for ensemble learning. Capsule networks connect different types of capsules through dynamic routing mechanisms to achieve a higher level of feature extraction. While these methods improve network efficiency and reduce the number of parameters, they struggle to generate stable and reliable functional specializations.

[0006] It is evident that existing modular network methods still face some bottlenecks in terms of functional specialization. Therefore, achieving more efficient, stable, and controllable functional specialization is an important direction for future research on modular networks. Summary of the Invention

[0007] In order to solve the problems existing in the prior art, the present invention provides the following technical solution.

[0008] The first aspect of this invention provides a training method for a modular network autoencoder based on equivariant constraints, comprising:

[0009] Acquire training data, which includes image I and image I′ obtained by transforming image I;

[0010] Based on the training data, the convolutional network autoencoder is trained using equivariant constraints to obtain a trained autoencoder; wherein, the equivariant constraint L... EC The following formula is used for calculation:

[0011]

[0012] L equ =∑ i ||f′ (i) -M (i) (δ)f (i) || 2 , i∈{0,1,...,k-1},

[0013]

[0014] L EC =L recon +λ1L equ +λ2L sym ,

[0015] f = I * W,

[0016] f′=I′*W,

[0017] Where, I′=L δ (I), where Lδ is a certain transformation with transformation parameter δ; W is the convolution kernel of the autoencoder of the convolutional network, and W = [W (0) W (1) , ..., W (k-1) ], that is, the convolution kernel is divided into k modules; f is the feature image obtained after convolving image I, and f = [f (0) f (1) , ..., f (k-1) f′ is the feature image obtained after convolving image I′, f′=[f′ (0) f′ (1) ,...,f′ (k-1) ]; * indicates a convolution operation. Indicates the deconvolution operation; f (i) f′ is the feature map obtained after convolving image I through the i-th module of the convolution kernel;(i) M is the feature map obtained after convolving image I′ through the i-th module of the convolution kernel; (i) (δ) is the prediction matrix of the i-th module with parameter δ, which is obtained by random initialization and optimization through model training; L recon To rebuild the losses, L equ For equal loss, L sym For symmetrical loss, L EC For equivariant constraints; λ1 and λ2 are the weighting coefficients of equivariant loss and symmetric loss, respectively; l represents the number of dimensions of each module; This represents the feature map of the i-th module where only the m-th dimension is 1 and the other dimensions are 0. This represents the feature map of the i-th module where only the n-th dimension is 1 and the other dimensions are 0. From arrive The optimal transformation parameters; Let the parameter be the prediction matrix M. (i) The value of (δ) in the m-th row and n-th column; The prediction matrix M with parameter δ′ (i) The value of (δ′) in the m-th row and n-th column; T is a hyperparameter greater than 0; For the transformation parameters are The prediction matrix is ​​δ′, which is the integral variable used to distinguish the transformation parameter δ under different integral terms.

[0018] Preferably, the transformation includes a translation transformation or a translation-rotation transformation.

[0019] The second aspect of the present invention provides a modular network autoencoder based on equivariant constraints, which is trained using the method described in the first aspect.

[0020] The third aspect of this invention provides the application of a modular network autoencoder based on equivariant constraints as described in the second aspect in classification, detection, segmentation, and associative generation.

[0021] A fourth aspect of the present invention provides a training apparatus for a modular network autoencoder based on equivariant constraints, comprising:

[0022] The training data acquisition module is used to acquire training data, which includes image I and image I′ obtained by transforming image I;

[0023] The training module is used to train the convolutional network autoencoder based on the training data using equivariant constraints, thereby obtaining a trained autoencoder; wherein, the equivariant constraint L... EC The following formula is used for calculation:

[0024]

[0025] L equ =∑ i ||f′ (i) -M (i) (δ)f (i) || 2 , i∈{0,1,...,k-1},

[0026]

[0027] L EC =L recon +λ1L equ +λ2L sym ,

[0028] f = I * W,

[0029] f′=I′*W,

[0030] Where, I′=L δ (I), where Lδ is a certain transformation with transformation parameter δ; W is the convolution kernel of the autoencoder of the convolutional network, and W = [W (0) W (1) , ..., W (k-1) ], that is, the convolution kernel is divided into k modules; f is the feature image obtained after convolving image I, and f = [f (0) f (1) , ..., f (k-1) f′ is the feature image obtained after convolving image I′, f′=[f′ (0) f′ (1) ,...,f′ (k-1) ]; * indicates a convolution operation. Indicates the deconvolution operation; f (i) f′ is the feature map obtained after convolving image I through the i-th module of the convolution kernel; (i) M is the feature map obtained after convolving image I′ through the i-th module of the convolution kernel; (i) (δ) is the prediction matrix of the i-th module with parameter δ, which is obtained by random initialization and optimization through model training; L recon To rebuild the losses, L equ For equal loss, L sym For symmetrical loss, L EC For equivariant constraints; λ1 and λ2 are the weighting coefficients of equivariant loss and symmetric loss, respectively; l represents the number of dimensions of each module; This represents the feature map of the i-th module where only the m-th dimension is 1 and the other dimensions are 0. This represents the feature map of the i-th module where only the n-th dimension is 1 and the other dimensions are 0. From arrive The optimal transformation parameters; Let the parameter be the prediction matrix M. (i) The value of (δ) in the m-th row and n-th column; The prediction matrix M with parameter δ′ (i) The value of (δ′) in the m-th row and n-th column; T is a hyperparameter greater than 0; For the transformation parameters are The prediction matrix is ​​δ′, which is the integral variable used to distinguish the transformation parameter δ under different integral terms.

[0031] Preferably, the transformation includes a translation transformation or a translation-rotation transformation.

[0032] The fifth aspect of the present invention provides a memory that stores a plurality of instructions for implementing the training method of the modular network autoencoder based on equivariant constraints as described in the first aspect.

[0033] A sixth aspect of the present invention provides an electronic device including a processor and a memory connected to the processor, the memory storing a plurality of instructions which can be loaded and executed by the processor to enable the processor to perform a training method for a modular network autoencoder based on equivariant constraints as described in the first aspect.

[0034] The beneficial effects of this invention are as follows: The training method and apparatus for a modular network autoencoder based on equivariant constraints provided by this invention enhance the correlation within the autoencoder modules and the independence between modules by constructing equivariant constraints and using these constraints to train the modular network autoencoder on training data of homologous image pairs, thereby promoting the specialization of network module functions. The functionally differentiated modular network, when applied to downstream tasks such as classification, detection, segmentation, and associative generation, can provide a better representation space for downstream tasks and improve their robustness and generalization. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the training method for the modular network autoencoder based on equivariant constraints described in this invention:

[0036] Figure 2 This is a schematic diagram of the training process of a modular network autoencoder.

[0037] Figure 3 This is a schematic diagram illustrating the functional differentiation effect under modular and equivariant constraints.

[0038] Figure 4 This is a schematic diagram of the functional structure of the training device for the modular network autoencoder based on equivariant constraints described in this invention. Detailed Implementation

[0039] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0040] The method provided by this invention can be implemented in a terminal environment that may include one or more of the following components: a processor, a memory, and a display screen. The memory stores at least one instruction, which is loaded and executed by the processor to implement the method described in the following embodiments.

[0041] A processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the terminal, and performs various functions and processes data by running or executing instructions, programs, code sets or instruction sets stored in memory, and by calling data stored in memory.

[0042] Memory can include random access memory (RAM) or read-only memory (ROM). Memory can be used to store instructions, programs, code, code sets, or instructions.

[0043] The display screen is used to show the user interface of each application.

[0044] In addition, those skilled in the art will understand that the structure of the terminal described above does not constitute a limitation on the terminal. The terminal may include more or fewer components, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, power supplies, and other components, which will not be described in detail here.

[0045] Example 1

[0046] like Figure 1 As shown, this embodiment of the invention provides a training method for a modular network autoencoder based on equivariant constraints, comprising:

[0047] S101, Obtain training data, the training data including image I and image I′ obtained by transforming image I;

[0048] S102, Based on the training data, the convolutional network autoencoder is trained using equivariant constraints to obtain a trained autoencoder; wherein, the equivariant constraint L EC The following formula is used for calculation:

[0049]

[0050] L equ =∑ i ||f′ (i) -M(i) (δ)f (i) || 2 , i∈{0,1,...,k-1},

[0051]

[0052] L EC =L recon +λ1L equ +λ2L sym ,

[0053] f = I * W,

[0054] f′=I′*W,

[0055] Where, I′=L δ (I), where Lδ is a certain transformation with transformation parameter δ; W is the convolution kernel of the autoencoder of the convolutional network, and W = [W (0) W (1) , ..., W (k-1) ], that is, the convolution kernel is divided into k modules; f is the feature image obtained after convolving image I, and f = [f (0) f (1) , ..., f (k-1) f′ is the feature image obtained after convolving image I′, f′=[f′ (0) f′ (1) , ..., f, (k-1) ]; * indicates a convolution operation. Indicates the deconvolution operation; f (i) f′ is the feature map obtained after convolving image I through the i-th module of the convolution kernel; (i) M is the feature map obtained after convolving image I′ through the i-th module of the convolution kernel; (i) (δ) is the prediction matrix of the i-th module with parameter δ, which is obtained by random initialization and optimization through model training; L recon To rebuild the losses, L equ For equal loss, L sym For symmetrical loss, L EC For equivariant constraints; λ1 and λ2 are the weighting coefficients of equivariant loss and symmetric loss, respectively; l represents the number of dimensions of each module; This represents the feature map of the i-th module where only the m-th dimension is 1 and the other dimensions are 0. This represents the feature map of the i-th module where only the n-th dimension is 1 and the other dimensions are 0. From arrive The optimal transformation parameters; Let the parameter be the prediction matrix M. (i)The value of (δ) in the m-th row and n-th column; The prediction matrix M with parameter δ′ (i) The value of (δ′) in the m-th row and n-th column; T is a hyperparameter greater than 0; For the transformation parameters are The prediction matrix is ​​given by δ′, which is the integration variable used to distinguish the transformation parameter δ under different integration terms. It should be noted that... The subscript "m" in the middle The subscript "m" in the matrix, although representing the dimension of the feature map and the row number of the matrix respectively, is used because the feature map and the matrix correspond. The subscript "m" in the middle The subscript "m" in the original text corresponds to the same subscript. In this invention, to illustrate the correspondence between the two, the same letter "m" is used. Similarly, The subscript "n" in the middle The subscript "n" in the matrix, although representing the dimension of the feature map and the column number of the matrix respectively, is used because the feature map and the matrix correspond. The subscript "n" in the middle The subscript "n" in the text is corresponding. In order to illustrate the correspondence between the two, the same letter "n" is used to represent them in this invention.

[0056] In the above training method, the training process of the modular network autoencoder can be as follows: Figure 2 As shown.

[0057] In a preferred embodiment of the present invention, step S101 may include a translation transformation or a translation-rotation transformation, thereby obtaining the source data pair image I and image I′.

[0058] In practical applications, this invention first constructs an autoencoder composed of convolutions, dividing the convolution kernels into k modules. During training, the untransformed image I and the transformed image I′ are simultaneously input into the constructed network, and convolution operations are performed on each of the k modules to obtain the feature image corresponding to each module. Furthermore, equivariant constraints can be constructed and used to train and optimize the convolution kernel W and prediction matrix M of the constructed convolutional network autoencoder. After training, all parameters are fixed, resulting in a functionally differentiated modular network. This modular network can be used for downstream tasks such as recognition and associative generation.

[0059] This invention proposes a training method for modular autoencoders by constructing modular equivariant constraints. Equivariant constraints enhance the correlation within modules and the independence between modules, thereby enabling modules to spontaneously differentiate their functions. Figure 3This demonstrates the effect of functional differentiation under modular and equivariant constraints. Figure 3 In Figures a, b, and c, each row represents a module, and each square visualizes a learned convolutional kernel. The figures show clear functional differentiation between modules, such as differentiation in spatial frequencies, brightness, and color. Figure d visualizes the reconstruction of features extracted from each module in Figure c, providing a more intuitive view of the functional differentiation between modules in terms of color, brightness, and edges at different spatial frequencies. This modular feature differentiation is beneficial for providing a better representation space for downstream tasks, such as classification, detection, and associative generation. It also improves the robustness and generalization of downstream tasks. For example, when the style of the input image for a recognition task changes significantly, conventional classification networks may fail to recognize the image, exhibiting poor generalization. However, with the modular representation in this invention, even if the image style changes, it only affects the representation of a few modules, while having little impact on others. Therefore, through ensemble learning, it can still achieve good recognition, thereby improving the network's generalization ability.

[0060] Example 2

[0061] like Figure 4 As shown, another aspect of the present invention also includes a functional module architecture that is completely consistent with the aforementioned method flow. That is, the embodiments of the present invention also provide a training device for a modular network autoencoder based on equivariant constraints, including:

[0062] The training data acquisition module 301 is used to acquire training data, which includes image I and image I′ obtained by transforming image I.

[0063] Training module 302 is used to train the convolutional network autoencoder based on the training data using equivariant constraints, to obtain a trained autoencoder; wherein, the equivariant constraint L EC The following formula is used for calculation:

[0064]

[0065] L equ =∑ i ||f′ (i) -M (i) (δ)f (i) || 2 , i∈{0,1,...,k-1},

[0066]

[0067] L EC =L recon +λ1L equ +λ2Lsym ,

[0068] f = I * W,

[0069] f′=I′*W,

[0070] Where, I′=L δ (I), where Lδ is a certain transformation with transformation parameter δ; W is the convolution kernel of the autoencoder of the convolutional network, and W = [W (0) W (1) , ..., W (k-1) ], that is, the convolution kernel is divided into k modules; f is the feature image obtained after convolving image I, and f = [f (0) f (1) , ..., f (k-1) f′ is the feature image obtained after convolving image I′, f′=[f′ (0) f′ (1) ,...,f′ (k-1) ]; * indicates a convolution operation. Indicates the deconvolution operation; f (i) f′ is the feature map obtained after convolving image I through the i-th module of the convolution kernel; (i) M is the feature map obtained after convolving image I′ through the i-th module of the convolution kernel; (i) (δ) is the prediction matrix of the i-th module with parameter δ, which is obtained by random initialization and optimization through model training; L recon To rebuild the losses, L equ For equal loss, L sym For symmetrical loss, L EC For equivariant constraints; λ1 and λ2 are the weighting coefficients of equivariant loss and symmetric loss, respectively; l represents the number of dimensions of each module; This represents the feature map of the i-th module where only the m-th dimension is 1 and the other dimensions are 0. This represents the feature map of the i-th module where only the n-th dimension is 1 and the other dimensions are 0. From arrive The optimal transformation parameters; Let the parameter be the prediction matrix M. (i) The value of (δ) in the m-th row and n-th column; The prediction matrix M with parameter δ′ (i) The value of (δ′) in the m-th row and n-th column; T is a hyperparameter greater than 0; For the transformation parameters are The prediction matrix is ​​δ′, which is the integral variable used to distinguish the transformation parameter δ under different integral terms.

[0071] The device can be implemented using the training method of the modular network autoencoder based on equivariant constraints provided in Embodiment 1 above. For the specific implementation method, please refer to the description in Embodiment 1, which will not be repeated here.

[0072] The present invention also provides a memory that stores multiple instructions for implementing the training method of the modular network autoencoder based on equivariant constraints as described in Embodiment 1.

[0073] The present invention also provides an electronic device, including a processor and a memory connected to the processor, the memory storing a plurality of instructions which can be loaded and executed by the processor to enable the processor to perform a training method for a modular network autoencoder based on equivariant constraints as described in Embodiment 1.

[0074] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A training method for a modular network autoencoder based on equivariant constraints, characterized in that, include: Acquire training data, which includes image I and image I′ obtained by transforming image I; Based on the training data, the convolutional network autoencoder is trained using equivariant constraints to obtain a trained autoencoder; wherein, the equivariant constraint L... EC The following formula is used for calculation: L equ =∑ i ||f′ (i) -M (i) (δ)f (i) || 2 ,i∈{0,1,...,k-1}, L EC =L recon +λ1L equ +λ2L sym , f = I * W, f′=I′*W, Where, I′=L δ (I), where Lδ is a certain transformation with transformation parameter δ; W is the convolution kernel of the autoencoder of the convolutional network, and W = [W (0) W (1) , ..., W (k-1) ], that is, the convolution kernel is divided into k modules; f is the feature image obtained after convolving image I, and f = [f (0) f (1) , ..., f (k-1) f′ is the feature image obtained after convolving image I′, f′=[f′ (0) f′ (1) ,...,f′ (k-1) ]; * indicates a convolution operation. Indicates the deconvolution operation; f (i) f′ is the feature map obtained after convolving image I through the i-th module of the convolution kernel; (i) M is the feature map obtained after convolving image I′ through the i-th module of the convolution kernel; (i) (δ) is the prediction matrix of the i-th module with parameter δ, which is obtained by random initialization and optimization through model training; L recon To rebuild the losses, L equ For equal loss, L sym For symmetrical loss, L EC For equivariant constraints; λ1 and λ2 are the weighting coefficients of equivariant loss and symmetric loss, respectively; l represents the number of dimensions of each module; This represents the feature map of the i-th module where only the m-th dimension is 1 and the other dimensions are 0. This represents the feature map of the i-th module where only the n-th dimension is 1 and the other dimensions are 0. From arrive The optimal transformation parameters; Let the parameter be the prediction matrix M. (i) The value of (δ) in the m-th row and n-th column; The prediction matrix M with parameter δ′ (i) The value of (δ′) in the m-th row and n-th column; T is a hyperparameter greater than 0; For the transformation parameters are The prediction matrix is ​​δ′, which is the integral variable used to distinguish the transformation parameter δ under different integral terms.

2. The training method for a modular network autoencoder based on equivariant constraints as described in claim 1, characterized in that, The transformation includes translation transformation or translation-rotation transformation.

3. A modular network autoencoder based on equivariant constraints, characterized in that, It is trained using the method described in any one of claims 1-2.

4. The application of the modular network autoencoder based on equivariant constraints as described in claim 3 in classification, detection, segmentation, and associative generation.

5. A training device for a modular network autoencoder based on equivariant constraints, characterized in that, include: The training data acquisition module is used to acquire training data, which includes image I and image I′ obtained by transforming image I; The training module is used to train the convolutional network autoencoder based on the training data using equivariant constraints, thereby obtaining a trained autoencoder; wherein, the equivariant constraint L... EC The following formula is used for calculation: L equ =∑ i ||f′ (i) -M (i) (δ)f (i) || 2 ,i∈{0,1,...,k-1}, L EC =L recon +λ1L equ +λ2L sym , f = I * W, f′=I′*W, Where, I′=L δ (I), where Lδ is a certain transformation with transformation parameter δ; W is the convolution kernel of the autoencoder of the convolutional network, and W = [W (0) W (1) , ..., W (k-1) ], that is, the convolution kernel is divided into k modules; f is the feature image obtained after convolving image I, and f = [f (0) f (1) , ..., f (k-1) f′ is the feature image obtained after convolving image I′, f′=[f′ (0) f′ (1) ,...,f′ (k-1) ]; * indicates a convolution operation. Indicates the deconvolution operation; f (i) f' is the feature map obtained after convolving image I through the i-th module of the convolution kernel; (i) M is the feature map obtained after convolving image I′ through the i-th module of the convolution kernel; (i) (δ) is the prediction matrix of the i-th module with parameter δ, which is obtained by random initialization and optimization through model training; L recon To rebuild the losses, L equ For equal loss, L sym For symmetrical loss, L EC The constraint is equivariant; λ1 and λ2 are the weight coefficients of the equivariant loss and symmetric loss, respectively; l represents the number of dimensions of each module. This represents the feature map of the i-th module where only the m-th dimension is 1 and the other dimensions are 0. This represents the feature map of the i-th module where only the n-th dimension is 1 and the other dimensions are 0. From arrive The optimal transformation parameters; Let the parameter be the prediction matrix M. (i) The value of (δ) in the m-th row and n-th column; The prediction matrix M with parameter δ′ (i) The value of (δ′) in the m-th row and n-th column; T is a hyperparameter greater than 0; For the transformation parameters are The prediction matrix is ​​δ′, which is the integral variable used to distinguish the transformation parameter δ under different integral terms.

6. The training device for a modular network autoencoder based on equivariant constraints as described in claim 5, characterized in that, The transformation includes translation transformation or translation-rotation transformation.

7. A memory, characterized in that, The system stores multiple instructions for implementing the training method of the modular network autoencoder based on equivariant constraints as described in any one of claims 1-2.

8. An electronic device, characterized in that, The system includes a processor and a memory connected to the processor, the memory storing multiple instructions that can be loaded and executed by the processor to enable the processor to perform the training method for a modular network autoencoder based on equivariant constraints as described in any one of claims 1-2.

Images