Image classification model training method based on Hilbert-Schmidt independence criterion

Abstract

The invention discloses an image classification model training method based on a Hilbert-Schmidt independence criterion, and belongs to the field of deep learning, the method proposes an implicit model design framework under an optimization perspective, innovatively combines an image classification task with multi-scale representation learning and an implicit dynamic optimization process deeply, and improves the image classification efficiency. A Hilbert independence criterion (HSIC) constraint training strategy is introduced into implicit model construction, and explicit optimization regulation and control of feature independence and scale complementarity in an image classification task are achieved. The framework not only breaks through the performance bottleneck of a traditional model in a complex classification scene, but also remarkably improves the accuracy and generalization ability of a classification task through the collaborative design of an optimization target and a model structure.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of deep learning, and more particularly relates to an image classification model training method based on a Hilbert-Schmidt independence criterion. BACKGROUND

[0002] An explicit model has problems of feature redundancy, insufficient generalization ability and high computational complexity when processing high-dimensional feature classification.

[0003] As an innovative design paradigm in the field of neural networks, an implicit neural network breaks through the inherent limitations of traditional architectures. As an implicit neural network module, a deep equilibrium model (DEQ) can model an infinite-layer neural network as a fixed point of a dynamic system. On the premise of realizing constant memory overhead, the DEQ achieves the goal of implicit deep modeling. The core design concept is to abstract the forward propagation process as a fixed point search, and to construct an efficient back propagation mechanism by means of the implicit function theorem, thereby laying an important foundation for the theoretical development and practical application of implicit neural networks.

[0004] In the field of vision, a multiscale deep equilibrium model (MDEQ) significantly enhances the hierarchical feature learning efficiency of visual tasks by virtue of its unique multiscale implicit equilibrium structure. In the fields of graph neural networks, flow models and diffusion models, the DEQ has also become a key driving force for technological innovation, providing a new idea for efficient processing of complex tasks. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the application provides an image classification model training method based on a Hilbert-Schmidt independence criterion, thereby solving the problems of feature redundancy, insufficient generalization ability and high computational complexity of an explicit model when processing high-dimensional feature classification.

[0006] To achieve the above-mentioned purpose, according to a first aspect of the application, an image classification model training method based on a Hilbert-Schmidt independence criterion is provided, comprising:

[0007] calculating an HSIC value HSIC(z (i)* , z (j)* ) between the equilibrium point z (i)* of the image feature of the sample image extracted by the i-th branch of the MDEQ model after iterative convergence of the fixed point equation and the equilibrium point z (j)*), the MDEQ model is trained under the constraint of the regular term, and the trained MDEQ model is used as an image classification model;

[0008] wherein, B is the number of sample images, Tr(·) represents the trace operation of a matrix, K (a) is the kernel matrix of the feature similarity matrix of z (a)* and z (a)* , K (a) is the element in the mth row and the nth column of K are respectively the mth row and the nth column elements, is obtained after tensor flattening and standardization processing of z (a)* , a = i, j, i = 1, 2, …, L, j = 1, 2, …, L, and L is the total number of branches of the MDEQ model.

[0009] According to the second aspect of the present application, an image classification method is provided, comprising: inputting an image to be classified into an image classification model obtained by the method according to the first aspect, to obtain a classification result.

[0010] According to the third aspect of the present application, an electronic device is provided, comprising: a computer readable storage medium and a processor;

[0011] The computer readable storage medium is used to store executable instructions;

[0012] The processor is used to read the executable instructions stored in the computer readable storage medium, and execute the method according to the first aspect or the second aspect.

[0013] According to the fourth aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for causing a processor to execute the method according to the first aspect or the second aspect.

[0014] According to the fifth aspect of the present application, a computer program product is provided, comprising computer programs or instructions, which are executed by a processor to implement the method according to the first aspect or the second aspect.

[0015] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0016] The application provides a method for training an image classification model based on a Hilbert-Schmidt independence criterion, uses a multi-scale deep equilibrium model as a training target of an optimization heuristic model architecture, introduces a constraint function optimized based on a Hilbert-Schmidt independence criterion, regards a forward propagation process of an implicit model as an iterative optimization problem of a target function, and realizes fine regulation of each potential factor in a latent space by introducing an explicit optimization target and constraint (HSIC constraint), thereby providing a theoretical basis for unified design of the implicit model in a multi-task scenario and effectively improving feature representation capability and optimization efficiency of the model in an image classification task. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 An optimization heuristic model architecture flowchart provided for an embodiment of the application;

[0018] Figure 2 A calculation flowchart of an HSIC constraint term provided for an embodiment of the application. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions and advantages of the application clearer, the application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0020] An embodiment of the application provides a method for training an image classification model based on a Hilbert-Schmidt independence criterion, comprising:

[0021] calculating an HSIC value HSIC(z (i)* , z (j) ) between the equilibrium point z (i) * of the image feature of the sample image extracted by the i th branch of the MDEQ model after iterative convergence of the fixed point equation and the equilibrium point z (j)* of the image feature of the sample image extracted by the j th branch, taking the HSIC value as a regularization term, training the MDEQ model under the constraint of the regularization term, and taking the trained MDEQ model as an image classification model;

[0022] wherein, B is the number of sample images, Tr(·) represents a trace operation of a matrix, K (a) is a kernel matrix of a feature similarity matrix of z (a)* and z (a)* , and the m th row n th column element K (a) in K are respectively The element in the m-th row and n-th column, For z (a)* After tensor flattening and standardization, we get a = i, j, i = 1, 2, ..., L, j = 1, 2, ..., L, where L is the total number of branches of the MDEQ model.

[0023] To eliminate the mean shift of the kernel matrix, preferably, when calculating HSIC(z) (i) , z (j) Before that, it also includes centering the kernel matrix to eliminate mean bias;

[0024] Specifically, a centralized matrix is ​​adopted. According to the formula For K respectively (i) K (j) Centralized processing is implemented; among them, For K (a) The matrix I obtained after centering B Let be an identity matrix of size B×B, and 1 be a vector of all 1s of dimension B.

[0025] Preferably, the loss function used in the training process in, For cross-entropy loss, W (i) Let y be the linear classifier head weight of the i-th branch of the MDEQ model, and y be the label. Here, θ is the parameter regularization term, and θ is the learnable parameter of the MDEQ model.

[0026] Preferably, according to the formula For z (a) Perform tensor flattening and standardization;

[0027] Where b, c, h, and w are the batch index, channel index, height index, and width index, respectively. Let μ be the feature value of the a-th branch at the b-th sample, c-th channel, and h×w-th spatial location. (a) and σ (a) denoted as the mean and standard deviation of the a-th branch feature, respectively.

[0028] The method provided by this invention utilizes a multiscale deep equalization model (MDEQ) as the training objective of an optimization-inspired model architecture, and introduces a constraint function optimized based on the Hilbert-Schmidt Independence Criterion (HSIC), which can improve the accuracy of the model for image classification tasks.

[0029] This invention employs an implicit model design framework based on optimization heuristics. The design process of this framework is as follows: Figure 1 As shown; and within this framework, a calculation based on HSIC constraint terms is introduced. The specific process for calculating this HSIC constraint term is as follows. Figure 2 As shown.

[0030] The implicit model design process based on HSIC optimization is as follows:

[0031] Step 1: Mathematical representation of the implicit model.

[0032] Suppose the input of the implicit neural network is x∈R d Then its equilibrium state z * ∈R n It can be defined by fixed-point equations:

[0033] z * =f(z) * ,x) (1)

[0034] in, A differentiable mapping is a nonlinear mapping consisting of network weights and activation functions. In practical solutions, a fixed-point iterative algorithm is typically used for its definition.

[0035] z t+1 =f(z) t ,x),t=0,1,...,T (2)

[0036] T is the number of iterations in solving the fixed-point equation, z t This represents the intermediate state during the t-th iteration.

[0037] As T→∞, the iteration converges to z. * .

[0038] Step Two: Optimize the Introduction of Objectives

[0039] By analyzing the iterative form of the DEQ model, it can be viewed as a gradient descent model.

[0040] Specifically, let the iterative form of the deep equilibrium network be the fixed point iteration, i.e., the form of equation (1). If there is a target function G(z, x) with certain good properties (derivable, convex, smooth, etc.), the z-f θ (z,x) is regarded as the gradient of the target function G(z, x) (z is the intermediate state of the fixed point iteration, f θ is the implicit neural network forward mapping, and θ refers to the set of learnable parameters of the network. Then, in the iterative process of DEQ (equation (2)), it is equivalent to the gradient descent optimization of the target function G(z, x), which can be expressed as:

[0041]

[0042] is the gradient of the target function.

[0043] Then, for the process of finding the fixed point in equation (1), it is equivalent to the gradient descent process of G(z, x), as shown in equation (4):

[0044]

[0045] z t represents the equilibrium point of the t-th iteration.

[0046] The process of solving equation (4) corresponds exactly to the iterative process of the deep equilibrium model z * -f(z * ,x) = 0, and the conditions for obtaining the extreme point are the same, so the original DEQ iteration process for the fixed point is converted into the optimization problem shown in equation (5):

[0047]

[0048] Thus, the forward process of the deep equilibrium model is modeled as an optimization problem, providing a theoretical possibility for solving the black box defect of traditional methods. By constructing the implicit iteration z t+1 =f(z t ,x) and the optimization of the target function mathematical equivalence, the traditional implicit network's trial-and-error paradigm of relying on empirical structure design is changed, and the design focus is shifted from parameterization of the complex mapping function f to adjustment of the mathematical construction of the target function G(z, x).

[0049] Therefore, the optimization function based on HSIC is added to the target function (including the image classification cross-entropy loss and the parameter regularization term) in the training process of the MDEQ model. The image features of the sample image extracted by the i-th branch of the MDEQ model are subjected to the equilibrium point z (i)*The equilibrium point z of the image features of the sample image extracted from the j-th branch after iterative convergence of the fixed-point equation. (j)* HSIC values ​​between HSIC(z) (i)* , z (j)* The MDEQ model is trained under the constraints of the regularization term.

[0050] Step 3: Calculation of HSIC value

[0051] 1. Feature Tensor Reconstruction: To eliminate the interference of spatial dimension on independence measurement, tensor flattening and standardization are performed on each branch feature of MDEQ:

[0052]

[0053] Among them, z (i)* This represents the equilibrium point after the image features extracted from the i-th branch of MDEQ, obtained by inputting sample images from the training set into the fixed-point equation, converge through iterative convergence. Its shape is... C i H i W i These represent the number of channels, height, and width of the i-th branch, respectively. Indicates z (i) The feature tensor after tensor flattening and standardization. Let D represent the feature value of the b-th sample at the c-th channel, h-th row, and w-th column; i μ represents the feature dimension of a single sample after the i-th branch is flattened. (i) and σ (i) These are the mean and standard deviation of the i-th branch feature, calculated along the channel dimension.

[0054] 2. Kernel Function Selection and Calculation: The feature similarity matrix is ​​constructed using the Gaussian Radial Basis Function (RBF). For branch i, the kernel matrix K is the feature similarity matrix of its own feature similarity matrix (i.e., the feature similarity matrix between branch i and branch i). (i) The element in the m-th row and n-th column satisfy:

[0055]

[0056] In the formula, γ i By adaptively determining the kernel function using a median-inspired approach, suboptimal solutions caused by manual parameter tuning are avoided, thereby improving the robustness of the kernel function to the data distribution.

[0057] 3. Centralization and Decorrelation: Introducing a Centralization Matrix The kernel matrix is ​​centered to eliminate mean bias:

[0058]

[0059] This step is a preferred step, considering the nuclear matrix K (i) Essentially reflects the similarity between the samples, but if the data has a non-zero mean, the kernel matrix will be affected by the global offset, so that the subsequent independence measure is not accurate enough. The centering operation is equivalent to removing the mean of each sample after mapping the feature, ensuring that subsequent calculations only reflect "relative" differences and are not disturbed by the overall offset.

[0060] 4. Independence measure calculation: the HSIC value between branches i and j is calculated by the trace operation of the kernel matrix:

[0061]

[0062] Where B represents the number of samples used to calculate the kernel matrix, that is, the batch size.

[0063] And The higher the "alignment" degree (i.e., the sample similarity pattern is consistent), the greater the trace product, which corresponds to a larger HSIC value, indicating that the two sets of features have a strong statistical dependence;

[0064] On the contrary, if the two sets of features are independent of each other, then the kernel matrix after centering is almost "irrelevant", the trace product is close to zero, and the HSIC value is also close to zero.

[0065] Through the above centering and trace operation, HSIC can efficiently quantify the nonlinear dependence between two feature sets without assuming the form of the distribution, which is exactly the point that makes it an ideal constraint term for measuring the independence of different branch features in the multi-scale implicit model (MDEQ). When applied to image classification, it can improve the accuracy of image classification.

[0066] Step four, training of the MDEQ model

[0067] Take HSIC(z (i)* , z (j)* ) as a regularization term, and train the MDEQ model under the constraint of the regularization term.

[0068] In order to improve the representation ability of the MDEQ model for the original image input, preferably, before the sample image is input to the image feature of the sample image extracted by the a-th branch, it further comprises:

[0069] Input the sample image to the feature extractor g(·) for feature extraction;

[0070] Input the feature extracted by the feature extractor g(·) to the image feature of the sample image extracted by the a-th branch.

[0071] The feature extractor is used to:

[0072] The original input image is subjected to preliminary convolution transformation, down-sampling and non-linear activation, etc. An initial feature representation with uniform size and semantic initial information is output as the initial value for solving the fixed point of each branch.

[0073] The training process is shown in Table 1:

[0074] Table 1 MDEQ training process based on HSIC constraint optimization

[0075]

[0076]

[0077] In Table 1, z 0(i) is the initial value of z (i)* , W0 is the initial value of the learnable parameters of g(·), T i (·) is a non-linear feature mapping function on the i-th branch, is the image classification cross-entropy loss, is a parameter regularization term used to constrain the model complexity to prevent overfitting loss.

[0078] To verify the superiority of the design framework of the implicit model providing optimization inspiration of the present application, experiments are performed on the MDEQ based on HSIC optimization. Different weights of the constraint term 0, 0.01, 0.1, 0.5 and 1.0 are selected for the experiments. The classification accuracy of the experiments is the result of five times of training and mean value. The classification results of the CIFAR-10 data set are shown in Table 2:

[0079] Table 2 MDEQ classification accuracy with different constraint strengths

[0080]

[0081] The embodiment of the present application provides an image classification method, comprising: inputting an image to be classified into an image classification model obtained by using the training method of any one of the above embodiments to obtain a classification result.

[0082] The embodiment of the present application provides an electronic device, comprising: a computer readable storage medium and a processor.

[0083] The computer readable storage medium is used to store executable instructions.

[0084] The processor is used to read the executable instructions stored in the computer readable storage medium, and execute the training method of any one of the above embodiments or the classification method of any one of the above embodiments.

[0085] An embodiment of the present application provides a computer readable storage medium, which stores computer instructions, the computer instructions being used for causing a processor to execute the training method according to any one of the preceding embodiments or the classification method according to any one of the preceding embodiments.

[0086] An embodiment of the present application provides a computer program product, which comprises computer programs or instructions, the computer programs or instructions being executed by a processor to implement the training method according to any one of the preceding embodiments or the classification method according to any one of the preceding embodiments.

[0087] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A training method for an image classification model based on the Hilbert-Schmidt independence criterion, characterized in that, include: The equilibrium point z of the image features of the sample image extracted by the i-th branch of the MDEQ model after iterative convergence using the fixed-point equation is calculated. (i)* The equilibrium point z of the image features of the sample image extracted from the j-th branch after iterative convergence of the fixed-point equation. (j)* HSIC values ​​between HSIC(z) (i)* , z (j)* ), which is used as a regularization term, and the MDEQ model is trained under the constraint of the regularization term. The trained MDEQ model is then used as an image classification model. in, B is the number of sample images, Tr(·) represents the trace operation of the matrix, and K (a) For z (a)* With z (a)* The kernel matrix of the feature similarity matrix, K (a) The element in the m-th row and n-th column They are respectively The element in the m-th row and n-th column, For z (a)* After tensor flattening and standardization, we get a = i, j, i = 1, 2, ..., L, j = 1, 2, ..., L, where L is the total number of branches of the MDEQ model.

2. The method as described in claim 1, characterized in that, In calculating HSIC(z) (i)* , z (j)* Before that, it also includes: Using a centralized matrix According to the formula For K respectively (i) K (j) Centralized processing is implemented; among them, For K (a) The matrix I obtained after centering B Let be an identity matrix of size B×B, and 1 be a vector of all 1s of dimension B.

3. The method as described in claim 1, characterized in that, The loss function used in the training process in, For cross-entropy loss, W (i) Let y be the linear classifier head weight of the i-th branch of the MDEQ model, and y be the label. Here, θ is the parameter regularization term, and θ is the learnable parameter of the MDEQ model.

4. The method as described in claim 1, characterized in that, According to the formula For z (a) Perform tensor flattening and standardization; Where b, c, h, and w represent the batch dimension, channel dimension, height, and width, respectively. For the image features extracted from the b-th sample image of the a-th branch, the c-th channel, h-th row, and w-th column are μ. (a) and σ (a) denoted as the mean and standard deviation of the image features extracted from the a-th branch, respectively.

5. The method as described in claim 1, characterized in that, Before inputting the sample image into the image features extracted from the sample image in the a-th branch, the following steps are also included: The sample image is input into the feature extractor for feature extraction. The features extracted by the feature extractor are input into the image features of the sample image extracted by the a-th branch.

6. An image classification method, characterized in that, include: The image to be classified is input into the image classification model obtained by the method described in any one of claims 1-5, and the classification result is obtained.

7. An electronic device, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the training method as described in any one of claims 1-5 or the image classification method as described in claim 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the training method as described in any one of claims 1-5 or the image classification method as described in claim 6.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the training method as described in any one of claims 1-5 or the image classification method as described in claim 6.

Images