Image recognition model training method and device based on feature recombination, image recognition method, equipment, medium and product

Abstract

The present disclosure relates to the technical field of deep learning and computer vision, and provides a training method and device of an image recognition model based on feature reorganization, an image recognition method, equipment, a medium and a product. The training method comprises: inputting training sample images into a student model and a teacher model respectively to obtain first intermediate features and second intermediate features; determining a first distribution of overall features and detail features in the first intermediate features and a second distribution of overall features and detail features in the second intermediate features; reorganizing the first intermediate features by adjusting the first distribution towards the second distribution to obtain reorganized features; obtaining a reorganized feature loss by comparing the second intermediate features and the reorganized features; and training the student model based on the reorganized feature loss to obtain a trained student model. The present disclosure can solve the problem of difficulty in achieving more accurate image recognition, improve the recognition effect of overall features and detail features of images, and more accurately recognize images.

Description

Technical Field

[0001] This disclosure relates to the fields of deep learning and computer vision technology, and in particular to a training method and apparatus for an image recognition model based on feature recombination, as well as an image recognition method, device, medium, and product. Background Technology

[0002] Knowledge distillation (KD) can transfer knowledge from a high-capacity model (the teacher model) to a lightweight model (the student model) using a teacher-student paradigm, thereby improving the deployment capability of the lightweight model on resource-constrained devices.

[0003] In knowledge distillation training, due to the differences between the teacher model and the student model in terms of receptive field modeling methods, feature calculation paradigms, and model layer structures, there may be significant differences in the feature distribution of coarse-grained and fine-grained representations in the feature space. In some cases, this difference in feature distribution between the teacher model and the student model can be disregarded.

[0004] However, in image recognition scenarios, the representation dimensions of image features are often more complex. It is necessary to consider both coarse-grained features of the overall image (such as the image's structural outline, main color, and main brightness / darkness) and fine-grained features of the image details (such as the image's texture, lines, edges, color changes, and local brightness / darkness changes). Therefore, the difference in feature distribution between the teacher model and the student model at different granularities may cause the student model to be unable to achieve the image recognition capabilities of the teacher model. For example, it may not perform well in recognizing either coarse-grained or fine-grained features, making it difficult to achieve more accurate image recognition. Summary of the Invention

[0005] This disclosure provides a training method and apparatus for an image recognition model based on feature recombination, as well as an image recognition method, device, medium, and product, to at least solve the problem in related technologies where the final trained student model struggles to achieve more accurate image recognition due to differences in feature distribution at different granularities between the teacher model and the student model. The technical solution of this disclosure is as follows: According to a first aspect of this disclosure, a training method for an image recognition model based on feature recombination is provided. The training method includes: inputting training sample images from a training sample set into a student model and a teacher model respectively to obtain a first intermediate feature of the student model and a second intermediate feature of the teacher model, wherein both the student model and the teacher model are image recognition models used to identify target objects in the training sample images; determining a first distribution of global features and detail features in the first intermediate feature and a second distribution of global features and detail features in the second intermediate feature, wherein the global features represent the overall information of the training sample image and the detail features represent the detail information of the training sample image; recombining the first intermediate feature by adjusting the first distribution toward the second distribution to obtain recombined features; obtaining a recombined feature loss by comparing the second intermediate feature and the recombined features; and training the student model based on the recombined feature loss to obtain a trained student model.

[0006] Optionally, the first distribution is represented by a first spectral ratio between a first overall feature and a first detail feature in the first intermediate feature, and the second distribution is represented by a second spectral ratio between a second overall feature and a second detail feature in the second intermediate feature. The step of recombining the first intermediate feature by adjusting the first distribution toward the second distribution to obtain the recombined feature includes: determining a balance factor based on the first spectral ratio and the second spectral ratio, wherein the balance factor characterizes the difference between the first spectral ratio and the second spectral ratio; weighting the detail features and / or overall features in the first intermediate feature based on the balance factor; and obtaining the recombined feature by recombining the weighted detail features and / or overall features.

[0007] Optionally, the training method includes multiple training rounds, wherein determining the balance factor based on the first spectrum ratio and the second spectrum ratio includes: determining the ratio of the current training round to the preset total training rounds; and determining the balance factor based on the first spectrum ratio, the second spectrum ratio, and the ratio.

[0008] Optionally, the first global feature and the first detail feature are obtained by: aligning the first intermediate feature to the second intermediate feature according to the channel dimension and spatial resolution of the second intermediate feature to obtain the aligned first intermediate feature; performing feature decomposition on the aligned first intermediate feature to obtain the first global feature and the first detail feature.

[0009] Optionally, aligning the first intermediate feature to the second intermediate feature based on the channel dimension and spatial resolution of the second intermediate feature to obtain the aligned first intermediate feature includes: performing convolutional feature refinement on the first intermediate feature to obtain the refined first intermediate feature; and performing position-aware cross-attention alignment on the refined first intermediate feature based on the channel dimension and spatial resolution of the second intermediate feature to obtain the aligned first intermediate feature, wherein the spatial resolution of the aligned first intermediate feature is consistent with the spatial resolution of the second intermediate feature.

[0010] Optionally, the student model and the teacher model include corresponding feature processing stages, and there are multiple feature processing stages. The recombination feature loss is determined for each feature processing stage. The step of training the student model based on the recombination feature loss to obtain a trained student model includes: weighting and summing the recombination feature losses corresponding to each feature processing stage based on the distillation weights set for each feature processing stage to obtain a total recombination feature loss; and training the student model based on the total recombination feature loss to obtain a trained student model.

[0011] According to a second aspect of this disclosure, an image recognition method is provided, the image recognition method comprising: acquiring a target image; inputting the target image into an image recognition model, using the image recognition model to recognize a target object in the target image, and obtaining a recognition result, wherein the image recognition model is trained according to the image recognition model training method described in this disclosure.

[0012] According to a third aspect of this disclosure, a training apparatus for an image recognition model based on feature recombination is provided. The training apparatus includes: an input unit configured to input training sample images from a training sample set into a student model and a teacher model respectively, to obtain a first intermediate feature of the student model and a second intermediate feature of the teacher model, wherein both the student model and the teacher model are image recognition models used to identify target objects in the training sample images; a determination unit configured to determine a first distribution of global features and detail features in the first intermediate feature and a second distribution of global features and detail features in the second intermediate feature, wherein the global features represent the overall information of the training sample image and the detail features represent the detail information of the training sample image; a recombination unit configured to recombine the first intermediate feature by adjusting the first distribution toward the second distribution to obtain recombined features; a comparison unit configured to obtain a recombination feature loss by comparing the second intermediate feature and the recombined features; and a training unit configured to train the student model based on the recombination feature loss to obtain a trained student model.

[0013] According to a fourth aspect of this disclosure, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein, when executed by the processor, the processor causes the processor to perform a training method or image recognition method for an image recognition model based on feature reorganization according to this disclosure.

[0014] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform a training method or image recognition method for an image recognition model based on feature recombination according to this disclosure.

[0015] According to a sixth aspect of this disclosure, a computer program product is provided, comprising computer-executable instructions that, when executed by at least one processor, implement a training method or image recognition method for an image recognition model based on feature reorganization according to this disclosure.

[0016] The technical solution provided in this disclosure brings at least the following beneficial effects: According to the feature-recombined image recognition model training and image recognition scheme disclosed herein, during the training of the student model using the teacher model, the distribution of overall and detail features in the intermediate features of the teacher model and the distribution of overall and detail features in the intermediate features of the student model can be analyzed. By adjusting the distribution of overall and detail features in the intermediate features of the student model towards the feature distribution of the teacher model, and calculating the loss between the adjusted features and the features of the teacher model, the feature distribution of the student model can be made closer to the feature distribution of the teacher model during training. This makes the image recognition ability of the finally trained student model closer to that of the teacher model, thereby improving the recognition effect of both overall features (e.g., structural contours) and detail features (e.g., texture details) of the image, achieving more accurate image recognition, and avoiding the problem of unbalanced recognition ability of the model for overall and detail features of the image.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0019] Figure 1This is a schematic flowchart of a training method for an image recognition model based on feature reconstruction according to an exemplary embodiment of the present disclosure.

[0020] Figure 2 This is a schematic flowchart illustrating the process of obtaining a first global feature and a first detail feature in a training method for an image recognition model based on feature reorganization according to an exemplary embodiment of the present disclosure.

[0021] Figure 3 This is a schematic flowchart illustrating the process of obtaining aligned first intermediate features in a training method for an image recognition model based on feature reorganization according to an exemplary embodiment of the present disclosure.

[0022] Figure 4 This is a schematic block diagram of the first aligned intermediate feature obtained in a training method for an image recognition model based on feature reorganization according to an exemplary embodiment of the present disclosure.

[0023] Figure 5 This is a schematic block diagram illustrating the first global feature and the first detail feature obtained in a training method for an image recognition model based on feature reorganization according to an exemplary embodiment of the present disclosure.

[0024] Figure 6 This is a schematic flowchart illustrating the process of obtaining reconstructed features in a training method for an image recognition model based on feature reconstructing according to an exemplary embodiment of the present disclosure.

[0025] Figure 7 This is a schematic block diagram illustrating the determination of knowledge distillation loss in a training method for an image recognition model based on feature reconstruction, according to an exemplary embodiment of the present disclosure.

[0026] Figure 8 This is a schematic diagram illustrating the effect of a training method for an image recognition model based on feature recombination according to an exemplary embodiment of the present disclosure.

[0027] Figure 9A and Figure 9B This is a schematic diagram comparing the image recognition results of models trained using conventional training methods and training methods of image recognition models according to exemplary embodiments of the present disclosure. Detailed Implementation

[0028] In order to enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0030] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0031] As mentioned earlier, in related technologies, the differences in feature distribution between teacher and student models at different granularities may make it difficult for the final trained student model to achieve more accurate image recognition.

[0032] For example, with the diversification of visual backbone network architectures, architectures such as Convolutional Neural Networks (CNN), Vision Transformers (ViT), and Multilayer Perceptrons (MLP) coexist. In distillation scenarios where teacher and student models operate across architectures, differences in receptive field modeling methods, feature computation paradigms, and model layer structures lead to significant differences in the spectral distribution of coarse-grained and fine-grained representations in their feature spaces. In some cases, transfer learning can be achieved by establishing surrogate paths or direct feature fitting. However, when the student model is forced to fit a feature distribution that is difficult to form naturally, distillation instability and performance degradation can easily occur, making it difficult to achieve good training results. Furthermore, similar problems exist in distillation scenarios where teacher and student models operate within the same architecture.

[0033] In view of the above problems, exemplary embodiments of this disclosure provide a training method for an image recognition model based on feature reconstruction, a training apparatus for an image recognition model based on feature reconstruction, an image recognition method, an electronic device, a computer-readable storage medium, and a computer program product, which can solve or at least alleviate the above problems.

[0034] In a first aspect of an exemplary embodiment of this disclosure, a method for training an image recognition model based on feature recombination is provided.

[0035] The training method for an image recognition model based on feature recombination according to an exemplary embodiment of the present disclosure can be applied to training scenarios of image recognition models. For example, software can be loaded on a user terminal, and the user can input model training instructions on the user terminal. The user terminal can obtain a trained image recognition model by executing the training method for an image recognition model according to an exemplary embodiment of the present disclosure. The image recognition model can be used to recognize images, such as identifying target objects in images, for image classification, image segmentation, image semantic segmentation, etc.

[0036] The aforementioned user terminal can be such as a tablet computer, laptop computer, digital assistant, wearable device, etc. However, the implementation scenario of the above method is only an example scenario. The training method of the image recognition model according to the exemplary embodiment of this disclosure can also be applied to other application scenarios. For example, the user can request the training model from the server via the network on the user terminal (e.g., mobile phone, desktop computer, tablet computer, etc.). The server can complete the request by executing the training method of the image recognition model according to the exemplary embodiment of this disclosure. Here, the server can be an independent server, a server cluster, a cloud computing platform, or a virtualization center.

[0037] The training method for an image recognition model based on feature reorganization according to an exemplary embodiment of the present disclosure can make the feature distribution of the student model move closer to the feature distribution of the teacher model during training, so that the image recognition ability of the finally trained student model is closer to that of the teacher model. This improves the recognition effect of both the overall features (e.g., structural contours) and the detailed features (e.g., texture details) of the image, achieving more accurate image recognition and avoiding the problem of the model's unbalanced recognition ability of the overall features and detailed features of the image.

[0038] Examples of training methods for image recognition models according to embodiments of the present disclosure will now be described with reference to the accompanying drawings.

[0039] In this training method, the student model can be trained by knowledge distillation based on the training sample images and teacher model in the training sample set to obtain the trained student model.

[0040] Specifically, such as Figure 1 As shown, the training method for an image recognition model based on feature reconstruction according to embodiments of this disclosure may include the following steps: In step S110, the training sample images in the training sample set can be input into the student model and the teacher model respectively to obtain the first intermediate feature of the student model and the second intermediate feature of the teacher model.

[0041] Here, both the student model and the teacher model can be image recognition models used to identify target objects in training sample images.

[0042] As an example, there can be multiple training sample images. The format, size, and content of the training sample images can be arbitrary, and the target object can be any preset content in the image. For example, the sample image can be a photograph or picture captured by an image capture device, and the target object can be a person, animal, building, object, etc. in the photograph or picture.

[0043] In addition, the training sample set may also include sample labels, which are used to annotate the target object to be identified in each training sample image. During training, the sample labels can be used to compare with the recognition results output by the student model to determine a portion of the training loss, which may be referred to as the image recognition task loss, as will be described in detail below.

[0044] As an example, the teacher model can be pre-trained (e.g., it can be trained using any training method), the student model can be a model to be trained, and the teacher model can be used to transfer knowledge to the student model in order to train the student model. For example, the prediction results of the teacher model can be used to train the student model.

[0045] As an example, the teacher model and the student model can be models with the same architecture (also known as "isomorphic models") or models with different architectures (also known as "heterogeneous models"). For example, the teacher model can be one of the CNN architecture, MLP architecture, and ViT architecture, and the student model can also be one of the CNN architecture, MLP architecture, and ViT architecture. It can be the same as or different from the teacher model's architecture.

[0046] The training method for the image recognition model according to embodiments of this disclosure can be combined with existing training schemes for knowledge distillation-based image recognition models. Based on existing knowledge distillation-based image recognition model training schemes, the training method for the image recognition model according to embodiments of this disclosure optimizes the knowledge distillation process. As an example, existing knowledge distillation-based image recognition model training schemes may include, but are not limited to, the ViT-CNN transfer learning scheme.

[0047] Furthermore, in this step, intermediate features can be features output between the input and output layers of the model, such as output features from any one or more layers of the model except the last layer.

[0048] As an example, the student model and the teacher model can include corresponding feature processing stages. There can be multiple feature processing stages; for example, the teacher model T and the student model S can each contain N feature processing stages, and the N feature processing stages of the teacher model T can correspond one-to-one with the N feature processing stages of the student model S. As an example, each feature processing stage can include one or more layers of the model. As an example, when the teacher model and the student model adopt different architectures, the two models can have different structures. In this case, the corresponding stages between them can be predefined. For example, layers in the teacher model and the student model that have the same or similar functions or feature processing can be used as corresponding stages, as described below. Figure 7 For example, stage 1 can be the stage that performs convolution operations on the input features, stage 2 can be the stage that performs attention interaction operations on the output features of stage 1, and stage 3 can be the stage that performs convolution operations on the output features of stage 2. Each of the above stages can include one or more network layers, and the number of layers included in the corresponding stages of the teacher model and the student model can be the same or different. For example, the number of network layers included in stage 1 of the student model can be different from the number of network layers included in stage 1 of the teacher model.

[0049] In step S110, the training sample images can be input into the teacher model T and the student model S respectively to obtain the first training sample image of the student model S. The first intermediate feature of each feature processing stage And the teacher model T's The second intermediate feature of each feature processing stage .

[0050] In step S120, the first distribution of global features and detailed features in the first intermediate feature and the second distribution of global features and detailed features in the second intermediate feature can be determined.

[0051] Here, the overall features can represent the overall information of the training sample image, while the detail features can represent the detailed information of the training sample image.

[0052] As an example, overall features may include at least one of the image's structural outline, main color, and main brightness (or darkness); detailed features may include at least one of the image's texture, grain, edges, color variations, and brightness (or darkness) variations.

[0053] As an example, in this step, the first intermediate feature and the second intermediate feature can be decomposed separately to obtain their respective overall features and detailed features. Then, based on their respective overall features and detailed features, their respective feature distributions can be determined. Specifically, the first distribution and the second distribution can be determined.

[0054] For example, feature decomposition can be performed on the first intermediate feature to obtain a first global feature and a first detail feature; feature decomposition can be performed on the second intermediate feature to obtain a second global feature and a second detail feature. The above feature decomposition can be implemented using existing feature decomposition schemes (e.g., frequency domain decomposition), and this disclosure does not impose any particular restrictions on it. An example of implementing the above feature decomposition will be given later.

[0055] Here, as an example, feature decomposition can be performed directly on the first intermediate feature to obtain the first overall feature and the first detailed feature.

[0056] As another example, when performing feature decomposition on the first intermediate feature, the first intermediate feature can be aligned with the feature space of the teacher model to obtain the aligned first intermediate feature, and then feature decomposition can be performed on the aligned first intermediate feature to obtain the first global feature and the first detail feature.

[0057] For example, such as Figure 2 As shown, the first global feature and the first detail feature can be obtained in the following ways: In step S210, the first intermediate feature can be aligned to the second intermediate feature according to the channel dimension and spatial resolution of the second intermediate feature to obtain the aligned first intermediate feature.

[0058] As an example, such as Figure 3 As shown, step S210 may include the following steps: In step S310, convolutional feature refinement can be performed on the first intermediate feature to obtain the refined first intermediate feature.

[0059] For example, such as Figure 4 As shown, the first intermediate feature can be tokenized using a tokenizer. Convolutional feature refinement is performed to obtain the refined first intermediate features. Specifically, the tokenizer can include a 3×3 2D convolutional layer (Conv2D), a batch normalization layer (BatchNorm), and an activation layer (ReLU), where the ReLU activation layer can be implemented using the ReLU function. Furthermore, the first intermediate features can be, for example, CNN features or MLP / ViT tokens, depending on the architecture of the student model.

[0060] In step S320, position-aware cross-attention alignment can be performed on the refined first intermediate feature based on the channel dimension and spatial resolution of the second intermediate feature to obtain the aligned first intermediate feature.

[0061] Here, the channel dimension and spatial resolution of the aligned first intermediate feature can be consistent with the channel dimension and spatial resolution of the second intermediate feature.

[0062] Specifically, in this step, position embedding can be performed on the refined first intermediate feature. For example, the position of each pixel can be added as a separate column to the refined first intermediate feature. Here, the position of each pixel can serve as labeling information, always marking the position of each pixel in the first intermediate feature. Figure 4 For example, the feature obtained by position embedding the refined first intermediate feature output by the marker can be represented as follows: ,in, n This indicates the batch size during training. Indicates the height of the first intermediate characteristic after refining. This indicates the width of the first intermediate feature after refinement. This indicates the number of channels in the first intermediate feature after refinement.

[0063] In this step, position-aware cross-attention alignment is performed on the first intermediate feature after position embedding to obtain the aligned first intermediate feature.

[0064] As an example, a key vector K and a value vector V can be generated based on the refined first intermediate features of the position embedding, and a trainable query vector Q can be generated based on the channel dimension and spatial resolution of the second intermediate features. The key vector, value vector and trainable query vector are then fed into a multi-head PosAttention mechanism to align the refined first intermediate features of the position embedding to the channel dimension and spatial resolution of the teacher model.

[0065] For example, such as Figure 4 As shown, channel dimension mapping can be performed on the refined first intermediate feature to match the number of channels of the second intermediate feature. Specifically, we can assume that the teacher model T has the following channel dimension mapping: The second intermediate feature of each feature processing stage is And the first student model S The first intermediate feature of each feature processing stage is Where H and W represent features such as the first intermediate feature, respectively. Second intermediate features The spatial resolution of the current features (e.g., feature maps). and These represent the number of channels in the teacher model and the number of channels in the student model, respectively.

[0066] like Figure 4 As shown, the second intermediate feature can be... Perform a convolutional refinement operation to obtain the refined second intermediate feature. Specifically, this refinement can be achieved using a 1×1 2D convolutional layer (Conv2D) and a ReLU activation layer, where the ReLU activation layer can be implemented using the ReLU function. Furthermore, the second intermediate feature can be represented, for example, as... ,in, n Indicates the batch size during training. This indicates the height of the second intermediate characteristic after refining. This indicates the width of the refined second intermediate feature. This indicates the number of channels for the refined second intermediate feature.

[0067] As mentioned above, the cross-attention mechanism can be a location-aware cross-attention mechanism.

[0068] As an example, such as Figure 4 As shown, the cross-attention mechanism can be represented as Where, Q = W_q·Φ( ), K = W_k·Ψ( V = W_v·Ψ( ), where Φ(·) represents the convolutional refinement operation on the second intermediate feature, Φ( ) represents the refined second intermediate feature, and Ψ(·) represents the convolutional refinement operation (or projection / normalization operation) on the first intermediate feature embedded at the position. ) represents the refined first intermediate feature after location embedding; W_q, W_k, and W_v are learnable parameters; d is the attention dimension.

[0069] Through such Figure 4 The feature alignment process shown can perform attention projection alignment on the first intermediate features to obtain aligned first intermediate features that are consistent with the feature space of the teacher model. For example, Figure 4 The example process shown illustrates performing attention projection alignment on a first intermediate feature to obtain an aligned first intermediate feature, which can be utilized by performing... Figure 4 The attention projector of the process shown makes the first intermediate feature The second intermediate feature of the teacher model in terms of channel dimension and spatial resolution Consistency. This process can be formally represented, for example, as follows: , ,in, This represents the process of attention projection alignment. Indicates the first intermediate feature. This represents the first intermediate feature after alignment. and This represents the spatial resolution of the feature space of the teacher model. Here, only the student features can be transformed, resulting in the second intermediate feature of the teacher model. Keep it unchanged to avoid perturbing the original distribution of the teacher model.

[0070] Return to reference Figure 2 In step S220, feature decomposition can be performed on the aligned first intermediate feature to obtain the first overall feature and the first detailed feature.

[0071] As an example, such as Figure 5 As shown, this can be applied to the first intermediate feature after alignment. Perform frequency domain decomposition. Specifically, for the aligned first intermediate feature... Perform a two-dimensional (2D) discrete cosine transform (DCT) on each channel to obtain the spatial frequency domain distribution.

[0072] For example, the first intermediate feature after alignment The two-dimensional discrete cosine transform can be expressed as , ,in, Represents the 2D DCT operator; This represents the spatial frequency domain representation of the first intermediate feature after alignment.

[0073] like Figure 5 As shown, a DC mask that retains only the DC coefficients (u=0, v=0) can be used. frequency domain features It can be decomposed into DC component and AC component, specifically, DC component and AC components They can be represented as follows: and ,in, This represents element-wise multiplication; This represents a DC mask, which is a binary mask that sets 1 at (u=0, v=0) and 0 at all other positions. This represents the AC mask, which is reserved for the DC mask. Complementary AC components. Here, the DC component can be considered as... As the first overall feature mentioned above, the AC component This can be considered as the first detailed feature mentioned above.

[0074] The above reference Figure 5 The process of determining a first global feature and a first detail feature for an aligned first intermediate feature is described. In embodiments of this disclosure, the process of directly determining the first global feature and the first detail feature for the original first intermediate feature, and the process of determining the second global feature and the second detail feature for a second intermediate feature, are described above. Figure 5 The process described is similar. Below is an example of a general feature decomposition.

[0075] As an example, for a first intermediate feature, a second intermediate feature, or an aligned first intermediate feature, the above feature decomposition may include: in the spatial dimension, for each channel, transforming the current feature (e.g., the first intermediate feature, the second intermediate feature, or the aligned first intermediate feature) to the frequency domain to obtain frequency domain features; and separating the overall features and detail features from the frequency domain features.

[0076] For example, for each intermediate feature, a two-dimensional discrete cosine transform (2D DCT) can be performed channel by channel to obtain the frequency domain features. Specifically, the pixel domain can be transformed to the frequency domain using the following equation (1): (1) in, This represents the first frequency domain feature at position (u,v) in the feature map after transformation on channel c. The feature map representing the current feature (e.g., the first intermediate feature (or the second intermediate feature or the aligned first intermediate feature)). Indicates the height of the image. Indicates the width of the image. The height of the image is represented by the first... 1 pixel, The width of the image is represented by the first... 1 pixel, This represents the pixel position along the height direction in the transformed feature map. This represents the pixel position in the width direction of the transformed feature map, where (u=0, v=0) corresponds to the low-frequency structural components, and the other frequency coefficients correspond to the high-frequency texture components.

[0077] As an example, a binary mask can be used to separate spatial global features and spatial detail features from frequency domain features; the spatial global features are used as the global features corresponding to the current features, and the spatial detail features are used as the detail features corresponding to the current features, where the spatial global features represent the image portion with concentrated energy in the current features, and the spatial detail features represent the image portion with dispersed energy in the current features.

[0078] Here, spatial global features represent the image portion with concentrated energy in the current features, such as large areas of color in the image; spatial detail features represent the image portion with dispersed energy in the current features, such as edges, textures, and noise in the image.

[0079] As an example, overall spatial features can be represented by spatial direct current (DC-spatial) components, while detailed spatial features can be represented by spatial alternating current (AC-spatial) components. The spatial DC and AC components can be separated from the frequency domain features using a binary mask. For example, the overall spatial features can be represented as: ,in, This represents a binary mask, for example, which can preserve coefficients (u=0, v=0); spatial detail features can be represented as: .

[0080] By performing the above method, a first global feature and a first detailed feature can be obtained from the first intermediate feature decomposition, and a second global feature and a second detailed feature can be obtained from the second intermediate feature decomposition.

[0081] In this case, as an example, the first distribution can be represented by a first spectral ratio between the first overall feature and the first detail feature in the first intermediate feature, and the second distribution can be represented by a second spectral ratio between the second overall feature and the second detail feature in the second intermediate feature.

[0082] As an example, the ratio of DC component intensity to AC component intensity can be defined for the first intermediate feature and the second intermediate feature, respectively. As a spectral ratio, for example, the first spectral ratio and the second spectral ratio can be calculated using the following equations (2) and (3), respectively: (2) (3) Where ||·||1 represents the L1 norm (for example, the L2 norm can also be used); ε represents a very small positive number used to prevent the denominator from being zero, which can be set according to actual needs; and These represent the second intermediate features of the teacher model. Applying a DCT and mask of the same type as the first intermediate feature of the student model yields DC and AC components (which can be used only for statistical purposes). Without changing the second intermediate feature (Itself). Here, , , He Jun This represents the feature in the frequency domain.

[0083] In step S130, the first intermediate feature can be recombined by adjusting the first distribution toward the second distribution to obtain the recombined feature.

[0084] As an example, in the case where the first distribution is represented by a first spectral proportion and the second distribution is represented by a second spectral proportion, such as Figure 6 As shown, step S130 may include the following steps: In step S610, the balance factor can be determined based on the first spectrum ratio and the second spectrum ratio.

[0085] Here, the balance factor can characterize the difference between the first spectral proportion and the second spectral proportion.

[0086] In one example, the balance factor can be determined based on the difference or ratio between the first spectral proportion and the second spectral proportion.

[0087] In another example, the training method according to an embodiment of the present disclosure may include multiple training rounds. In this example, step S610 may include: determining the ratio of the current training round to a preset total number of training rounds; and determining a balance factor based on a first spectral ratio, a second spectral ratio, and the ratio.

[0088] Specifically, in this example, the balance factor can be determined based on the difference (or ratio) between the first spectrum ratio and the second spectrum ratio, and the ratio of the current training epoch to the preset total training epochs. For example, the balance factor can be expressed by the following equation (4): (4) in, t represents the current training epoch or the epoch number corresponding to the current training epoch, and T represents the total number of training epochs; γ represents the hyperparameters associated with the network pair, which can be used to provide an initial balance bias, and can be preset according to actual needs. In the above equation (4), the balance factor r can be monotonically changed with the training process through the t / T term to gradually calibrate the cross-architecture spectrum ratio difference. Therefore, in the embodiments of this disclosure, the balance factor r can be dynamically changed with the training epoch (or with the training process).

[0089] In step S620, the detailed features and / or overall features in the first intermediate features can be weighted based on a balance factor.

[0090] In step S630, recombined features can be obtained by recombining the weighted detailed features and / or overall features.

[0091] In embodiments of this disclosure, the balance factor can be used to measure the difference in spectral proportion between the teacher model and the student model. The balance factor can be used to weight and recombine the DC and / or AC components of the first intermediate feature to adjust the distribution ratio of the overall feature and the detailed feature in the first intermediate feature, so that the distribution ratio of the recombined first intermediate feature is closer to that of the second intermediate feature than that before recombining.

[0092] For example, such as Figure 5 As shown, after determining the first overall feature of the first intermediate feature after alignment (e.g., the DC component mentioned above), ) and first detailed features (e.g., the AC components mentioned above) In the case of ), it can be achieved through the inverse transform of the two-dimensional discrete cosine transform ( iDCT ), thus obtaining the pixel-domain representation of the first global feature and the pixel-domain representation of the first detail feature.

[0093] In this case, a balance factor can be used to weight or adaptively scale the pixel domain representation of the first detail feature and / or the pixel domain representation of the first global feature, and the weighted detail feature and / or global feature can be recombined to obtain the recombined feature.

[0094] As an example, taking the weighting of only the pixel domain representation of the first detail feature as an example, the pixel domain representation of the first overall feature can be recombined with the weighted first detail feature. For example, the recombined feature can be represented by the following formula (5): (5) in, Indicates recombination characteristics, (·) denotes the inverse DCT operator. r This represents the balance factor. Here, the DC component mainly carries global structural information, while the AC component mainly carries local detail information. By adaptively adjusting the AC component using the balance factor based on the teacher model's perception, the recombined features can be optimized. A spectrum characteristic that is closer to that of the teacher.

[0095] Although the example described above uses Equation (5) to illustrate the weighting of only the pixel domain representation of the first detail feature, the embodiments of this disclosure are not limited thereto. Weighting can also be applied only to the pixel domain representation of the first global feature, or to both the pixel domain representations of the first detail feature and the first global feature. This only requires adjusting the balance factor in Equation (5). For example, the balance factor can be set as follows: r =b / (a+b), we can use b to weight the pixel domain representation of the first detail feature, use a to weight the pixel domain representation of the first overall feature, and sum the weighted features to obtain the recombined features.

[0096] Return to reference Figure 1 In step S140, the recombination feature loss can be obtained by comparing the second intermediate feature and the recombination feature.

[0097] In step S150, the student model can be trained based on the recombinant feature loss to obtain the trained student model.

[0098] In the embodiments of this disclosure, the recombined feature loss can be calculated based on the recombined features and the corresponding second intermediate features output by the teacher model. This recombined feature loss can be combined with the original knowledge distillation loss and the image recognition task loss to determine the total training loss. The learnable parameters of the student model are then adjusted based on this total training loss, thereby enabling the training of the student model. Here, the original knowledge distillation loss is determined based on a comparison between the first intermediate features of the student model and the second intermediate features of the teacher model, and the image recognition task loss is determined based on a comparison between the first intermediate features of the student model and the sample labels in the training sample set.

[0099] As an example, the student model and the teacher model may include multiple feature processing stages corresponding to each other, and a recombined feature and a second intermediate feature may be determined for any target feature processing stage to determine the recombined feature loss corresponding to that feature processing stage.

[0100] As another example, recombination features and second intermediate features can be determined for any number of target feature processing stages, and the recombination feature loss can be determined for each target feature processing stage. In this example, step S150 may include: weighting and summing the recombination feature losses corresponding to each feature processing stage based on the distillation weights set for each feature processing stage to obtain the total recombination feature loss; and training the student model based on the total recombination feature loss to obtain the trained student model.

[0101] For example, such as Figure 7 As shown, multi-stage distillation or multi-layer distillation (e.g., at least two stages or layers) can be employed. Specifically, this can be done in multiple stages of the teacher model and the student model (e.g. Figure 7 In stages 1, 2, and 3, the aforementioned "Alignment-Decomposition-Reassemble Module" (FDRM) is inserted, and the reassembled feature loss is calculated for the reassembled features obtained in each stage and the corresponding second intermediate features. And can be done through weights A balance is struck to achieve a trade-off between training efficiency and training effectiveness. In this example, the recombinant feature loss can be obtained by calculating and weighting the recombinant features from multiple stages and their corresponding second intermediate features. Here, FDRM is as described above. Figure 5 The operation shown.

[0102] Specifically, in the above multi-stage distillation scenario, the total distillation loss, which combines the total recombination feature loss and the original knowledge distillation loss, can be expressed by the following equation (6): (6) in, Indicates total distillation loss. Indicates the first i The distillation weights for each target feature processing stage can be set according to actual needs. This represents the logits layer distillation term (which can represent the original knowledge distillation loss). For example, the KL divergence loss is based on the softmax probability distribution; MSE(·) represents the mean square error; Indicates the first i Reconstruction feature loss in each target feature processing stage , The student model represents the first i Reorganization features in the target feature processing stage The corresponding first term of the teacher model i The second intermediate feature in the target feature processing stage; These represent the logits output by the student model and the teacher model, respectively. N This represents the total number of feature processing stages. Here, logits represents the raw values ​​output by the last layer of the model, such as the real number vector directly generated by the last layer of the model. As an example, in an embodiment of this disclosure, in order to balance training efficiency and training effect, N=4 can be set and only stages i∈{3,4} can be selected to participate in distillation.

[0103] The final training objective can be to jointly optimize the image recognition task loss and the total distillation loss mentioned above. For example, the total training loss can be represented by the following equation (7): (7) in, This represents the original task loss (e.g., classification cross-entropy or semantic segmentation loss), such as the loss between the student model's prediction for an image recognition task and the true label for the image recognition task.

[0104] Figure 8This is a schematic diagram illustrating the effect of a training method for an image recognition model according to an exemplary embodiment of the present disclosure. Specifically, it shows how the feature distribution ratio of the AC / DC space changes with the training process. Figure 8 In the test, the ImageNet-1K dataset was used, and the ResNet-18 architecture was used as the student model. The feature processing stage N consisted of 4 network layers, or 4 stages. Figure 8 In the diagram, the horizontal axis represents the number of training epochs, and the vertical axis represents the feature distribution ratio of the student model. For example, it can be equal to the spatial ratio of the AC components / DC components. In this test example, the feature distribution ratio of the student model is 2.500. Figure 8 As shown, by applying the training method of the embodiments of this disclosure, the feature distribution ratio of the student model can gradually approach the feature distribution ratio of the teacher model as the number of training rounds increases. Specifically, from training round 1 to training round 100, the feature distribution ratio decreases by a total of 0.883, which is 24.4%, and its goodness of fit reaches 0.9966.

[0105] Figure 9A and Figure 9B This is a schematic diagram comparing the image recognition results of models trained using conventional training methods and training methods of image recognition models according to exemplary embodiments of the present disclosure.

[0106] exist Figure 9A and Figure 9B In the test, the ImageNet-1K dataset was used, which has 1.28 million training images, 50,000 validation images, 1,000 classes, and an input resolution of 224×224. The evaluation metric is the validation set single-crop Top-1 accuracy.

[0107] In this test, for traditional training measurement, the optimizer was SGD, momentum was 0.9, batch size was 256, weight decay was 1e-4, and the learning rate strategy adopted was step-wise learning rate scheduling, with the learning rate decayed by a factor of 10 at predefined training epochs. The data augmentation method was standard inception-style augmentation. For strong training strategy, the optimizer was AdamW, weight decay was 0.05, batch size was 1024, number of training epochs was 300, initial learning rate was 1e-3, learning rate strategy was cosine decay, and data augmentation and training techniques included drop path, label smoothing, random flip, random augmentation, and random erasing.

[0108] exist Figure 9A In this example, an ostrich is set as the target object to be identified, and the image recognition task is to identify the target object from an image; Figure 9B In this context, image recognition tasks involve identifying and classifying objects of different categories within an image, such as a bed or a bookshelf. Figure 9A and Figure 9B As shown, compared with the student model trained using traditional methods, the image recognition results of the student model trained according to this method are closer to the image recognition results of the teacher model, and can be closer to the real image recognition labels.

[0109] In a second aspect of an exemplary embodiment of the present disclosure, an image recognition method is provided, which may include: acquiring a target image; inputting the target image into an image recognition model; using the image recognition model to recognize a target object in the target image and obtaining a recognition result, wherein the image recognition model is trained according to the image recognition model training method of the embodiments of the present disclosure, for example, it may be the aforementioned trained student model.

[0110] The specific steps and details of the image recognition method according to the exemplary embodiments of this disclosure can be implemented and understood similarly with reference to the training method of the image recognition model described in the first aspect above, and accordingly, the image recognition method has the same or similar beneficial effects as the training method of the image recognition model described above, which will not be repeated here.

[0111] The training method and image recognition method of the feature-reconstruction-based image recognition model according to the exemplary embodiments of the present disclosure can adaptively calibrate the differences in coarse-to-fine representation distribution between different network architectures, reduce the feature distribution gap across architectures, and thus improve the stability and effectiveness of cross-architecture knowledge distillation.

[0112] According to exemplary embodiments of this disclosure, the differences in spectral distribution between different architectures are explicitly calibrated by performing a pluggable "align-decompose-reassemble" process on student-side features, thereby improving the effectiveness and robustness of cross-architecture distillation.

[0113] In the training method of the exemplary embodiments of this disclosure, in each training round, intermediate features of the teacher model and the student model can be obtained first; attention projection alignment is performed only on the intermediate features of the student model to align with the teacher features in dimensions such as channel number and spatial resolution; then, frequency domain decomposition is performed on the aligned student features to separate the DC component and the AC component; a dynamic balance factor is determined based on the difference between the AC / DC intensity ratio of the teacher and the student, and the AC component is weighted by the balance factor and then recombined with the DC component to obtain the recombined features; finally, the student model is trained using the feature distillation loss between the recombined features and the teacher features combined with the knowledge distillation loss.

[0114] The method according to exemplary embodiments of this disclosure can explicitly model key barriers to cross-architecture distillation as “coarse-to-fine spectral distribution differences” and can eliminate the distribution gap with DC / AC controlled recombination.

[0115] Furthermore, the method according to exemplary embodiments of this disclosure can perform frequency domain decomposition and recombination of student features after feature alignment, so that the recombined features are more adapted to the teacher distribution without disturbing the teacher features.

[0116] Furthermore, the method according to an exemplary embodiment of this disclosure can introduce a dynamic balancing factor based on the difference between the first distribution and the second distribution, combined with the training process (e.g., the t / T term mentioned above), to achieve adaptive calibration of network pairs with cross-architectural differences.

[0117] Furthermore, the method according to the exemplary embodiments of this disclosure can significantly reduce the difference in spectral distribution between student features and teacher features without changing the teacher model distribution, thereby improving the stability and effectiveness of cross-architecture feature distillation, and can be integrated as a lightweight, pluggable module into the distillation training process of various vision tasks.

[0118] According to a third aspect of the embodiments of this disclosure, a training apparatus for an image recognition model based on feature recombination is provided. The training apparatus includes: an input unit configured to input training sample images from a training sample set into a student model and a teacher model respectively, to obtain a first intermediate feature of the student model and a second intermediate feature of the teacher model, wherein both the student model and the teacher model are image recognition models used to identify target objects in the training sample images; a determination unit configured to determine a first distribution of global features and detail features in the first intermediate feature and a second distribution of global features and detail features in the second intermediate feature, wherein the global features represent the overall information of the training sample image and the detail features represent the detail information of the training sample image; a recombination unit configured to recombine the first intermediate feature by adjusting the first distribution toward the second distribution to obtain recombined features; a comparison unit configured to obtain a recombination feature loss by comparing the second intermediate feature and the recombined features; and a training unit configured to train the student model based on the recombination feature loss to obtain a trained student model.

[0119] As an example, the first distribution is represented by a first spectral ratio between a first global feature and a first detail feature in the first intermediate feature, and the second distribution is represented by a second spectral ratio between a second global feature and a second detail feature in the second intermediate feature. The recombination unit is configured to: determine a balance factor based on the first and second spectral ratios, wherein the balance factor characterizes the difference between the first and second spectral ratios; weight the detail features and / or global features in the first intermediate feature based on the balance factor; and obtain recombined features by recombination of the weighted detail features and / or global features.

[0120] As an example, the training method includes multiple training rounds, wherein the recombination unit is configured to: determine the ratio of the current training round to the preset total training rounds; and determine a balance factor based on a first spectral ratio, a second spectral ratio, and the ratio.

[0121] As an example, the training device also includes a feature decomposition unit configured to obtain a first global feature and a first detail feature by: aligning the first intermediate feature to the second intermediate feature according to the channel dimension and spatial resolution of the second intermediate feature to obtain an aligned first intermediate feature; and performing feature decomposition on the aligned first intermediate feature to obtain the first global feature and the first detail feature.

[0122] As an example, the feature decomposition unit is configured to: perform convolutional feature refinement on the first intermediate feature to obtain the refined first intermediate feature; and perform position-aware cross-attention alignment on the refined first intermediate feature according to the channel dimension and spatial resolution of the second intermediate feature to obtain the aligned first intermediate feature, wherein the spatial resolution of the aligned first intermediate feature is consistent with the spatial resolution of the second intermediate feature.

[0123] As an example, the student model and the teacher model include corresponding feature processing stages, and there are multiple feature processing stages. The recombination feature loss is determined for each feature processing stage. The training unit is configured to: perform a weighted summation of the recombination feature losses corresponding to each feature processing stage based on the distillation weights set for each feature processing stage to obtain the total recombination feature loss; and train the student model based on the total recombination feature loss to obtain the trained student model.

[0124] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0125] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein, when executed by the processor, the processor-executable instructions cause the processor to perform a training method for an image recognition model based on feature reorganization according to embodiments of the present disclosure or an image recognition method according to embodiments of the present disclosure.

[0126] As an example, an electronic device does not necessarily have to be a single device; it can also be a collection of any means or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. An electronic device can also be part of an integrated control system or system manager, or can be configured to interface with a server, either locally or remotely (e.g., via wireless transmission).

[0127] In electronic devices, processors may include central processing units (CPUs), graphics processing units (GPUs), programmable logic devices, dedicated processor systems, microcontrollers, or microprocessors. By way of example and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0128] The processor can execute instructions or code stored in memory, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transport protocol.

[0129] Memory can be integrated with the processor; for example, RAM or flash memory can be housed within an integrated circuit microprocessor. Alternatively, memory can comprise a separate device, such as an external disk drive, storage array, or other storage device that can be used by any database system. Memory and processor can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the memory.

[0130] In addition, electronic devices may include video displays (such as liquid crystal displays) and user interaction interfaces (such as keyboards, mice, touch input devices, etc.). All components of the electronic device can be interconnected via buses and / or networks.

[0131] In an exemplary embodiment, a computer-readable storage medium may also be provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the training method or image recognition method of the feature-based image recognition model as described in the exemplary embodiments above. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: a read-only memory (ROM), a random access memory (RAM), a random access programmable read-only memory (PROM), an electrically erasable programmable read-only memory (EEPROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), flash memory, non-volatile memory, a CD-ROM, a CD-R, a CD+R, a CD-RW, a CD+RW, a DVD-ROM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a DVD-RAM, a BD-ROM, a BD-R, or a BD-R. LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0132] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, the computer program product including computer-executable instructions, which, when executed by at least one processor, implement a training method or image recognition method for a feature-reconstruction-based image recognition model according to exemplary embodiments of the present disclosure.

[0133] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0134] Furthermore, it should be noted that although several examples of each step have been described above with reference to the specific accompanying drawings, it should be understood that the embodiments of this disclosure are not limited to the combinations given in the examples. The steps appearing in different drawings can be combined, and the execution order of each step can be changed, which will not be exhaustive here.

[0135] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A training method for an image recognition model based on feature reconstruction, characterized in that, The training method includes: The training sample images in the training sample set are input into the student model and the teacher model respectively to obtain the first intermediate feature of the student model and the second intermediate feature of the teacher model. The student model and the teacher model are both image recognition models used to identify target objects in the training sample images. A first distribution of global features and detail features in the first intermediate features and a second distribution of global features and detail features in the second intermediate features are determined, wherein the global features represent the overall information of the training sample image and the detail features represent the detail information of the training sample image. The first distribution is represented by a first spectral ratio between the first global features and the first detail features in the first intermediate features, and the second distribution is represented by a second spectral ratio between the second global features and the second detail features in the second intermediate features. By adjusting the first distribution toward the second distribution, the first intermediate feature is recombined to obtain the recombined feature; The recombination feature loss is obtained by comparing the second intermediate feature and the recombination feature. Based on the recombination feature loss, the student model is trained to obtain a trained student model. The step of adjusting the first distribution toward the second distribution to recombine the first intermediate feature and obtain the recombined feature includes: A balance factor is determined based on the first spectrum ratio and the second spectrum ratio, wherein the balance factor characterizes the difference between the first spectrum ratio and the second spectrum ratio; Based on the balance factor, the detailed features and / or overall features in the first intermediate feature are weighted; The recombined features are obtained by reorganizing the weighted detailed features and / or overall features. The training method includes multiple training rounds, wherein determining the balance factor based on the first spectrum ratio and the second spectrum ratio includes: Determine the ratio of the current training round to the preset total training rounds; The balance factor is determined based on the first spectrum ratio, the second spectrum ratio, and the ratio, wherein the balance factor changes monotonically with the training process.

2. The training method according to claim 1, characterized in that, The first overall feature and the first detailed feature are obtained in the following manner: Based on the channel dimension and spatial resolution of the second intermediate feature, the first intermediate feature is aligned to the second intermediate feature to obtain the aligned first intermediate feature; Feature decomposition is performed on the aligned first intermediate feature to obtain the first overall feature and the first detailed feature.

3. The training method according to claim 2, characterized in that, The step of aligning the first intermediate feature to the second intermediate feature based on the channel dimension and spatial resolution of the second intermediate feature to obtain the aligned first intermediate feature includes: Perform convolutional feature refinement on the first intermediate feature to obtain the refined first intermediate feature; Based on the channel dimension and spatial resolution of the second intermediate feature, position-aware cross-attention alignment is performed on the refined first intermediate feature to obtain the aligned first intermediate feature, wherein the channel dimension and spatial resolution of the aligned first intermediate feature are consistent with the channel dimension and spatial resolution of the second intermediate feature.

4. The training method according to any one of claims 1 to 3, characterized in that, The student model and the teacher model each include corresponding feature processing stages, and there are multiple feature processing stages. The recombined feature loss is determined for each feature processing stage. The step of training the student model based on the recombined feature loss to obtain the trained student model includes: Based on the distillation weights set for each feature processing stage, the recombination feature loss corresponding to each feature processing stage is weighted and summed to obtain the total recombination feature loss; The student model is trained based on the total recombination feature loss to obtain a trained student model.

5. An image recognition method, characterized in that, The image recognition method includes: Acquire the target image; The target image is input into an image recognition model, and the target object in the target image is identified using the image recognition model to obtain a recognition result. The image recognition model is trained using the image recognition model training method according to any one of claims 1 to 4.

6. A training device for an image recognition model based on feature reconstruction, characterized in that, The training device includes: The input unit is configured to input training sample images from the training sample set into the student model and the teacher model respectively, to obtain a first intermediate feature of the student model and a second intermediate feature of the teacher model, wherein the student model and the teacher model are both image recognition models used to identify target objects in the training sample images; The determining unit is configured to determine a first distribution of global features and detail features in the first intermediate features and a second distribution of global features and detail features in the second intermediate features, wherein the global features represent the overall information of the training sample image, the detail features represent the detail information of the training sample image, the first distribution is represented by a first spectral ratio between the first global features and the first detail features in the first intermediate features, and the second distribution is represented by a second spectral ratio between the second global features and the second detail features in the second intermediate features; The recombination unit is configured to recombine the first intermediate feature by adjusting the first distribution toward the second distribution to obtain a recombined feature. The comparison unit is configured to obtain a recombination feature loss by comparing the second intermediate feature and the recombination feature; The training unit is configured to train the student model based on the recombined feature loss to obtain a trained student model. The recombination unit is configured to: determine a balance factor based on the first spectral ratio and the second spectral ratio, wherein the balance factor characterizes the difference between the first spectral ratio and the second spectral ratio; weight the detail features and / or overall features in the first intermediate feature based on the balance factor; and obtain the recombined feature by recombination of the weighted detail features and / or overall features. The training includes multiple training rounds, and the recombination unit is configured to: determine the ratio of the current training round to the preset total training rounds; and determine the balance factor based on the first spectrum ratio, the second spectrum ratio, and the ratio.

7. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions. When the processor executes the processor, it causes the processor to perform the training method of the image recognition model based on feature reconstruction according to any one of claims 1 to 4 or the image recognition method according to claim 5.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the training method of the image recognition model based on feature reconstruction according to any one of claims 1 to 4 or the image recognition method according to claim 5.

9. A computer program product comprising computer-executable instructions, characterized in that, When the computer-executable instructions are executed by at least one processor, they implement the training method for the image recognition model based on feature reconstruction according to any one of claims 1 to 4 or the image recognition method according to claim 5.

Images