Method, apparatus, computer program product and image classification system for classifying images

Abstract

The application provides a kind of image classification method, device, computer program product and image classification system.The method comprises: obtaining an image to be classified;Feature extraction is carried out on the image to be classified, and a feature vector is obtained;Class adaptive coding mechanism is used to rotate the feature vector, and the encoded feature vector is obtained, so that the feature vectors of the same category have consistency in the feature space, and the feature vectors of different categories have differences in angle and / or direction in the feature space;According to the encoded feature vector, the image to be classified is classified, and a classification result is obtained.The scheme introduces category information into the feature encoding process of the classification network, improves the discrimination between categories, and further improves the accuracy of the classification result.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to an image classification method, an image classification device, a computer program product, and an image classification system. Background Technology

[0002] With the widespread application of deep neural networks in image classification tasks, the feature extraction capabilities of these models have been significantly enhanced. However, most current classification networks still face two core problems:

[0003] 1. Insufficient inter-class discrimination: The feature distributions of different categories may overlap significantly, affecting classification accuracy;

[0004] 2. Class information is not involved in the feature encoding process: Current feature extraction networks often use a uniform encoding method and fail to dynamically adjust the feature space structure according to class information.

[0005] In natural language processing and computer vision, Rotary Positional Embedding (RoPE) is widely used to incorporate the relative positional information of elements in a sequence. This method achieves an encoding scheme with relative semantic structure by performing sine and cosine rotations on feature vectors along different frequency dimensions.

[0006] Inspired by RoPE, this application proposes a novel encoding strategy: Class-Adaptive Rotary Feature Encoding (CARFE). This method constructs a class-specific rotation encoding structure, enabling different classes to have distinguishable rotation patterns in the feature space, thereby improving the model's inter-class discrimination capability. Summary of the Invention

[0007] The main objective of this application is to provide an image classification method, an image classification device, a computer program product, and an image classification system, aiming to solve the following problems:

[0008] 1. Improve the inter-class separability of image classification models in large-scale category scenarios;

[0009] 2. Introducing category information during the feature encoding stage gives the features "category awareness" in space;

[0010] 3. A class-adaptive rotation encoding mechanism suitable for CNN and Transformer architectures is proposed, which has periodic encoding capability and can be extended to massive class tasks.

[0011] To achieve the above objectives, according to one aspect of this application, an image classification method is provided, comprising: acquiring an image to be classified; extracting features from the image to be classified to obtain feature vectors; performing rotation encoding on the feature vectors using a category-adaptive encoding mechanism to obtain encoded feature vectors, such that feature vectors of the same category have consistency in the feature space, and feature vectors of different categories have differences in angle and / or direction in the feature space; classifying the image to be classified based on the encoded feature vectors to obtain a classification result.

[0012] Optionally, feature extraction is performed on the image to be classified to obtain a feature vector, including: extracting features from the image to be classified using the following formula:

[0013] Rot C =FC1(X d (1-1),

[0014] Where X represents the input image to be classified, FC1 represents a linear mapping layer that maps the image to be classified into features of dimension d and outputs a feature vector Rot with class size C. C .

[0015] Optionally, a category-adaptive encoding mechanism is used to rotate and encode the feature vector to obtain the encoded feature vector, including: extracting the maximum value in the feature vector; calculating the rotation angle of the feature vector of the maximum value based on the RoPE algorithm; and rotating and encoding the feature vector of the maximum value based on the rotation angle to obtain the encoded feature vector.

[0016] Optionally, extracting the maximum value from the feature vector includes: extracting the maximum value of the elements in the feature vector using the following formula:

[0017] ω j j = Max(Rot C ), 0≤j≤C-1(1-2),

[0018] The Max() function takes the maximum value of the input data, ω. j ω in j j Represents the feature vector Rot C The maximum value in, ω j In , j represents the index of the maximum value. During training, j is trained using regression supervision to fit the target class.

[0019] Optionally, based on the RoPE algorithm, the rotation angle of the eigenvector with the maximum value is calculated, including: calculating the rotation angle using the following formula:

[0020]

[0021] Where, θ i,j This represents the rotation angle of the j-th category.

[0022] Optionally, the feature vector of the maximum value is rotated and encoded based on the rotation angle to obtain the encoded feature vector, including: rotating the feature vector of the maximum value using the following formula:

[0023]

[0024] in, Let represent the rotation matrix of the j-th category; rotation encoding is performed based on the rotation matrix using the following formula:

[0025]

[0026] in, X represents d Data after processing by the linear mapping layer FC2 This represents the encoded feature vector.

[0027] Optionally, classifying the image to be classified based on the encoded feature vector to obtain a classification result includes: mapping the encoded feature vector onto a classification space, and calculating the probability value of each category corresponding to the encoded feature vector according to a classification function; and determining the category with the highest probability value as the classification result of the image to be classified.

[0028] According to another aspect of this application, an image classification apparatus is provided, comprising: an acquisition unit for acquiring an image to be classified; an extraction unit for extracting features from the image to be classified to obtain a feature vector; an adaptive encoding unit for performing rotation encoding on the feature vector using a category-adaptive encoding mechanism to obtain an encoded feature vector, such that feature vectors of the same category have consistency in the feature space, and feature vectors of different categories have differences in angle and / or direction in the feature space; and a classification unit for classifying the image to be classified based on the encoded feature vector to obtain a classification result.

[0029] According to another aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of any one of the image classification methods.

[0030] According to another aspect of this application, an image classification system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any of the image classification methods described herein.

[0031] By applying the technical solution of this application, a category-adaptive rotational encoding is introduced. This encoding is non-linear and essentially performs secondary supervision on the classification results, achieving an optimized distribution of feature vectors in the feature space, which can effectively improve the classification accuracy. Attached Figure Description

[0032] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0033] Figure 1 A hardware structure block diagram of a mobile terminal performing an image classification method according to an embodiment of this application is shown;

[0034] Figure 2 A flowchart illustrating an image classification method according to an embodiment of this application is shown.

[0035] Figure 3 A framework diagram of this application is shown;

[0036] Figure 4 The data flow diagram of this application is shown;

[0037] Figure 5 A structural block diagram of an image classification apparatus provided according to an embodiment of this application is shown.

[0038] The above figures include the following reference numerals:

[0039] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0040] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0042] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0044] gt: ground truth, the category of the target marked in the image.

[0045] CNN: Convolutional Neural Network.

[0046] Softmax is a very commonly used activation function, especially when dealing with multi-class classification problems. It can transform an unnormalized real vector into a probability distribution, where each element in the vector is compressed to between 0 and 1, and the sum of all elements is 1.

[0047] RoPE (Rotary Position Embedding) is an innovative positional encoding method used to inject positional information into large models. Its core idea is to perform position-related transformations on feature vectors using rotation matrices, thereby preserving relative positional relationships within the attention mechanism.

[0048] In the field of natural language processing, a token is translated as a "lexical unit". A token is the result of dividing text into the smallest semantic or syntactic units, which can be words, subwords, characters, or even punctuation marks.

[0049] As described in the background section, the accuracy of image classification in the prior art is low. To solve the above problems, embodiments of this application provide an image classification method, an image classification device, a computer program product, and an image classification system.

[0050] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0051] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for an image classification method according to an embodiment of the present invention. For example... Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0052] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the image classification method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0053] This embodiment provides a method for classifying images that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0054] Figure 2 This is a schematic flowchart illustrating an image classification method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0055] Step S201: Obtain the image to be classified;

[0056] Specifically, prepare a dataset of images to be classified. These images can be collected from various sources, such as photos taken by cameras, images downloaded from the Internet, or images in a database.

[0057] Step S202: Extract features from the above image to be classified to obtain a feature vector;

[0058] Specifically, advanced image feature extraction techniques, such as Convolutional Neural Networks (CNNs) or Visual Transformers (ViTs), are used to convert the images to be classified into a high-dimensional feature vector. This feature vector contains key visual information of the image and forms the basis for subsequent encoding and classification. The feature extraction process encodes the structure and texture of the image, providing important representational data for subsequent classification.

[0059] Step S203: The above feature vectors are rotated and encoded using a category-adaptive encoding mechanism to obtain encoded feature vectors, so that the feature vectors of the same category have consistency in the feature space, and the feature vectors of different categories have differences in angle and / or direction in the feature space.

[0060] Specifically, a category-adaptive rotation encoding mechanism is used to perform a nonlinear transformation on the feature vectors extracted in the above steps. This transformation process dynamically adjusts the rotation angle and direction according to the category information of the feature vectors, ensuring that feature vectors of the same category are more compact after encoding, and the distinguishability of feature vectors of different categories is significantly improved. The encoded feature vectors can not only better reflect the category features of the image, but also construct clear category boundaries in the high-dimensional feature space.

[0061] Step S204: Classify the image to be classified based on the encoded feature vector to obtain the classification result.

[0062] Specifically, the encoded feature vector is input into a classifier. The model learned by the classifier performs a final category determination on the image to be classified, thus obtaining the classification result. The classifier can use the Softmax function or other multi-class decision methods. Based on the spatial distribution of the feature vector after adaptive rotation encoding, it outputs the probability of each category, thereby determining the image's classification.

[0063] Existing image classification techniques typically extract features from an image and then classify the image based on those features. However, they lack direct class supervision in the feature space (implicitly, they are indirectly supervised based on the rules of composite function differentiation), which can affect classification accuracy due to insufficient inter-class discrimination. In contrast, the proposed solution introduces class-adaptive rotational encoding. This non-linear encoding essentially provides secondary supervision of the classification results, optimizing the distribution of feature vectors in the feature space and effectively improving classification accuracy.

[0064] Specifically, most existing classification networks still face two core problems:

[0065] 1. Insufficient inter-class discrimination: The feature distributions of different categories may overlap significantly, affecting classification accuracy;

[0066] 2. Class information is not involved in the feature encoding process: Current feature extraction networks often use a uniform encoding method and fail to dynamically adjust the feature space structure according to class information.

[0067] In natural language processing and computer vision, Rotary Positional Embedding (RoPE) is widely used to incorporate the relative positional information of elements in a sequence. This method achieves an encoding scheme with relative semantic structure by performing sine and cosine rotations on feature vectors along different frequency dimensions.

[0068] Inspired by RoPE, the inventors of this application propose a novel encoding strategy: Class-Adaptive Rotary Feature Encoding (CARFE). This method constructs a class-specific rotational encoding structure, enabling different classes to have distinguishable rotational patterns in the feature space, thereby improving the model's inter-class discrimination ability. The scheme in this application can improve the inter-class separability of image classification models in large-scale class scenarios; it introduces class information during the feature encoding stage, giving features "class awareness" in space; it proposes a class-adaptive rotational encoding mechanism suitable for structures such as CNNs and Transformers, and has periodic encoding capabilities, scalable to massive class tasks.

[0069] Specifically, the classification features incorporate category-related relative positional information, breaking the category independence of traditional feature spaces and explicitly enhancing the separability of category features. This system includes the following modules: image and label data input module, image feature extraction module, image feature rotation encoding module, and image feature classification module (system block diagram as shown). Figure 3 (As shown), the following sections will introduce them separately.

[0070] The image and label data input module is used to receive image data. During the training phase, it also includes label information (gt). The category information in the label is used to supervise the image classification features for classification.

[0071] The image feature extraction module extracts features from the input image data. The model for this module can be any CNN network (such as VGG, ResNet, RegNet, etc.) or a ViT architecture network (such as SwinTransformer, etc.). This module's model extracts a feature vector of dimension d from the input image data.

[0072] In the specific implementation process, feature extraction is performed on the above-mentioned image to be classified to obtain a feature vector. This can be achieved through the following steps: The following formula is used to extract features from the above-mentioned image to be classified:

[0073] Rot C =FC1(X d (1-1),

[0074] Where X represents the input image to be classified, FC1 represents a linear mapping layer that maps the image to be classified into a feature vector of dimension d and outputs a feature vector Rot of class size C. C .

[0075] In this scheme, the backpropagation algorithm is used to continuously update the model during training, learning how to map the original feature vector of the image to a feature space that is more conducive to subsequent class adaptive rotation encoding. The feature vector in this space can more accurately reflect the true class of the image, and the inter-class discrimination is further enhanced through rotation encoding.

[0076] Specifically, when processing the images to be classified, the linear mapping layer FC1 is first used. This is a key neural network layer responsible for converting the image data into a compact feature representation. Here, X... d It refers to the high-dimensional features obtained after an image undergoes a series of processing steps, such as convolution and pooling.

[0077] Equation (1-1) defines the transformation process from image features to category feature vectors. The weights of FC1 are learned through a large amount of training data. It can effectively capture visual features within the image and transform them into feature vectors that can be used for subsequent category adaptive rotation encoding.

[0078] After FC1 mapping, the original image features X d It is transformed into a new feature vector Rot C This vector not only contains image information but also implicit category information, and its dimension is fixed at (d). The output size is the number of categories C, providing an important input for subsequent category-adaptive rotation encoding.

[0079] In some embodiments, a category-adaptive encoding mechanism is used to rotate and encode the feature vectors to obtain encoded feature vectors. Specifically, this can be achieved through the following steps: extracting the maximum value from the feature vectors; calculating the rotation angle of the feature vector with the maximum value based on the RoPE algorithm; and rotating and encoding the feature vector with the maximum value based on the rotation angle to obtain encoded feature vectors.

[0080] In this scheme, by extracting the maximum value and calculating the rotation angle based on the RoPE algorithm, a layout that is more conducive to classification can be created in the feature space. Feature vectors of the same category are more compact after rotation, while feature vectors of different categories can produce significant differences in angle and direction, which significantly improves the class discrimination of feature vectors.

[0081] The feature vectors extracted from the image are analyzed, and the Max() function is used to determine the maximum value. This step is crucial for subsequent class-adaptive rotation encoding because the maximum value and its corresponding index help locate the most significant class features in the feature vectors, providing key information for the subsequent rotation encoding process.

[0082] After determining the maximum value in the feature vector, the RoPE algorithm is used to calculate the rotation angle. The core of the RoPE algorithm is to assign frequency-based positional information to each dimension of the feature vector using sine and cosine functions, thereby introducing relative positional relationships within the feature vector. In this scenario, by using the category information of the maximum value, a specific rotation matrix can be constructed for the feature vector, ensuring that the rotation encoding process can adapt to a specific category and further enhancing the discriminative power of the features.

[0083] With the rotation angle determined, the feature vectors can be rotated and encoded using a rotation matrix. This process adjusts the position of the feature vectors in high-dimensional space through nonlinear transformation, thereby enhancing the consistency of feature vectors of the same category and distinguishing feature vectors of different categories by angle and direction, providing higher-quality feature representations for subsequent classification tasks.

[0084] In the specific implementation process, extracting the maximum value from the above feature vector can be achieved through the following steps: The maximum value of the elements in the above feature vector is extracted using the following formula:

[0085] ω j j = Max(Rot C ), 0≤j≤C-1(1-2),

[0086] The Max() function takes the maximum value of the input data, ω. j ω in j j Represents the above eigenvector Rot C The maximum value in, ω j In ,j represents the index of the maximum value mentioned above. During training, j is trained using regression supervision to fit the target class.

[0087] This approach identifies the elements in the feature vector that are most relevant to the category, enabling the model to focus on category-sensitive features and improving the targeting of feature selection. By determining the maximum value and its index, more precise supervised training can be performed, especially using regression-based supervision. This enhances the model's ability to capture category features, accelerates training convergence, and ensures higher training accuracy.

[0088] Specifically, we use the Max() function to process the feature vector extracted from the image. The goal of this operation is to find the element in the feature vector that is most relevant (maximum value) to a certain category and its index.

[0089] Equation (1-2) illustrates how to extract a feature value corresponding to a category from a feature vector and determine the index of that feature value, i.e., the position of the category in the feature vector. Here, ω... j "J" refers to the maximum value itself, while "j" is the index value corresponding to the maximum value, i.e., the position of the maximum value, which is the focus of subsequent supervised training.

[0090] In some embodiments, the rotation angle of the aforementioned eigenvector with the maximum value is calculated based on the RoPE algorithm, specifically by using the following formula to calculate the aforementioned rotation angle:

[0091]

[0092] Where, θ i,j This represents the rotation angle of the j-th category.

[0093] In this scheme, after extracting the maximum value, the rotation angle is calculated according to formula (1-3), and this angle will be used for subsequent rotation encoding operations. In this formula, i represents the rotation frequency index after grouping in the feature vector, and d is the dimension of the feature vector. By combining the maximum value with an exponential decay function, a specific rotation angle can be constructed for each rotation frequency group, ensuring the class adaptability of rotation encoding.

[0094] The above steps determine the maximum value in the feature vector, which is the starting point for category-adaptive rotation coding and provides information on the most likely category to which the image belongs.

[0095] Using formula (1-3), the rotation angle for different rotation frequency indices is calculated based on the maximum value and the dimensionality information of the eigenvector. Here, θ i,j The calculation takes into account the influence of the exponential decay function, ensuring that the rotation angle exhibits regular changes in different frequency dimensions of the feature vector, thus optimizing the distribution of the feature vector in space.

[0096] After obtaining the rotation angles, these angles are used to perform rotation encoding on the feature vectors. This encoding process adjusts the orientation of the feature vectors according to the category information, ensuring the inherent consistency of feature vectors of the same category, while increasing the angular differences between feature vectors of different categories, thereby improving inter-class discrimination.

[0097] In the specific implementation process, the eigenvector of the maximum value is rotated and encoded based on the above rotation angle to obtain the encoded eigenvector. This can be achieved through the following steps: The eigenvector of the maximum value is rotated using the following formula:

[0098]

[0099] in, Let represent the rotation matrix of the j-th category; rotation encoding is performed based on the rotation matrix using the following formula:

[0100]

[0101] in, X represents d Data after processing by the linear mapping layer FC2 This represents the feature vector after the above encoding.

[0102] In this scheme, a clear inter-class boundary can be constructed in the feature space through a class-adaptive rotation encoding mechanism. The design of the rotation matrix ensures the inherent convergence of feature vectors of the same class and the angular separation of feature vectors of different classes, thereby optimizing the layout of feature vectors in space and improving classification accuracy. The combined application of formulas (1-4) and (1-5) achieves fine control over the angle and direction of feature vectors. This control strategy ensures that even when the feature vector dimension is high, the position of the feature vectors can be accurately adjusted to match the feature space layout of a specific class, enhancing classification robustness. Through fine angle adjustment in the rotation encoding process, the model can learn more complex and accurate feature representations, and even when facing new data or unseen classes, it can make accurate classification decisions based on the optimized feature distribution, improving the model's generalization ability.

[0103] Specifically, the rotation moments constructed above are used to perform rotation encoding on the feature vectors processed by the linear mapping layer FC2, resulting in encoded feature vectors. This operation, through rotation and weighted combination, performs a non-linear transformation on the feature vectors in space, ensuring enhanced consistency of feature vectors of the same class and angular differences between feature vectors of different classes.

[0104] An image feature rotation encoding module is included, featuring class-adaptive rotation encoding capabilities. It introduces class-related learnable fundamental frequencies (Rot).C Its dimension is the number of categories C, as shown in Formula 1-1. During training initialization, 1 / C is used to assign a value to the linear mapping layer FC1. This module assigns the rotational fundamental frequency ω to the input image target of category j using Formula 1-2. j During training, the objective function (1-7) is optimized to ensure that the class j obtained by this module matches the true target class. Then, using formula (1-3), the obtained base rotation frequency ω is utilized... j For features of dimension d, a rotation angle was constructed. Finally, a rotation matrix similar to RoPE was designed, as shown in Equation 1-4. The feature vectors extracted by the image feature extraction module were re-encoded using this rotation matrix, making features of the same category more consistent and features of different categories exhibiting significant angle / direction differences, thereby improving inter-class separability.

[0105] Rot C =FC1(X d (1-1);

[0106] In Equation 1-1, FC1 is a linear mapping layer that maps features with input dimension d to outputs with output dimension C of class size, i.e., mapping from low-dimensional space to high-dimensional space. The variable (x) represents the feature vector output by the image feature extraction module.

[0107] ω j j = Max(Rot C ), 0≤j≤C-1(1-2);

[0108] In Formula 1-2, the Max() function takes the maximum value from the input data and returns the maximum value along with its index. ω j It is the Rot that is obtained. C The maximum value in the vector, where j is the category index of the corresponding maximum value.

[0109]

[0110] In formulas 1-3, θ i,j This represents the method for calculating the rotation angle of the j-th category, which references the RoPE algorithm. For a d-dimensional feature vector, there are d / 2 rotation angles, meaning that a d-dimensional feature is grouped into d / 2 rotation frequencies.

[0111]

[0112] In formulas 1-4, This represents the rotation matrix for the j-th category.

[0113]

[0114] In formulas 1-5, This represents the image features X extracted by the image feature extraction module. d The data processed by the linear mapping layer FC2 still has a dimension of d. From the rotation matrix defined in Equation 1-4, we can see that the eigenvectors... Each pair of feature values ​​is grouped together for rotational feature encoding. This divides the d-dimensional features into d / 2 groups, resulting in d / 2 combinations of different frequencies.

[0115]

[0116] Equations 1-6 are the expressions for the Cross-Entropy loss in multi-class classification. i It is the one-hot encoding of the category label of the input image, X C This is the target category vector output by the feature classification module, with a dimension equal to the number of categories, C. The `Func()` function represents the category probability calculation function, which can be either a softmax function or a sigmoid function. FC3 is a linear mapping layer, L... CE It is cross-entropy loss.

[0117]

[0118] Formula 1-7 is the training objective function for the result category j obtained from Formula 1-2, ensuring that the category j obtained through inference is consistent with the true target category y. (x) refers to the result of the feature vector output by the image feature extraction module after certain processing (such as through the (FC1) layer). SmoothL1 is the loss function, called Huber loss, which combines the advantages of mean squared error (MSE) and mean absolute error (MAE). rot The rotational fundamental frequency loss is used to optimize the predicted rotational fundamental frequencies to ensure that they reflect the true category information of the input image as accurately as possible.

[0119] L total =α*L CE +(1-α)*L rot ,0<α<1 (1-8);

[0120] Equations 1-8 represent the overall objective optimization function of the training system, which consists of the classification loss (1-6) and the class rotation fundamental frequency loss (1-7), with α modulating the contribution of both. total α is the overall loss function, and α is a hyperparameter used to control the weight of the two types of loss functions.

[0121] Specifically, the image classification model with adaptive category rotation encoding is trained, and the data processing flow during training is as follows: Figure 4 As shown, the image data and label information of the training dataset are fed into the training system. The label information includes the category information of the target image. Here, it is assumed that the training dataset has C categories. During training, the category label numbers will be converted into one-hot encoding.

[0122] Next, the image feature extraction module extracts features from the input image, obtaining a feature vector X of dimension d. d The image feature extraction module corresponds to the Backbone in the data flow diagram, and the extracted features correspond to the Feature module.

[0123] The extracted image features are input into the image feature rotation encoding module. This module uses a linear mapping layer FC1 to encode the input image features X. d Transformation according to Formula 1-1 yields the fundamental frequency vector Rot. C It has C values, and the fundamental rotational frequency ω of the input image is obtained by processing it using formula 1-2. j Then, the rotation angle of the image vector at this point is obtained using Equation 1-3, and the rotation matrix is ​​given by Equation 1-4. Finally, the rotation matrix is ​​used to transform the feature vector after the linear mapping layer FC2. Rotation encoding is performed. The feature vector after rotation encoding according to formula 1-5 is output to the image feature classification module.

[0124] Formulas 1-2 simultaneously yield the target base frequency category j of the input image. During training, Formulas 1-7 are used as the objective function for optimization, so that the target category j is optimized in the same direction as the classification objective.

[0125] The image feature classification module corresponds to the linear mapping layer FC3 in the data flow diagram. It maps the input image rotation feature vector into a vector for classification. During training, Equations 1-6 are used as the optimization objective function for classification.

[0126] The final objective function of the training system is expressed by Equation 1-8, and the hyperparameter α is used to modulate the contribution of classification loss and class rotation base frequency loss to the system.

[0127] In another implementation, an image classification model with adaptive category rotation encoding is deployed for inference. Image data is input into the system through the image and label data input module. The data processing flow of the image feature extraction module, image feature rotation encoding module, and image feature classification module is roughly the same as that in the first embodiment. The difference is that in the image feature rotation encoding module, the rotation base frequency category j obtained by formula 1-2 is no longer optimized; and in the image feature classification module, the classification vector obtained is directly output as the final category probability after being processed by the Func function.

[0128] In some embodiments, classifying the image to be classified based on the encoded feature vector to obtain a classification result includes: mapping the encoded feature vector onto a classification space, and calculating the probability value of each category corresponding to the encoded feature vector according to a classification function; and determining the category with the highest probability value as the classification result of the image to be classified.

[0129] In this scheme, the encoded feature vectors are mapped to a dedicated classification space designed to maximize the discriminative power between feature vectors of different classes. Within this space, a classification function (such as Softmax) is applied to further process the feature vectors, transforming them into probability distribution vectors, where each element represents the probability that the feature vector belongs to the corresponding class. The class with the highest probability value is selected as the final classification result. This decision-making process relies on the work done in the encoding and optimization stages, because the accuracy of the classification result can only be ensured when the classification space is properly laid out and the classification function accurately reflects the class probabilities.

[0130] After encoding and optimizing the feature vectors, the encoded feature vectors are mapped to a specially designed classification space. In this space, the feature vectors of different classes have been optimized in layout for more efficient classification. Next, a classification function (such as Softmax or Sigmoid) is used to calculate the probability value corresponding to each class. This process is achieved by using a linear mapping layer FC3 (as shown in Equation 1-6) to map the feature vectors to a probability distribution of the classes, where each element represents the probability of a class.

[0131] The feature classification module maps the feature vectors output by the image feature rotation encoding module into target category vectors during the training phase. The objective function used to supervise the training of this module during the training phase can be the cross-entropy loss function, as shown in Equation 1-6.

[0132] In summary, the inventors of this application designed a rotation encoder based on category information, directly integrating category information into the feature rotation matrix. A category-adaptive rotation feature mapping method was constructed, allowing different category features to naturally generate relative displacement and angular separation in space. The rotation matrix has periodicity and frequency control capabilities, scalable to millions of categories, and the method is compatible with CNN networks and existing Transformer architectures without requiring excessive additional parameters or complex calculations. This patent introduces category-related relative position information into classification features, breaking the category independence of traditional feature spaces and explicitly enhancing the separability of category features. Thus, the following is achieved:

[0133] 1. The inter-class features are naturally separated, increasing the class discriminative power.

[0134] 2. Keep intra-class features compact to improve classification robustness.

[0135] The encoding method has good periodicity and frequency characteristics, and can adapt to training tasks with millions of categories.

[0136] This application also provides an image classification apparatus. It should be noted that the image classification apparatus of this application can be used to execute the image classification method provided in this application. This apparatus is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0137] The image classification apparatus provided in the embodiments of this application will be described below.

[0138] Figure 5 is a structural block diagram of an image classification apparatus according to an embodiment of this application. Figure 5 As shown, the device includes:

[0139] Acquisition unit 10 is used to acquire the image to be classified;

[0140] Extraction unit 20 is used to extract features from the above-mentioned image to be classified to obtain a feature vector;

[0141] The adaptive encoding unit 30 is used to perform rotation encoding on the above feature vectors using a category-adaptive encoding mechanism to obtain encoded feature vectors, so that the above feature vectors of the same category have consistency in the feature space, and the above feature vectors of different categories have differences in angle and / or direction in the above feature space.

[0142] The classification unit 40 is used to classify the image to be classified based on the encoded feature vector to obtain the classification result.

[0143] Existing image classification techniques typically extract features from an image and then classify the image based on those features. However, they lack direct class supervision in the feature space (implicitly, they are indirectly supervised based on the rules of composite function differentiation), which can affect classification accuracy due to insufficient inter-class discrimination. In contrast, the proposed solution introduces class-adaptive rotational encoding. This non-linear encoding essentially provides secondary supervision of the classification results, optimizing the distribution of feature vectors in the feature space and effectively improving classification accuracy.

[0144] In the specific implementation process, the extraction unit includes a first extraction module, which is used to extract features from the above-mentioned image to be classified using the following formula:

[0145] Rot C =FC1(X d (1-1),

[0146] Where X represents the input image to be classified, FC1 represents a linear mapping layer that maps the image to be classified into a feature vector of dimension d and outputs a feature vector Rot of class size C. C .

[0147] In this scheme, the backpropagation algorithm is used to continuously update the model during training, learning how to map the original feature vector of the image to a feature space that is more conducive to subsequent class adaptive rotation encoding. The feature vector in this space can more accurately reflect the true class of the image, and the inter-class discrimination is further enhanced through rotation encoding.

[0148] In some embodiments, the adaptive encoding unit includes a second extraction module, a calculation module, and an encoding module. The second extraction module is used to extract the maximum value in the feature vector. The calculation module is used to calculate the rotation angle of the feature vector of the maximum value based on the RoPE algorithm. The encoding module is used to perform rotation encoding on the feature vector of the maximum value based on the rotation angle to obtain the encoded feature vector.

[0149] In this scheme, by extracting the maximum value and calculating the rotation angle based on the RoPE algorithm, a layout that is more conducive to classification can be created in the feature space. Feature vectors of the same category are more compact after rotation, while feature vectors of different categories can produce significant differences in angle and direction, which significantly improves the class discrimination of feature vectors.

[0150] In the specific implementation, the second extraction module includes an extraction sub-module, which is used to extract the maximum value of the elements in the above feature vector using the following formula:

[0151] ωj j = Max(Rot C ), 0≤j≤C-1(1-2),

[0152] The Max() function takes the maximum value of the input data, ω. j ω in j j Represents the above eigenvector Rot C The maximum value in, ω j In ,j represents the index of the maximum value mentioned above. During training, j is trained using regression supervision to fit the target class.

[0153] This approach identifies the elements in the feature vector that are most relevant to the category, enabling the model to focus on category-sensitive features and improving the targeting of feature selection. By determining the maximum value and its index, more precise supervised training can be performed, especially using regression-based supervision. This enhances the model's ability to capture category features, accelerates training convergence, and ensures higher training accuracy.

[0154] In some embodiments, the calculation module includes a calculation submodule, which calculates the aforementioned rotation angle using the following formula:

[0155]

[0156] Where, θ i,j This represents the rotation angle of the j-th category.

[0157] In this scheme, after extracting the maximum value, the rotation angle is calculated according to formula (1-3), and this angle will be used for subsequent rotation encoding operations. In this formula, i represents the rotation frequency index after grouping in the feature vector, and d is the dimension of the feature vector. By combining the maximum value with an exponential decay function, a specific rotation angle can be constructed for each rotation frequency group, ensuring the class adaptability of rotation encoding.

[0158] In the specific implementation, the encoding module includes a rotation submodule and an encoding submodule. The rotation submodule is used to rotate the above eigenvector of the maximum value using the following formula:

[0159]

[0160] in, Let represent the rotation matrix for the j-th category; the encoding submodule is used to perform rotation encoding based on the rotation matrix using the following formula:

[0161]

[0162] in, X represents d Data after processing by the linear mapping layer FC2 This represents the feature vector after the above encoding.

[0163] In this scheme, a clear inter-class boundary can be constructed in the feature space through a class-adaptive rotation encoding mechanism. The design of the rotation matrix ensures the inherent convergence of feature vectors of the same class and the angular separation of feature vectors of different classes, thereby optimizing the layout of feature vectors in space and improving classification accuracy. The combined application of formulas (1-4) and (1-5) achieves fine control over the angle and direction of feature vectors. This control strategy ensures that even when the feature vector dimension is high, the position of the feature vectors can be accurately adjusted to match the feature space layout of a specific class, enhancing classification robustness. Through fine angle adjustment in the rotation encoding process, the model can learn more complex and accurate feature representations, and even when facing new data or unseen classes, it can make accurate classification decisions based on the optimized feature distribution, improving the model's generalization ability.

[0164] In some embodiments, the classification unit includes a mapping module and a determination module. The mapping module is used to map the encoded feature vector onto the classification space and calculate the probability value of each category corresponding to the encoded feature vector according to the classification function. The determination module is used to determine the category with the highest probability value as the classification result of the image to be classified.

[0165] In this scheme, the encoded feature vectors are mapped to a dedicated classification space designed to maximize the discriminative power between feature vectors of different classes. Within this space, a classification function (such as Softmax) is applied to further process the feature vectors, transforming them into probability distribution vectors, where each element represents the probability that the feature vector belongs to the corresponding class. The class with the highest probability value is selected as the final classification result. This decision-making process relies on the work done in the encoding and optimization stages, because the accuracy of the classification result can only be ensured when the classification space is properly laid out and the classification function accurately reflects the class probabilities.

[0166] The image classification device described above includes a processor and a memory. The acquisition unit, extraction unit, calculation unit, processing unit, and classification unit are all stored as program units in the memory. The processor executes these program units stored in the memory to achieve their respective functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0167] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured. By adjusting the kernel parameters, category information is incorporated into the feature encoding process of the classification network, improving the distinguishability between categories and thus further enhancing the accuracy of the classification results.

[0168] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0169] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the image classification method.

[0170] This invention provides a processor for running a program, wherein the program executes the image classification method during runtime.

[0171] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements at least the steps of an image classification method. The device described herein can be a server, PC, PAD, mobile phone, etc.

[0172] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform a program that initializes a classification method step having at least images.

[0173] This application also provides an image classification system, including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any of the above-described image classification methods.

[0174] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0175] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0176] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0177] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0178] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0179] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0180] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0181] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0182] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0183] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0184] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image classification method, characterized in that, include: Obtain the image to be classified; Feature extraction is performed on the image to be classified to obtain a feature vector; The feature vectors are rotated and encoded using a category-adaptive encoding mechanism to obtain encoded feature vectors, so that feature vectors of the same category have consistency in the feature space, and feature vectors of different categories have differences in angle and / or direction in the feature space. The image to be classified is classified based on the encoded feature vector to obtain the classification result; Feature extraction is performed on the image to be classified to obtain a feature vector, including: feature extraction of the image to be classified using the following formula: ,in, This represents the input image to be classified. This represents a linear mapping layer that maps the image to be classified to a linear dimension. Features, output category size is The feature vector ; The feature vector is rotated and encoded using a category-adaptive encoding mechanism to obtain the encoded feature vector, including: extracting the maximum value in the feature vector; calculating the rotation angle of the feature vector with the maximum value based on the RoPE algorithm; and rotating and encoding the feature vector with the maximum value based on the rotation angle to obtain the encoded feature vector. Extracting the maximum value from the feature vector includes: extracting the maximum value of each element in the feature vector using the following formula: in, The function is an operation that takes the maximum value of the input data. In Represents the feature vector The maximum value in, In The index value representing the maximum value, during the training process, The system is trained using a regression-supervised approach to fit the target class. Based on the RoPE algorithm, the rotation angle of the eigenvector with the maximum value is calculated, including: calculating the rotation angle using the following formula: in, Indicates the first The rotation angles of each category; Based on the rotation angle, the feature vector of the maximum value is rotated and encoded to obtain the encoded feature vector, including: rotating the feature vector of the maximum value using the following formula: in, Let represent the rotation matrix of the j-th category; rotation encoding is performed based on the rotation matrix using the following formula: ),in, express After linear mapping layer The processed data, This represents the encoded feature vector.

2. The method according to claim 1, characterized in that, The image to be classified is classified based on the encoded feature vector to obtain the classification result, including: The encoded feature vector is mapped onto the classification space, and the probability value of each category corresponding to the encoded feature vector is calculated according to the classification function. The category with the highest probability value is determined as the classification result of the image to be classified.

3. An image classification device, characterized in that, include: The acquisition unit is used to acquire the image to be classified. An extraction unit is used to extract features from the image to be classified to obtain a feature vector. An adaptive encoding unit is used to perform rotation encoding on the feature vector using a category-adaptive encoding mechanism to obtain an encoded feature vector, so that the feature vectors of the same category have consistency in the feature space, and the feature vectors of different categories have differences in angle and / or direction in the feature space. A classification unit is used to classify the image to be classified based on the encoded feature vector to obtain a classification result; The extraction unit includes a first extraction module, which is used to extract features from the image to be classified using the following formula: in, This represents the input image to be classified. This represents a linear mapping layer that maps the image to be classified to a linear dimension. Features, output class size is The feature vector ; The adaptive encoding unit includes a second extraction module, a calculation module, and an encoding module. The second extraction module is used to extract the maximum value in the feature vector; the calculation module is used to calculate the rotation angle of the feature vector of the maximum value based on the RoPE algorithm; and the encoding module is used to perform rotation encoding on the feature vector of the maximum value based on the rotation angle to obtain the encoded feature vector. The second extraction module includes an extraction submodule, which is used to extract the maximum value of the elements in the feature vector using the following formula: in, The function is an operation that takes the maximum value of the input data. In Represents the feature vector The maximum value in, In The index value representing the maximum value, during the training process, The system is trained using a regression-supervised approach to fit the target class. The calculation module includes a calculation submodule, which calculates the rotation angle using the following formula: in, Indicates the first The rotation angles of each category; The encoding module includes a rotation submodule and an encoding submodule. The rotation submodule is used to rotate the eigenvector of the maximum value using the following formula: in, Let represent the rotation matrix for the j-th category; the encoding submodule is used to perform rotation encoding based on the rotation matrix using the following formula: ),in, express After linear mapping layer The processed data, This represents the encoded feature vector.

4. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the image classification method according to any one of claims 1 or 2.

5. An image classification system, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including a method for performing an image classification method according to any one of claims 1 or 2.

Images