Hierarchical image classification method and system based on CA-SwinT

Abstract

The invention discloses a hierarchical image classification method and system based on CA-SwinT, and belongs to the technical field of artificial intelligence. The method comprises the following steps: S1, acquiring an original image, and preprocessing the original image; s2, performing feature extraction on the image through a SwinT module; s3, fully fusing the extracted image features and classification label information by using a double-cross attention mechanism, and constructing fine mapping between the image and a hierarchical label; s4, according to the fine mapping in the S3, generating a corresponding label through a label generation module, and updating classification label information; the S2 comprises the steps of slicing and blocking processing, linear embedding processing, two-stage Swin module processing, block merging processing, two-stage Swin processing and linear embedding processing. According to the method, rich feature information of the image can be effectively extracted from different dimension spaces, and the multi-level label of the image can be more efficiently identified.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a hierarchical image classification method and system based on CA-SwinT. Background Technology

[0002] Hierarchical image classification tasks target multi-labeled images with hierarchical structures, playing a crucial role in many practical applications. To facilitate easy product searching for consumers, e-commerce platforms have built finely segmented product classification systems. In these systems, broader categories (e.g., "electronic products") extend into more detailed subcategories (e.g., "laptops"). In hierarchical image classification for design patents, an image labeled as the subcategory "bed" can also be classified under the broader category "furniture." In animal classification, the Leonberg dog can be subdivided under the category "dog" or broadly categorized under the parent category "animal." All these examples demonstrate the hierarchical relationships between labels.

[0003] In recent years, with significant breakthroughs in artificial intelligence technology, image classification and segmentation have received unprecedented attention. However, due to the more complex hierarchical dependency structure of image labels in the real world, hierarchical image classification tasks face even greater challenges. This is not only because the task difficulty far exceeds that of traditional single-label classification, but also because it requires a deep understanding and precise handling of the hierarchical relationships between multiple labels.

[0004] Previous research on hierarchical image classification has largely employed traditional convolutional neural network (CNN) methods. While these methods have achieved good results, they primarily focus on constructing and analyzing the correlation between images and labels, failing to fully consider the structured relationships between labels, and requiring multiple classifiers to complete multi-label hierarchical classification. To address this deficiency, subsequent research introduced a CNN-RNN-based approach. Although CNN-RNN methods have improved in modeling label relevance, they still rely on global image features and global correlations between labels, limiting feature description capabilities and a deeper understanding of potential label dependencies.

[0005] With the successful application of attention mechanisms in various fields, there is increasing interest in using these techniques to solve hierarchical image classification problems. Furthermore, attention-based hierarchical image classification methods have achieved better recognition results. However, despite these advancements, they still have shortcomings in fully capturing the correlation between images and labels and in terms of accuracy. Summary of the Invention

[0006] The present invention aims to improve the accuracy of hierarchical classification by using a lightweight framework, and effectively solves the shortcomings of the above-mentioned technologies.

[0007] Therefore, the purpose of this invention is to provide a hierarchical image classification method and system based on CA-SwinT. This method introduces the main mechanisms of SwinTransformer and double cross-attention mechanism. The SwinT module based on SwinTransformer effectively reduces model parameters, reduces model complexity, and extracts rich and fine-grained feature information of the image. At the same time, the double cross-attention mechanism can better construct a more comprehensive feature mapping relationship between the image and the label, thereby efficiently identifying multi-level labels of the image.

[0008] To solve the above-mentioned technical problems, the present invention is implemented as follows: This invention provides a hierarchical image classification method based on CA-SwinT, the method comprising: S1. Obtain the original image and preprocess the original image; S2. Extract features from the image using the SwinT module; S3. By utilizing the double cross-attention mechanism, the extracted image features and classification label information are fully integrated to construct a fine mapping between images and hierarchical labels; S4. Based on the fine mapping in S3, generate the corresponding tags through the tag generation module and update the category tag information.

[0009] In addition, the hierarchical image classification method based on CA-SwinT according to the present invention may also have the following additional technical features: In some implementations, step S2, which involves feature extraction of the image using the SwinT module, includes: S201, Slicing and Blocking Processing: Dividing the image into a series of small blocks; S202, Linear Embedding Processing: Flatten each block and pass it through a fully connected linear embedding layer to convert the pixel value of each block into a high-dimensional feature vector; S203, Two-level Swing module processing: The processing results of S202 are continuously input into the two-level Swing module for feature extraction; S204, Block merging processing: The image height and width are halved and the number of channels are doubled by downsampling the image; S205, Two-stage Swing processing: The processing results of S204 are continuously input into the two-stage Swing module for feature extraction; S206, Linear Embedding Processing: Perform a linear transformation on the processing result of S205.

[0010] In some implementations, the processing steps of each Swin module in steps S203 and S205 include: Layer normalization: to achieve data standardization; Multi-head self-attention processing: Divide all pixels of the corresponding input image into multiple windows, and calculate the correlation between each pixel and other pixels within each window to capture local features; The residual connection process involves adding the input features directly to the self-attention processing result from the previous step to form a residual connection. Multilayer perceptron processing: Features are transformed and combined in a complex manner through fully connected layers and nonlinear activation functions to improve the feature representation capability of the model; Moving window multi-head self-attention processing: Divide all pixels of the corresponding input image into multiple windows. When dividing the windows, shift the divided windows to the right and down by several pixels at the same time. Calculate the correlation between each pixel and other pixels within each window, thereby realizing information interaction between different windows. The residual connection process involves adding the input features directly to the self-attention processing result from the previous step to form a residual connection. Multilayer perceptron processing: By using fully connected layers and nonlinear activation functions to perform complex transformations and combinations on features, the feature representation capability of the model is improved.

[0011] In some implementations, step S3 utilizes a dual cross-attention mechanism to fully fuse extracted image features and label information, constructing a fine mapping between the image and hierarchical labels. S301. Use the features of each block extracted by the SwinT module as keys and values, use the current classification label information as queries, perform multi-head attention mechanism processing, merge the processing results and then perform layer normalization processing to obtain the output result of the first layer cross-attention mechanism. S302. Using the features of each block extracted by the SwinT module as keys and values, and the results of S301 as queries, perform multi-head attention mechanism processing, merge the processing results and then perform layer normalization processing to obtain the output results of the second layer cross-attention mechanism.

[0012] In some implementations, the content of generating the corresponding tags through the tag generation module includes: S401, Multilayer Perceptron Processing: Processes input data through two fully connected layers to improve feature representation capabilities; S402. Classification Processing: Confidence is calculated using the softmax function to achieve classification.

[0013] This invention also provides a hierarchical image classification system based on CA-SwinT, the system comprising: The preprocessing module is used to receive the original image and preprocess the original image; The feature extraction module is used to extract features from images using the SwinT module; The dual cross-attention module is used to fully integrate the extracted image features and classification label information using the dual cross-attention mechanism, and to construct a fine mapping between the image and the hierarchical label. The tag generation module is used to generate corresponding tags based on the fine mapping obtained by the dual cross-attention module and update the classification tag information.

[0014] In addition, the hierarchical image classification system based on CA-SwinT according to the present invention may also have the following additional technical features: In some embodiments, the feature extraction module includes the following components connected in sequence: Slicing and partitioning layer: Used to divide an image into a series of small blocks; Linear embedding layer: used to flatten each block and pass it through a fully connected linear embedding layer to transform the pixel values ​​of each block into a high-dimensional feature vector; Two-level Swin module: used to extract features from the processing results of the previous linear embedding layer; Block merging module: used to halve the height and width of an image and double the number of channels by downsampling; Two-level Swin module: used to extract features from the processing results of the previous block merging module; Linear Embedding Module: Used to perform linear transformations on the processing results of the previous one or two Swin modules.

[0015] In some implementations, each level of the Swin module includes sequentially connected: Layer normalization layer: Used to standardize input data; Multi-head self-attention module: It is used to divide all the pixels of the corresponding input image into multiple windows, calculate the correlation between each pixel and other pixels within each window, and thus capture local features; Residual connection + layer normalization module: used to directly add the input features to the self-attention processing result of the previous module to form a residual connection, and then perform layer normalization processing; Multilayer perceptron: used to perform complex transformations and combinations of features through fully connected layers and non-linear activation functions, thereby improving the feature representation capability of the model; The moving window multi-head self-attention module is used to divide all the pixels of the corresponding input image into multiple windows. When dividing the windows, the divided windows are simultaneously shifted to the right and down by several pixels. Within each window, the correlation between each pixel and other pixels is calculated, thereby realizing information interaction between different windows. Residual connection + layer normalization module: used to directly add the input features to the self-attention processing result of the previous module to form a residual connection, and then perform layer normalization processing; Multilayer perceptron: Used to perform complex transformations and combinations of features through fully connected layers and non-linear activation functions, thereby improving the feature representation capability of the model.

[0016] In some of these implementations, the dual cross-concern module includes: The first cross-attention mechanism layer: uses the features of each block extracted by the SwinT module as keys and values, the current classification label information as queries, performs multi-head attention mechanism processing, merges the processing results and performs layer normalization processing to obtain the output result of the first layer cross-attention mechanism. The second cross-focus mechanism layer is used to take the features of each block extracted by the SwinT module as keys and values, take the output of the first-layer cross-focus mechanism as a query, perform multi-head focus mechanism processing, merge the processing results and then perform layer normalization processing to obtain the output of the second-layer cross-focus mechanism.

[0017] In some embodiments, the label generation module includes: Multilayer perceptron: Used to process input data through two fully connected layers to improve feature representation capabilities; Classification module: Used to calculate confidence scores using the softmax function, thereby achieving classification.

[0018] Compared with the prior art, the present invention has at least the following beneficial effects: In this embodiment of the invention, the hierarchical image classification method based on CA-SwinT provided has better feature representation capabilities. This advantage mainly comes from the local window attention mechanism and window movement strategy of the SwinT module, which can better capture the global and detailed information of the image while reducing the number of model parameters and building a lightweight framework. In this embodiment of the invention, the hierarchical image classification method based on CA-SwinT provided has higher accuracy. This advantage mainly comes from the dual cross-attention mechanism, which can effectively exchange information between different modal features and enhance the mapping ability between global and local features of the image and the semantics of the label. This mechanism helps to more accurately represent the mapping between complex image content and semantics, and increase the recognition accuracy of the model. In this embodiment of the invention, the provided CA-SwinT-based hierarchical image classification method has higher efficiency. This advantage mainly comes from the combination of the SwinT module and the dual cross-attention module, which constitutes the entire CA-SwinT model. Specifically, the SwinT module segments the image into small windows using a moving window mechanism and calculates attention mechanisms within local regions. This significantly reduces computational complexity, and the moving window mechanism ensures that features across windows can share information, further improving model efficiency. The dual cross-attention module effectively reduces redundant computation by exchanging feature attention across modalities, focusing only on important feature relationship mappings, thereby improving computational efficiency.

[0019] The CA-SwinT-based hierarchical image classification system of the present invention can realize the content of the CA-SwinT-based hierarchical image classification method, and therefore has at least all the features and advantages of the CA-SwinT-based hierarchical image classification method, which will not be repeated here. Additional aspects and advantages of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] Figure 1 This is a flowchart of a hierarchical image classification method based on CA-SwinT disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of each module used in the hierarchical image classification method based on CA-SwinT disclosed in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific examples and application scenarios.

[0023] In some embodiments of the present invention, a hierarchical image classification method based on CA-SwinT is provided. This method introduces the main mechanisms of SwinTransformer and double cross-attention mechanism. The SwinT module based on SwinTransformer effectively reduces model parameters, reduces model complexity, and extracts rich and fine-grained feature information of the image. At the same time, the double cross-attention mechanism can better construct a more comprehensive feature mapping relationship between the image and the label, thereby efficiently identifying multi-level labels of the image.

[0024] Please see Figure 1 As shown, in some embodiments of the present invention, the method includes the following three main parts: a SwinT module, a dual cross-interest module, and a label generation module. The SwinT module is mainly responsible for extracting image features, the dual cross-interest module is responsible for modeling the association between the image and the label, and the label generation module is used to obtain the final label information of the image. The steps of this classification method include: Step 1: First, preprocess the image data to create a label file (JSON file) corresponding to the image data. This label file is used as input for model training and for calculating model accuracy during optimization.

[0025] Step 2: The image data is first processed by the SwinT module, which is mainly responsible for extracting image features. The module structure is as follows: Figure 2 As shown, it mainly includes several modules such as slicing and segmentation, linear embedding, Swin modules, and segment merging. Its structure includes a slicing and segmentation layer, a linear embedding layer, a two-level Swin module, a segment merging layer, a two-level Swin module, and a linear embedding layer arranged sequentially. Among them: The slice-and-block layer divides the image into a series of small, non-overlapping patches, allowing the model to perform local processing on each patch and effectively reducing the computational complexity of the model. Here, the input image (H×W×3) is divided into several 4×4 image patches, for a total of (H / 4)×(W / 4)×48 image patches, where 48 represents the number of channels.

[0026] In the linear embedding layer, each patch is flattened and passed through a fully connected linear embedding layer, which transforms the pixel values ​​of each patch into a higher-dimensional feature vector to better capture features, adjusting the number of channels to 96.

[0027] The Swin module, derived from the Swin Transformer Block in the SwinTransformer model, achieves feature perception and information fusion from local to global perspectives through a series of carefully designed sub-modules. The structure of the Swin module can be found in [link to Swin module documentation]. Figure 2As shown, it mainly includes, in sequence, a layer normalization module, a window multi-head self-attention module (W-MSA), a residual connection + layer normalization module, a multilayer perceptron, a shifted window multi-head self-attention module (SW-MSA), a residual connection + layer normalization module, and a multilayer perceptron.

[0028] In some embodiments of this application, layer normalization serves to: standardize the input of each layer, making the neural network easier to train, accelerating convergence, and enabling the model to achieve better performance with fewer training epochs; this is typically achieved by normalizing the activation values ​​of each sample to a mean of 0 and a variance of 1, and then applying learnable scaling and translation parameters. The Windowed Multi-Head Self-Attention (W-MSA) module divides all pixels (i.e., the input image) into multiple fixed-size windows, and then calculates the correlation between each pixel and other pixels within each window, thereby capturing local features. The Shifted Windowed Multi-Head Self-Attention (SW-MSA) module shifts the window division to the right and down by x pixels simultaneously, thereby enabling information interaction between different windows. This significantly reduces computational cost. Simultaneously, W-MSA and SW-MSA are performed alternately, allowing the self-attention module to gradually increase its perceptual range from local to global through layer-by-layer stacking. Residual connections: Input features are directly added to the output of the self-attention module to form residual connections. These connections reduce gradient vanishing, promote gradient flow, prevent degradation, and make the model easier to train, thereby optimizing deep models. Adding a normalization layer further stabilizes the training process. Multilayer Perceptron (MLP): Through fully connected layers and non-linear activation functions, MLPs perform complex transformations and combinations of features, enhancing the model's feature representation capabilities.

[0029] The attention mechanism algorithm for each Swin module is as follows:

[0030] The Swin module described above in this invention, by alternating between W-MSA and SW-MSA, can gradually increase the perceptual range in a layer-by-layer stacking process, capturing global features from local features; through window partitioning and shifting operations, the Swin module can significantly reduce the amount of computation and improve model efficiency; the use of layer normalization and residual connections makes the Swin module more stable and easier to optimize during training.

[0031] The slicing and merging layer primarily downsamples the image to reduce the size of the feature map and increase the number of channels. In this embodiment, the height and width are halved, and the number of channels is doubled. Here, the image size becomes (H / 8)×(W / 8)×192. Afterward, it undergoes two Swin modules and a linear transformation. The feature representation obtained after processing by the entire SwinT module is as follows:

[0032] The resulting image feature size is (H / 8)×(W / 8)× .

[0033] In this embodiment, the two-stage Swing processing after the block merging layer aims to further capture high-level features in the image, providing a more effective feature representation for subsequent classification. A further linear embedding process is then performed, primarily for channel number transformation.

[0034] Step 3: Introduce a dual cross-attention mechanism, using the text label information input stream to guide the features of the image input stream, to achieve full information fusion and construct a fine-level mapping between the image and the hierarchical labels.

[0035] Please see Figure 2 As shown, in the dual cross-focus module, the feature vector r obtained from the SwinT module is used as the key and value extracted from the image, and the corresponding label w is used as the query. Let... , and Given the learned weight matrix, then , , The attention weight matrix is ​​calculated based on the query Q and key K, and the similarity between the query and the key is measured using a dot product operation. As follows:

[0036] Here, j takes the value 1 or 2, representing the first-layer cross-attention mechanism or the second-layer cross-attention mechanism, respectively. , , The dimensions are all , Represents the number of followers. This is the dimension of the key, used to scale the dot product result. The total number of heads to follow is... ,and This mechanism allows each attention head to learn the correlation between global image features and the current label independently, generating image-label attention maps at different dimensional levels.

[0037] Subsequently, the outputs of the multi-head attention are concatenated and transformed using a linear transformation matrix W. h1Generate attention output. Reintegrate this output with the initial input via residual connections and perform layer normalization to obtain the output of the first layer of the cross-attention mechanism. This step helps to reduce overfitting and enhance the stability of the model.

[0038] In the second layer of the dual cross-attention mechanism, the output of the first layer is utilized. Further queries are performed using the original image features as keys and values ​​to learn the mapping between image features and hierarchical labels at a deeper level. For the attention of each head, a new set of linear transformation matrices is used to compute the attention scores of these inputs, which are then scaled and normalized to determine the weight distribution, as shown in Equation (6), where j is 2. Similar to the first layer, the attention outputs of all heads are concatenated and then processed through a linear matrix W. h2 Perform the transformation to generate the final output of this layer:

[0039] This represents the relational mapping obtained after fully considering the association between the image and the hierarchical label, providing information-rich fusion features for subsequent hierarchical classification.

[0040] The output of the second-layer cross-interest mechanism is connected to the output of the first-layer cross-interest mechanism via a residual link. This is then combined. Subsequently, the data undergoes layer normalization to generate the final output Z of the dual cross-concern mechanism. x The calculation is as follows:

[0041] A dual cross-attention mechanism is employed to establish a two-way cross-attention relationship between images and text labels. On the one hand, text label information is used to guide the learning of image features, making the image features more consistent with the descriptions of the text labels; on the other hand, image features can also be used to enrich the representation of text labels, making them more specific and accurate. This two-way information flow helps to achieve full integration between images and text labels, constructing a more refined mapping relationship between images and hierarchical labels.

[0042] Step 4: Obtain comprehensive attention information Z x Finally, the hierarchical label information is obtained using the label generation module. The label generation module consists of an MLP layer and a softmax layer. The MLP layer uses two fully connected layers, followed by a softmax function to obtain the confidence score for each label. .

[0043] Finally, the model training uses cross-entropy as the loss function. The formulas are as follows:

[0044] Where N is the total number of data samples, and M is the total number of label categories. ik This represents the true label value of the current sample. ik Predict the label value for the current sample.

[0045] For any parts of this invention not described in detail, please refer to the prior art or techniques known to those skilled in the art.

[0046] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A hierarchical image classification method based on CA-SwinT, characterized in that: The method comprises: S1, obtaining an original image, and preprocessing the original image; S2, extract features from the image through the SwinT module; S3, using the double cross attention mechanism, the extracted image features and classification label information are fully integrated to build a fine mapping between the image and the hierarchical label; S4. According to the refined mapping of S3, the label generation module generates corresponding labels and updates the classification label information.

2. The CA-SwinT-based hierarchical image classification method according to claim 1, characterized in that: The step of extracting features from the image by using the SwinT module in step S2 includes: S201, slicing and blocking processing: dividing the image into a series of small blocks; S202, linear embedding processing: flatten each block and pass it through a fully connected linear embedding layer to convert the pixel value of each block into a high-dimensional feature vector; S203, two-stage Swin module processing: continuously input the processing results of S202 into the two-stage Swin module for feature extraction; S204, block merging processing: downsampling the image to halve the height and width of the image and double the channels; S205, two-stage Swin processing: continuously inputting the processing results of S204 into the two-stage Swin module for feature extraction; S206, linear embedding processing: performing linear transformation on the processing result of S205.

3. The CA-SwinT-based hierarchical image classification method according to claim 2, characterized in that: The processing process of each level of Swin module in step S203 and step S205 includes: Layer normalization processing: realize data standardization; Window multi-head self-attention processing: All pixels of the corresponding input image are divided into multiple windows, and the correlation between each pixel and other pixels is calculated within each window to capture local features; Residual connection processing, and layer normalization processing on the processing result; the residual connection processing includes directly adding the input feature to the self-attention processing result of the previous step to form a residual connection; Multilayer perceptron processing: Perform complex transformations and combinations of features through fully connected layers and nonlinear activation functions to improve the model's feature expression capabilities; Moving window multi-head self-attention processing: All pixels of the corresponding input image are divided into multiple windows. When dividing the windows, the divided windows are shifted to the right and downward by several pixels at the same time. The correlation between each pixel and other pixels is calculated within each window, so as to realize information interaction between different windows; Residual connection processing, and layer normalization processing on the processing result; the residual connection processing includes directly adding the input feature to the self-attention processing result of the previous step to form a residual connection; Multi-layer perceptron processing: Perform complex transformations and combinations of features through fully connected layers and nonlinear activation functions to improve the model's feature expression capabilities.

4. The CA-SwinT-based hierarchical image classification method according to claim 1, characterized in that: In step S3, the double cross attention mechanism is used to fully integrate the extracted image features and label information, and the steps of constructing a fine mapping between the image and the hierarchical label include: S301, using the features of each block extracted by the SwinT module as the key and value, using the current classification label information as the query, performing multi-head attention mechanism processing, merging the processing results and performing layer normalization processing, and obtaining the output result of the first layer cross attention mechanism; S302: Use the features of each block extracted by the SwinT module as the key and value, use the result of S301 as the query, perform multi-head attention mechanism processing, merge the processing results and perform layer normalization processing to obtain the output result of the second-layer cross-attention mechanism.

5. The CA-SwinT-based hierarchical image classification method according to claim 1, characterized in that: The content of the corresponding labels generated by the label generation module includes: S401, Multilayer Perceptron Processing: Process the input data through two fully connected layers to improve the feature expression capability; S402, classification processing: confidence is calculated through the softmax function to achieve classification.

6. A hierarchical image classification system based on CA-SwinT, characterized in that: The system comprises: A preprocessing module, used for receiving an original image and performing preprocessing on the original image; A feature extraction module is used to extract features from images through a SwinT module; The double cross attention module is used to fully integrate the extracted image features and classification label information using the double cross attention mechanism to construct a fine mapping between the image and the hierarchical label; The label generation module is used to generate corresponding labels according to the refined mapping obtained by the double cross attention module and update the classification label information.

7. The CA-SwinT-based hierarchical image classification system according to claim 6, characterized in that: The feature extraction SwinT module includes the following connected in sequence: Slice and block layer: used to divide the image into a series of small blocks; Linear embedding layer: used to flatten each block and pass it through a fully connected linear embedding layer to convert the pixel value of each block into a high-dimensional feature vector; Two-level Swin module: used to extract features from the processing results of the previous linear embedding layer; Block merging module: used to halve the image height and width and double the channels by downsampling; Two-level Swin module: used to extract features from the processing results of the previous block merging module; Linear embedding module: used to perform linear transformation on the processing results of the previous two levels of Swin modules.

8. The CA-SwinT-based hierarchical image classification system according to claim 7, characterized in that: Each level of the Swin module includes the following connected in sequence: Layer normalization layer: used to standardize input data; Windowed multi-head self-attention module: used to divide all pixels of the corresponding input image into multiple small windows, and calculate the correlation between each pixel and other pixels in each window to capture local features; Residual connection + layer normalization module: used to directly add the input features to the self-attention processing results of the previous module to form a residual connection, and then perform layer normalization to alleviate the gradient vanishing problem; Multilayer Perceptron: used to perform complex transformations and combinations of features through fully connected layers and nonlinear activation functions to improve the model's feature expression capabilities; Moving window multi-head self-attention module: It is used to divide all pixels of the corresponding input image into multiple small windows. When dividing the windows, the divided windows are shifted to the right and downward by several pixels at the same time. The correlation between each pixel and other pixels is calculated inside each window, so as to realize information interaction between different windows. Residual connection + layer normalization module: used to directly add the input features to the self-attention processing results of the previous module to form a residual connection, and then perform layer normalization to alleviate the gradient vanishing problem; Multilayer Perceptron: It is used to perform complex transformations and combinations of features through fully connected layers and nonlinear activation functions to improve the feature expression capabilities of the model.

9. The CA-SwinT-based hierarchical image classification system according to claim 8, characterized in that: The dual cross-focus module includes: The first cross-attention mechanism layer: It is used to use the features of each block extracted by the SwinT model as the key and value, and the current classification label information as the query, perform multi-head attention mechanism processing, merge the processing results and perform layer normalization processing to obtain the output result of the first layer cross-attention mechanism; The second cross-attention mechanism layer is used to take the features of each block extracted by the SwinT model as the key and value, and the output results of the first-layer cross-attention mechanism as the query, perform multi-head attention mechanism processing, merge the processing results and perform layer normalization processing to obtain the output results of the second-layer cross-attention mechanism.

10. The CA-SwinT-based hierarchical image classification system according to claim 8, characterized in that: The label generation module includes: Multilayer Perceptron: used to process input data through two fully connected layers to improve feature expression capabilities; Classification module: used to calculate confidence through the softmax function to achieve classification.

Images