Drawing style recognition method and system based on lightweight deep learning model

Abstract

The application discloses a painting style recognition method and system based on a lightweight deep learning model, and belongs to the field of artificial intelligence and artistic image recognition. The application designs an IRA-HCT-KAN end-to-end classification network, integrates six IRA (inverse residual attention) modules, three HCT (convolution and Transformer hybrid) modules and a KAN (Kolmogorov-Arnold network) classification layer, extracts, converts and classifies painting style categories through features, and realizes efficient local feature extraction and down-sampling through the IRA module. The HCT module fuses the advantages of convolution local perception and Transformer global modeling, and the KAN classification layer replaces the fully connected layer to improve the interpretability. The application considers the recognition performance, lightweight and decision transparency, can be deployed on mobile or edge devices, is suitable for the fields of art history research and digital cultural relics, and is helpful to intelligent understanding of cultural and artistic images.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and art image recognition technology, and relates to a method and system for painting style recognition based on a lightweight deep learning model. Background Technology

[0002] Automatic identification of painting styles is a key technology in fields such as art history research, museum digitization, and intelligent art analysis. In recent years, deep learning techniques, especially convolutional neural networks (CNNs) and visual transformers (ViTs), have achieved remarkable results in image classification tasks and have been applied to painting style recognition.

[0003] However, existing technologies still have the following drawbacks: Inconsistency between local and global modeling: CNN models excel at capturing local texture and edge features, but limited by their local receptive field, they struggle to effectively model global semantic information such as the overall brushstroke distribution and color harmony in painting styles. While ViT models can capture long-distance dependencies through self-attention mechanisms, their ability to model local details is relatively weak, and these models typically have a large number of parameters.

[0004] Lightweight design and high performance are difficult to balance: To adapt to deployment on mobile or embedded devices, researchers have proposed lightweight CNN models such as MobileNet, but their accuracy is often lower than that of larger models. ViT-type models have superior performance, but their computational cost is high, making them difficult to apply in resource-constrained scenarios.

[0005] Poor interpretability of classification decisions: Most mainstream models use fully connected layers as classification heads, which have problems such as parameter redundancy and black-box decision-making, and lack a clear explanation of the classification logic.

[0006] Therefore, there is an urgent need for a painting style classification model that can effectively integrate local and global features, is both lightweight and high-performance, and has more interpretable classification decisions. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a painting style recognition method and system based on a lightweight deep learning model (IRA-HCT-KAN) to solve the problems of inconsistent local and global feature modeling, difficulty in balancing lightweight and high performance, and insufficient interpretability of classification results in existing painting style recognition technologies.

[0008] To achieve the above objectives, the present invention provides the following technical solution: Scheme 1: An IRA-HCT-KAN model for painting style recognition, comprising an input layer and an output layer, characterized in that it further comprises six cascaded inverse residual attention (IRA) modules, three convolutional and Transformer hybrid (HCT) modules respectively cascaded after the last three IRA modules, and a classification layer based on a Kolmogorov-Arnold network (KAN).

[0009] The input layer is used to receive painting image data; The IRA module is used for local feature extraction and spatial dimension downsampling; while reducing computational load, it strengthens key features and suppresses redundant information, providing the model with more discriminative feature representations. The HCT module is used to balance the advantages of local perception in convolution and global modeling in Transformer, and to realize the transformation from local features to global semantics. The KAN-based classification layer replaces the fully connected layer with a combination of univariate functions as the final classification decision module, reducing parameters and improving interpretability. The output layer uses Softmax to generate the probability distribution of painting style categories.

[0010] Furthermore, the IRA module includes two variants: a downsampling IRA module located at positions 2, 4, 5, and 6 of the cascade, with a stride of 2 for its depthwise separable convolutions, used to compress the spatial dimension of the feature map, and without residual connections; and a same-dimensional IRA module located at positions 1 and 3 of the cascade, with a stride of 1 for its depthwise separable convolutions, preserving the spatial dimension of the feature map, and including cross-layer residual connections to mitigate gradient vanishing.

[0011] Furthermore, the IRA module achieves "dimensionality enhancement - feature refinement - attention enhancement - dimensionality reduction" processing of input features through dimensionality transformation, depthwise separable convolution (DW), and attention mechanism (CBAM) (including channel attention module and spatial attention module), focusing on efficient feature extraction and enhancement.

[0012] Dimensionality increase: The number of channels of the input features is expanded by 1×1 convolution (expansion factor β=4), and the SiLU activation function is used.

[0013] Feature refinement: Spatial features are extracted using 3×3 depthwise separable convolution (DW Conv), which significantly reduces the computational cost.

[0014] Attention enhancement: An attention mechanism is applied after depthwise separable convolution; the attention mechanism first focuses on important feature channels through the channel attention module (CAM), and then focuses on key spatial regions through the spatial attention module (SAM), thereby enhancing key features.

[0015] Dimensionality reduction: Compress the number of channels back to the target dimension using 1×1 convolution.

[0016] Furthermore, the HCT module specifically includes: Local feature extraction: The input feature map is subjected to 3×3 convolution and 1×1 convolution in sequence to obtain the local feature map of the intermediate dimension.

[0017] Global feature modeling: The local feature map is segmented into a non-overlapping sequence of patches and processed by a multi-layer Transformer encoder; the Transformer encoder includes layer normalization, multi-head self-attention mechanism and feedforward neural network.

[0018] Feature fusion output: The sequence output by the Transformer is folded back into a two-dimensional feature map through the inverse operation, and then concatenated with the original input feature map in the local feature extraction after channel alignment. Finally, it is fused through a 3×3 convolutional layer to obtain the final output.

[0019] Furthermore, the KAN-based classification layer includes: Global average pooling layer is used to aggregate spatial information from feature maps; The flattening layer is used to convert the pooled feature vector into a one-dimensional vector; the dropout layer is used for regularization to prevent overfitting. At least one KAN network layer, each layer including weights for the linear transformation from the input to the univariate function, a set of learnable univariate activation functions, and linear weights from the univariate function output to the layer output; The Softmax layer is used to normalize the output of the KAN layer into a class probability distribution.

[0020] Solution 2: A painting style recognition method based on the IRA-HCT-KAN model, comprising the following steps: S1: Obtain the painting images to be classified and preprocess them; S2: Input the preprocessed image into the input layer of the IRA-HCT-KAN model; S3: Image features are extracted and transformed through the IRA module and HCT module; S4: After processing by the KAN-based classification layer, output the probability of each painting style category; S5: The category with the highest probability is taken as the final style classification result for the painting image.

[0021] Furthermore, in step S1, the preprocessing step includes adjusting the image size, data augmentation, and normalization processing. The data augmentation employs random cropping and random horizontal flipping.

[0022] Solution 3: A painting style recognition system based on the IRA-HCT-KAN model, comprising a data preprocessing module, an IRA-HCT-KAN model, and a result output module. The data preprocessing module preprocesses the painting image data; the IRA-HCT-KAN model performs painting style recognition on the preprocessed image data; and the result output module outputs the painting style classification results. The data preprocessing module is built on the PyTorch framework and runs on a hardware platform equipped with an NVIDIA GPU.

[0023] Solution 4: A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the painting style recognition method described in Solution 2.

[0024] Solution 5: An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the painting style recognition method described in Solution 2.

[0025] The beneficial effects of this invention are as follows: The IRA-HCT-KAN model proposed in this invention achieves synergistic optimization of recognition performance, model efficiency, and decision transparency by integrating an inverse residual attention module (IRA), a hybrid convolutional and Transformer module (HCT), and a classification layer based on a Kolmogorov-Arnold network (KAN). Specific advantages are as follows: (1) Superior performance: By enhancing local details through the IRA module and capturing global semantics through the HCT module, a deep fusion of local and global features is achieved. On the constructed dataset of 9 painting styles, a Top-1 accuracy of 85.5% is achieved, outperforming mainstream lightweight CNN and ViT models. Its main purpose is to provide an efficient, deployable, and interpretable solution for art style recognition, applicable to fields such as art history research, cultural heritage digitization, and intelligent art analysis, thus promoting the development of intelligent image understanding technology for cultural and artistic fields.

[0026] (2) Highly lightweight: It adopts depthwise separable convolution, inverted residual structure and KAN classification layer. The total number of model parameters is only 2.3M, which is significantly lower than similar performance models, making it easy to deploy on mobile devices or edge computing devices.

[0027] (3) Strong interpretability: The CBAM module clearly defines the channels and spatial regions that the model focuses on, and the KAN classification layer provides a clear input-output mapping path, which enhances the transparency of the model's decision-making.

[0028] (4) Good robustness: Ablation experiments show that each module contributes significantly to the final performance, and the model design is reasonable and effective.

[0029] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is the overall architecture diagram of the IRA-HCT-KAN model proposed in this invention; Figure 2 This is a schematic diagram of the IRA module, where (a) is a downsampling variant and (b) is a same-dimensional variant; Figure 3 This is a schematic diagram of the HCT module; Figure 4 This is a structural diagram of the KAN classification module; Figure 5 These are sample images of nine painting style categories in the dataset constructed in this invention; Figure 6 It is a distribution chart of the number of samples in each category of the dataset; Figure 7 This is a schematic diagram of the cosine annealing learning rate scheduling strategy; Figure 8 It is a graph showing how the model's training and testing accuracy changes over the training period; Figure 9 This is the confusion matrix of the model on the test set; Figure 10 It is a bar chart showing the model's precision, recall, and F1 score across various categories; Figure 11 These are visualization comparisons of t-SNE features under different attention mechanisms, where (a) no attention, (b) SE attention, and (c) CBAM attention. Figure 12 This is a comparison of confusion matrices for different decision layers, including (a) fully connected layer (FC), (b) support vector machine (SVM), and (c) KAN layer. Detailed Implementation

[0031] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0032] Please see Figures 1-12 This invention provides a painting style recognition model IRA-HCT-KAN, the structure of which is as follows: Figure 1 As shown, it mainly includes an input layer (inputting a 224×224 RGB image), an inverse residual attention (IRA) module (4 IRA modules progressively compressing the spatial dimension), a hybrid convolutional and transformer (HCT) module (3 HCT blocks, containing 2 / 4 / 3 Transformer encoders), a classification layer based on the Kolmogorov-Arnold network (KAN) (global pooling → Flatten → Dropout → KAN Layer → Softmax), and an output layer (outputting style probabilities).

[0033] Inverse Residual Attention (IRA) module (such as Figure 2 As shown, the IRA module, serving as the front end of the model, is responsible for efficiently extracting local features and downsampling spatial dimensions. Its structure borrows from the inverse residual structure of MobileNet, comprising four steps: dimensionality increase, depthwise separable convolution, attention enhancement, and dimensionality reduction. Dimensionality increase: The number of channels in the input features is expanded using a 1×1 convolution (expansion factor β=4), and the SiLU activation function is applied. Depthwise separable convolution: Spatial features are extracted using a 3×3 depthwise separable convolution (DW Conv), significantly reducing computational cost. Attention enhancement: The CBAM module is connected after DW Conv. CBAM first focuses on important feature channels through the Channel Attention Module (CAM), and then focuses on key spatial regions through the Spatial Attention Module (SAM), thereby enhancing key features. Dimensionality reduction: The number of channels is compressed back to the target dimension using a 1×1 convolution. Based on its structure, we designed two variants: downsampling (stride=2, no residual) and same-dimensional (stride=1, with residual) IRA modules, which can be flexibly applied to different stages of the model.

[0034] Hybrid Convolutional and Transformer (HCT) modules (such as...) Figure 3As shown in the diagram, the core of the model is to transform local features into global semantics, addressing the insufficient global modeling capabilities of CNNs. Its main process is as follows: Local Feature Extraction: The input feature map is convolved with a 3×3 layer, and then the number of channels is adjusted using a 1×1 convolution to obtain the local feature map. Global Feature Modeling: The local feature map is segmented into non-overlapping patches, forming a sequence. This sequence is processed by a multi-layer Transformer encoder. Each Transformer layer includes layer normalization, a multi-head self-attention mechanism (used to capture the global relationships between all patches), and a feedforward neural network. Feature Fusion Output: The sequence output by the Transformer is restored to a two-dimensional feature map through a "fold" operation. This global feature map is then concatenated and fused with the original input feature map of the HCT module, and output through a 3×3 convolutional layer. This design allows the effective receptive field of the HCT module to be the entire input image.

[0035] Classification layers based on Kolmogorov-Arnold networks (KAN) (such as...) Figure 4 As shown in the diagram, this replaces the traditional fully connected layer as the final classification decision module, improving efficiency and interpretability. Its structure includes global average pooling, flattening, Dropout, KAN layers, and Softmax. The KAN layer, based on the Kolmogorov-Arnold representation theorem, replaces the fixed activation function in the fully connected layer with a set of learnable univariate functions (such as spline functions). Input features, after linear transformation, are fed into these univariate functions for nonlinear mapping, and then combined with another set of linear weights to obtain the output. This structure has fewer parameters, and the transformation path for each input dimension is clear, resulting in stronger interpretability.

[0036] Example 1: like Figure 1 As shown, this embodiment provides a painting style recognition method based on the IRA-HCT-KAN model, which specifically includes the following steps: Step 1: Construct the IRA-HCT-KAN model: The input is a 224×224 drawing image; Front-end feature extraction: The input image is first processed by a 3×3 convolution and an IRA module with a stride of 1. Figure 2 In (a), the feature map is compressed to 128×128. Then, it is processed through an IRA module with a stride of 2 ( Figure 2 The feature map is further compressed to 64×64 using two IRA modules with a step size of 1 (b) and two IRA modules with a step size of 1.

[0037] Core semantic modeling: Three HCT blocks are sequentially connected on a 64×64 feature map. Each HCT block is preceded by an IRA module with a stride of 2 for downsampling. Finally, the feature map is compressed to 8×8. The specific implementation of the HCT block is as follows... Figure 3 As shown: First, perform 3×3 and 1×1 convolutions, then segment the features into patches and input them into the Transformer, and finally fuse the global features with the original features for output.

[0038] Final classification decision: The output of the HCT module is reduced in dimensionality by 1×1 convolution and then fed into the KAN classification layer (e.g., Figure 4 First, global average pooling is performed, then flattening and Dropout are applied, followed by two layers of KAN network, and finally, the probability of 9 painting style categories is output through Softmax.

[0039] Step 2: Training and Reasoning Dataset: A balanced dataset containing nine mainstream painting styles (such as Impressionism, Expressionism, Cubism, etc.) was constructed, and its sample illustration is shown in the figure below. Figure 5 As shown, there are a total of 1573 images, divided into training and test sets in an 8:2 ratio. The data sample distribution for each category is as follows. Figure 6 As shown.

[0040] Preprocessing: During training, random cropping and random horizontal flipping are used for data augmentation, and the mean and standard deviation of ImageNet are used for standardization.

[0041] Hyperparameters: AdamW optimizer used, initial learning rate 0.001, cosine annealing strategy employed. Figure 7 The weight decay is 0.01, the dropout rate is 0.2, and the training lasts for 100 epochs.

[0042] Inference: After processing the image to be classified according to the above preprocessing method, input it into the trained model. The class with the highest probability output by the model is the classification result.

[0043] Step 3: Experimental Verification like Figure 8 As shown, the model training process was stable, and overfitting did not occur. Figure 9 and Figure 10 As shown, the model achieved excellent classification performance on the test set. As shown in Tables 1 and 2, this model significantly outperforms lightweight CNNs such as MobileNet and ShuffleNet, and heavyweight CNNs such as DenseNet and EfficientNet, with the same or lower parameter count. As shown in Table 3, this model also outperforms ViT models such as CrossViT and PiT, which require advanced augmentations, even when using only basic data augmentation. Figure 11As shown, the CBAM attention mechanism can significantly improve feature discriminative power. Figure 12 As shown, the KAN classification layer can construct a more accurate decision boundary compared to FC and SVM.

[0044] Table 1 Comparison with lightweight CNNs

[0045] Table 2 Comparison of CNN weights

[0046] Table 3 Comparison with advanced ViT models

[0047] In summary, this invention provides an efficient, accurate, and interpretable solution for classifying painting styles.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An IRA-HCT-KAN model for painting style recognition, comprising an input layer and an output layer, characterized in that, It also includes six cascaded IRA modules, three HCT modules cascaded after the last three IRA modules, and a KAN-based classification layer, where IRA stands for inverse residual attention, HCT stands for a mixture of convolution and Transformer, and KAN stands for Kolmogorov-Arnold network. The input layer is used to receive painting image data; The IRA module is used for local feature extraction and spatial dimension downsampling; The HCT module is used to balance the advantages of local perception in convolution and global modeling in Transformer, and to realize the transformation from local features to global semantics. The KAN-based classification layer replaces the fully connected layer with a combination of univariate functions as the final classification decision module, reducing parameters and improving interpretability. The output layer uses Softmax to generate the probability distribution of painting style categories.

2. The IRA-HCT-KAN model according to claim 1, characterized in that, The IRA module includes two variants: a downsampling IRA module located at positions 2, 4, 5, and 6 of the cascade, with a stride of 2 for its depthwise separable convolutions, used to compress the spatial dimension of the feature map and without residual connections; and a same-dimensional IRA module located at positions 1 and 3 of the cascade, with a stride of 1 for its depthwise separable convolutions, preserving the spatial dimension of the feature map and including cross-layer residual connections to mitigate gradient vanishing.

3. The IRA-HCT-KAN model according to claim 2, characterized in that, The IRA module achieves "dimensionality enhancement - feature refinement - attention enhancement - dimensionality reduction" processing of input features through dimensionality transformation, depthwise separable convolution, and attention mechanisms, focusing on efficient feature extraction and enhancement; the attention mechanism includes a channel attention module and a spatial attention module. Dimensionality increase: The number of channels of the input features is expanded by 1×1 convolution and the SiLU activation function is used; Feature refinement: Spatial features are extracted using 3×3 depthwise separable convolution; Attention enhancement: An attention mechanism is applied after depthwise separable convolution; the attention mechanism first focuses on important feature channels through the channel attention module, and then focuses on key spatial regions through the spatial attention module, thereby enhancing key features; Dimensionality reduction: Compress the number of channels back to the target dimension using 1×1 convolution.

4. The IRA-HCT-KAN model according to claim 1, characterized in that, The HCT module specifically includes: Local feature extraction: The input feature map is subjected to 3×3 convolution and 1×1 convolution in sequence to obtain the local feature map of the intermediate dimension; Global feature modeling: The local feature map is segmented into a non-overlapping sequence of patches, which is then processed by a multi-layer Transformer encoder; the Transformer encoder includes layer normalization, multi-head self-attention mechanism and feedforward neural network; Feature fusion output: The sequence output by the Transformer is folded back into a two-dimensional feature map through the inverse operation, and then concatenated with the original input feature map in the local feature extraction after channel alignment. Finally, it is fused through a 3×3 convolutional layer to obtain the final output.

5. The model according to claim 1, characterized in that, The KAN-based classification layer includes: Global average pooling layer is used to aggregate spatial information from feature maps; The flattening layer is used to convert the pooled feature vector into a one-dimensional vector; the dropout layer is used for regularization to prevent overfitting. At least one KAN network layer, each layer including weights for the linear transformation from the input to the univariate function, a set of learnable univariate activation functions, and linear weights from the univariate function output to the layer output; The Softmax layer is used to normalize the output of the KAN layer into a class probability distribution.

6. A method for identifying painting styles based on the model described in any one of claims 1 to 5, characterized in that, Includes the following steps: S1: Obtain the painting images to be classified and preprocess them; S2: Input the preprocessed image into the input layer of the IRA-HCT-KAN model; S3: Image features are extracted and transformed through the IRA module and HCT module; S4: After processing by the KAN-based classification layer, output the probability of each painting style category; S5: The category with the highest probability is taken as the final style classification result for the painting image.

7. The painting style recognition method according to claim 6, characterized in that, The preprocessing step includes adjusting image size, data augmentation, and normalization. Data augmentation employs random cropping and random horizontal flipping.

8. A painting style recognition system based on the model described in any one of claims 1 to 5, characterized in that, The system includes a data preprocessing module, an IRA-HCT-KAN model, and a result output module. The data preprocessing module preprocesses the painting image data; the IRA-HCT-KAN model performs painting style recognition on the preprocessed image data; and the result output module outputs the painting style classification results.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the painting style recognition method as described in claim 6 or 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the painting style recognition method as described in claim 6 or 7.

Images