Oil painting image style feature expression method, system and equipment based on comparative learning and medium

Abstract

The invention relates to an oil painting image style feature expression method, system and device based on comparative learning and a medium. The method comprises the following steps: performing double-branch feature extraction on an original oil painting image by using a double-branch structure style recognition model to obtain an emotion semantic feature vector and a visual texture feature vector; based on the emotional semantic feature vector and the visual texture feature vector, constructing and optimizing a double-branch structure style recognition model according to comparison learning loss to obtain an optimized style recognition model; and based on the optimized style recognition model, performing double-branch feature extraction on the target oil painting image, obtaining and splicing a target emotional semantic feature vector and a target visual texture feature vector, and obtaining a target structured style feature expression. According to the method, through double-branch structure style recognition model construction and contrast learning training, the characterization capability and interpretability of oil painting image style features are remarkably improved, and the independence and structured output capability of the features are enhanced.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a method, system, device and medium for expressing the stylistic features of oil painting images based on contrastive learning. Background Technology

[0002] Digital analysis and understanding of oil painting styles is an important research direction at the intersection of computer vision and artificial intelligence. It is widely applied in scenarios such as digital preservation of cultural heritage, popularization of art education, and development of digital creative industries. Its core requirement lies in accurately extracting discriminative and interpretable style features from oil painting images, providing technical support for advanced applications such as style classification, tracing, retrieval, and controllable fusion. However, traditional oil painting style feature extraction methods mainly rely on manually designed visual features, such as color histograms, brushstroke direction statistics, and wavelet texture coefficients. These methods can only capture low-dimensional visual patterns on the surface of the image and are unable to depict the high-level abstract semantics and complex texture patterns inherent in the style, resulting in limited feature representation capabilities and failing to meet the needs of refined style analysis.

[0003] With the development of deep learning technology, feature learning methods based on convolutional neural networks have gradually become mainstream. Among them, the technical solution proposed by Gatys et al. to represent style features using Gram matrices has had a profound impact. This type of method extracts the activation value statistics of the intermediate layers of pre-trained convolutional neural networks as style representations, which significantly improves the performance of tasks such as style transfer. However, existing deep learning methods still suffer from three major technical bottlenecks: First, the problem of feature hybridity is prominent. The style features extracted by existing analysis methods are mostly high-dimensional and dense hybrid vectors, which encode high-level emotional semantics, mid-level compositional rules and low-level visual textures indiscriminately, forming a "black box" feature expression that is difficult for humans to understand and cannot be edited or controlled in a targeted manner. Second, the features are too dependent on the content. The model is easily affected by the specific objects depicted in the painting, such as starry sky, figures, landscapes, etc., which leads to inaccurate judgment of paintings with different content but similar styles, or misclassification of paintings with the same content but different styles as the same type. Third, there is a lack of structured output capability. Existing features are mostly single-dimensional task-optimized vectors, which are only suitable for specific scenarios such as style classification and transfer, and cannot provide a directly operable structured data foundation for advanced applications such as style attribute editing and cross-dimensional style mixing. Summary of the Invention

[0004] Therefore, it is necessary to provide methods, systems, devices, and media for expressing the stylistic features of oil painting images based on contrastive learning to address the aforementioned technical problems. The aim is to improve the representational ability of oil painting stylistic features and enhance the interpretability, independence, and structured output capability of the features.

[0005] Firstly, this application provides a method for representing the style features of oil painting images based on contrastive learning, including:

[0006] Using a pre-defined dual-branch style recognition model, dual-branch feature extraction is performed on the original oil painting image to obtain emotional semantic feature vectors and visual texture feature vectors. The dual-branch style recognition model includes a shared convolutional neural network backbone, an emotional semantic feature extraction branch, and a visual texture feature extraction branch.

[0007] Based on the emotional semantic feature vector and the visual texture feature vector, a contrastive learning loss is constructed. Based on the contrastive learning loss, the network parameters of the dual-branch style recognition model are optimized to obtain the optimized style recognition model.

[0008] Based on the optimized style recognition model, a dual-branch feature extraction process is performed on the target oil painting image. The target emotional semantic feature vector and the target visual texture feature vector are then concatenated to obtain the target structured style feature expression.

[0009] In one embodiment, a preset dual-branch style recognition model is used to perform dual-branch feature extraction on the original oil painting image to obtain an emotional semantic feature vector and a visual texture feature vector, including:

[0010] The original oil painting image is input into a shared convolutional neural network backbone for feature extraction processing to obtain a primary multi-scale visual feature map.

[0011] The primary multi-scale visual feature maps are input into the sentiment semantic feature extraction branch and the visual texture feature extraction branch, respectively;

[0012] In the sentiment semantic feature extraction branch, channel attention weighting is performed on the primary multi-scale visual feature map to obtain the attention weighted feature map; global context aggregation is performed on the attention weighted feature map to obtain the global semantic description vector; and the first dimensionality reduction is performed on the global semantic description vector to obtain the sentiment semantic feature vector.

[0013] In the visual texture feature extraction branch, multi-scale feature extraction is performed on the primary multi-scale visual feature map to obtain multiple feature maps at different levels; the feature maps at different levels are processed by grouping Gram matrix calculation to obtain multiple sets of texture statistical descriptions; the multiple sets of texture statistical descriptions are aggregated to obtain a joint texture description vector; the joint texture description vector is subjected to a second dimensionality reduction process to obtain a visual texture feature vector.

[0014] In one embodiment, a contrastive learning loss is constructed based on the emotional semantic feature vector and the visual texture feature vector, including:

[0015] Obtain a pre-defined training dataset of oil painting images, which contains oil painting images created by multiple different painters; randomly sample multiple oil painting images from the training dataset to construct a training batch; each oil painting image in the training batch is used as an original oil painting image;

[0016] Random data augmentation is performed on each original oil painting image in the training batch to obtain the augmented oil painting image corresponding to each original oil painting image; using a dual-branch structure style recognition model, dual-branch feature extraction is performed on each augmented oil painting image to obtain the augmented sentiment semantic feature vector and augmented visual texture feature vector corresponding to each augmented oil painting image.

[0017] For the target original oil painting image in the training batch, the sentiment semantic feature vector corresponding to the target original oil painting image is used as the first anchor point sample, and the enhanced sentiment semantic feature vector of the enhanced oil painting image corresponding to the target original oil painting image is used as the first positive sample. The first anchor point sample and the first positive sample are combined to form the first positive sample pair. The sentiment semantic feature vectors corresponding to non-target oil paintings (excluding the target original oil painting image) and the enhanced sentiment semantic feature vectors of the enhanced oil paintings corresponding to the non-target oil paintings are used as the first negative sample set. Based on the first anchor point sample and each sample in the first negative sample set, multiple first negative sample pairs are formed. The InfoNCE loss function is used to calculate and process each first positive sample pair and each first negative sample pair to obtain the sentiment-content alignment contrast loss corresponding to the target original oil painting image.

[0018] The visual texture feature vector corresponding to the original target oil painting image is used as the second anchor point sample. In the same training batch, oil paintings belonging to the same painter as the original target oil painting image are considered as homogeneous oil paintings, and the visual texture feature vectors corresponding to the homogeneous oil paintings are used as the second positive samples. The second anchor point samples and each second positive sample are used to form a second positive sample pair. The visual texture feature vectors corresponding to oil paintings belonging to different painters than the original target oil painting image in the training batch are used as the second negative sample set. The second anchor samples and each sample in the second negative sample set are used to form a second negative sample pair. The InfoNCE loss function is used to calculate the texture-style invariant contrast loss corresponding to the original target oil painting image.

[0019] The global sentiment-content alignment contrast loss is obtained by averaging the sentiment-content alignment contrast loss corresponding to each original oil painting image in the training batch; the global texture-style invariant contrast loss is obtained by averaging the texture-style invariant contrast loss corresponding to each original oil painting image in the training batch; and the contrastive learning loss is obtained by combining the global sentiment-content alignment contrast loss and the global texture-style invariant contrast loss.

[0020] In one embodiment, the network parameters of the dual-branch style recognition model are optimized based on the contrastive learning loss to obtain an optimized style recognition model, including:

[0021] The global sentiment-content alignment contrast loss and the global texture-style invariance contrast loss are weighted and summed to obtain the total loss function.

[0022] The gradient of the total loss function with respect to the network parameters is obtained by performing gradient calculation on the total loss function using the backpropagation algorithm.

[0023] The gradient descent optimization algorithm is used to iteratively update the network parameters based on the gradient until the total loss function meets the preset convergence condition, thus obtaining the optimized style recognition model.

[0024] In one embodiment, in the emotion semantic feature extraction branch, the primary multi-scale visual feature map is subjected to channel attention weighting to obtain an attention-weighted feature map, including:

[0025] Global average pooling is performed on the primary multi-scale visual feature map to obtain the channel global statistical vector;

[0026] The channel global statistical vector is input into the preset fully connected network in the sentiment semantic feature extraction branch for nonlinear transformation processing to obtain the channel weight vector;

[0027] The channel weight vector is multiplied channel by channel with the primary multi-scale visual feature map to obtain the attention-weighted feature map.

[0028] In one embodiment, grouping Gram matrix calculations are performed on feature maps at multiple different levels to obtain multiple sets of texture statistical descriptions, including:

[0029] The channel dimension of the feature map at each level is evenly divided into a preset number of channel groups;

[0030] For each channel group, calculate the correlation matrix between the feature channels within the group to obtain the Gram matrix corresponding to the channel group;

[0031] Flatten the Gram matrices corresponding to the feature maps at the same level to obtain the texture description vectors corresponding to the feature maps at the same level.

[0032] By aggregating the texture description vectors corresponding to the feature maps at each level, multiple sets of texture statistical descriptions are obtained.

[0033] In one embodiment, the mathematical expression for the Gram matrix corresponding to the channel group is:

[0034]

[0035]

[0036] in, For the first The first in the hierarchical feature map The Gram matrix corresponding to each channel group has a dimension of ; For feature-level indexing, The total number of layers in the multi-scale feature map; k is the channel grouping index, k= ,K K is the quantity at each level; For the first The first in the hierarchical feature map The number of channels, and satisfying , Indicates the total number of channels; For the first The first in the hierarchical feature map Feature matrices of each channel group; This indicates a dimension reshaping operation, used to convert a 3D feature map into a 2D matrix.

[0037] Secondly, this application also provides a system for representing the style features of oil painting images based on contrastive learning, including:

[0038] The feature extraction module is used to perform bi-branch feature extraction on the original oil painting image using a preset bi-branch style recognition model to obtain emotional semantic feature vectors and visual texture feature vectors. The bi-branch style recognition model includes a shared convolutional neural network backbone, an emotional semantic feature extraction branch, and a visual texture feature extraction branch.

[0039] The model optimization module is used to construct a contrastive learning loss based on the emotional semantic feature vector and the visual texture feature vector. Based on the contrastive learning loss, the network parameters of the dual-branch structure style recognition model are optimized to obtain the optimized style recognition model.

[0040] The style expression module is used to perform dual-branch feature extraction on the target oil painting image based on the optimized style recognition model, and then concatenates the target emotional semantic feature vector and the target visual texture feature vector to obtain the target structured style feature expression.

[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the first aspect.

[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the first aspect.

[0043] The aforementioned method, system, device, and medium for representing oil painting image style features based on contrastive learning first extracts sentiment semantics and visual texture feature vectors through a dual-branch style recognition model, effectively solving the problems of mixed style features and poor interpretability in traditional methods, and improving the structure and independence of feature representation. Second, a contrastive learning loss is constructed based on the dual feature vectors, and the model is optimized to weaken the interference of content on style features, improving the discriminativeness and robustness of the features. Finally, a structured style expression is obtained by concatenating the target dual feature vectors, providing a direct operational basis for style editing, blending, and other applications, improving the method's practical adaptability, and further meeting the needs of refined style analysis. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart of a method for expressing the style features of oil painting images based on contrastive learning, provided as an exemplary embodiment of the present invention;

[0046] Figure 2 A flowchart of a method for performing bi-branch feature extraction on an original oil painting image is provided as an exemplary embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the structure of an oil painting image style feature representation system based on contrastive learning, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] In one embodiment, such as Figure 1As shown, a method for representing the style features of oil painting images based on contrastive learning is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0050] S101: Using a pre-defined dual-branch style recognition model, the original oil painting image is processed by dual-branch feature extraction to obtain emotional semantic feature vectors and visual texture feature vectors; the dual-branch style recognition model includes a shared convolutional neural network backbone, an emotional semantic feature extraction branch, and a visual texture feature extraction branch.

[0051] Specifically, oil painting style consists of two core dimensions: a high-level semantic dimension that conveys abstract information such as emotions and artistic conception, and a low-level texture dimension that reflects visual patterns such as brushstrokes and color usage. However, traditional single-branch models cannot effectively separate these two dimensions. Therefore, this S101 adopts a dual-branch parallel design for a dual-branch style recognition model, sharing the same convolutional neural network backbone to ensure consistency and efficiency in feature extraction. Illustratively, the original oil painting image is input into the shared convolutional neural network backbone of the model. This backbone network can be a deep convolutional network pre-trained for image recognition tasks, such as the ResNet series, which can extract general primary multi-scale visual features from the original image, providing a unified feature input basis for the subsequent two specialized branches and avoiding efficiency losses caused by repeated computation. Subsequently, the primary multi-scale visual features are simultaneously input into the emotional semantic feature extraction branch and the visual texture feature extraction branch. The two branches respectively perform feature enhancement and mapping for different dimensions of the style. The emotional semantic feature extraction branch focuses on capturing high-level abstract information that is relatively separate from the specific content of the image. Through semantic aggregation and dimensional mapping of primary features, it outputs emotional semantic feature vectors that can represent the emotional tone and artistic conception of the painting. The visual texture feature extraction branch focuses on extracting visual pattern information related to the painter's techniques. Through capturing and quantifying the texture patterns of primary features, it outputs visual texture feature vectors that can represent attributes such as brushstroke shape, color matching, and texture. This dual-branch structure physically separates emotional semantic and visual texture features, ensuring that each feature vector carries only a single dimension of style information. This significantly improves the interpretability of the features and provides an independent feature carrier for subsequent targeted comparative learning training.

[0052] S102: Based on the emotional semantic feature vector and the visual texture feature vector, a contrastive learning loss is constructed. Based on the contrastive learning loss, the network parameters of the dual-branch style recognition model are optimized to obtain the optimized style recognition model.

[0053] Specifically, by constructing a contrastive learning loss and optimizing model parameters, the discriminativeness and robustness of features can be enhanced, further weakening the interference of content on style features. Illustratively, the corresponding contrastive learning loss can be constructed based on the sentiment semantic feature vector and visual texture feature vector obtained in step S101. For the sentiment semantic feature vector, the design logic of its contrastive learning task is to "maintain the consistency of sentiment semantics within the same painting," meaning that regardless of any semantically non-destructive transformations performed on the image, its emotional expression should remain stable. Therefore, positive sample pairs of the original image and the enhanced image can be constructed to ensure that the sentiment semantic feature vector is robust to data enhancement operations such as local image cropping and color fine-tuning. For the visual texture feature vector, since different paintings by the same artist share common core brushstrokes, color usage, and other texture features despite differences in content, positive sample pairs of different paintings by the same artist can be constructed to guide the model to ignore content differences and focus on the artist's unique texture patterns. Furthermore, during loss calculation, the InfoNCE loss function can be used as the core loss calculation criterion. This function adjusts the similarity weights of sample pairs through a temperature parameter, effectively improving the model's ability to distinguish difficult samples and thus enhancing the discriminative power of features. After obtaining the contrastive learning loss, the network parameters of the dual-branch style recognition model can be optimized based on this loss. These network parameters encompass all trainable parameters in the shared convolutional neural network backbone, the sentiment semantic feature extraction branch, and the visual texture feature extraction branch. The optimization process can employ gradient descent algorithms such as the Adam algorithm, and the gradient of the loss function with respect to each parameter can be calculated through backpropagation. The parameters are iteratively updated according to the gradient direction until the loss function converges to a preset threshold, resulting in the optimized style recognition model.

[0054] S103: Based on the optimized style recognition model, the target oil painting image is subjected to dual-branch feature extraction processing, and the target emotional semantic feature vector and the target visual texture feature vector are concatenated to obtain the target structured style feature expression.

[0055] Specifically, after parameter optimization in step S102, the dual-branch style recognition model possesses stable decoupled feature extraction capabilities. Therefore, the target oil painting image can be input into the model. Based on the dual-branch feature extraction logic in step S101, the target emotional semantic feature vector and the target visual texture feature vector are obtained respectively. These two vectors inherit the decoupling characteristics from the training phase, carrying the emotional semantic and visual texture information of the target oil painting, respectively. The two feature vectors are then concatenated according to a preset dimensional order. The concatenated feature vector retains the independence of the two dimensional features while possessing overall representational capabilities, allowing it to be directly used for tasks such as style similarity calculation. This breaks through the black-box mode of traditional single feature vectors, as its internal dimensions have clear physical meaning, directly supporting advanced downstream applications. For example, in style mixing tasks, the weights of the two feature vectors can be independently adjusted to achieve controllable fusion of emotion and texture. In style retrieval tasks, targeted retrieval can be performed based on the two feature vectors separately, meeting the needs of refined analysis.

[0056] In one embodiment, such as Figure 2 As shown, a pre-defined dual-branch style recognition model is used to perform dual-branch feature extraction on the original oil painting image, resulting in an emotional semantic feature vector and a visual texture feature vector, including:

[0057] S201: Input the original oil painting image into the shared convolutional neural network backbone for feature extraction to obtain a primary multi-scale visual feature map; input the primary multi-scale visual feature map into the emotion semantic feature extraction branch and the visual texture feature extraction branch respectively;

[0058] S202: In the sentiment semantic feature extraction branch, channel attention weighting is performed on the primary multi-scale visual feature map to obtain an attention-weighted feature map; global context aggregation is performed on the attention-weighted feature map to obtain a global semantic description vector; the global semantic description vector is subjected to the first dimensionality reduction process to obtain the sentiment semantic feature vector.

[0059] S203: In the visual texture feature extraction branch, multi-scale feature extraction is performed on the primary multi-scale visual feature map to obtain multiple feature maps at different levels; the feature maps at different levels are processed by grouping Gram matrix calculation to obtain multiple sets of texture statistical descriptions; the multiple sets of texture statistical descriptions are aggregated to obtain a joint texture description vector; the joint texture description vector is subjected to a second dimensionality reduction process to obtain a visual texture feature vector.

[0060] Specifically, the shared convolutional neural network backbone can adopt a deep residual network architecture. This architecture, through residual connections, can alleviate the gradient vanishing problem in deep network training, ensuring the stability and effectiveness of the feature extraction process. For example, after preprocessing and adjusting the original oil painting image to a uniform size, it can be input into this backbone network and processed sequentially through multiple convolutional blocks, batch normalization layers, and activation function layers. Each convolutional block contains multiple stacked convolutional layers, with 3×3 kernels used to balance feature extraction capability and computational efficiency. The activation function uses the ReLU function to introduce a non-linear transformation. Simultaneously, multiple max-pooling layers can be set in the backbone network. Through pooling operations with a stride of 2, the spatial dimension of the feature maps is gradually reduced, increasing the receptive field of the features. The final output includes primary multi-scale visual feature maps with multiple different spatial resolutions and channel dimensions. This feature map set covers multi-dimensional feature representations ranging from low-level visual information such as image edges and colors to mid-level visual information such as object shape and local structure.

[0061] Furthermore, the primary multi-scale visual feature map can be used to generate two identical feature map copies through a tensor copy operation. These copies are then input into the sentiment semantic feature extraction branch and the visual texture feature extraction branch, respectively. The two branches employ a parallel processing architecture to ensure the independence and efficiency of the feature extraction process, avoiding mutual interference between different types of feature extraction processes. This allows for the targeted capture of both the sentiment semantic information and visual texture information of the oil painting image. Illustratively, in the sentiment semantic feature extraction branch, channel attention weighting can be applied to the primary multi-scale visual feature map. By adaptively learning the importance weights of each feature channel, the response of feature channels related to the emotional expression of the oil painting is strengthened, while noise interference from irrelevant channels is suppressed.

[0062] For example, a global average pooling operation is first performed on the primary multi-scale visual feature map to calculate the mean of each feature channel in the spatial dimension, resulting in a channel global statistical vector with the same dimension as the number of feature channels. This vector reflects the global information intensity of each channel. Then, this channel global statistical vector is input into a pre-defined fully connected network. This fully connected network contains two fully connected layers and one activation function layer. The first fully connected layer maps the channel global statistical vector to a low-dimensional space, and the activation function layer uses the sigmoid function to transform the mapping result to the [0,1] interval. The second fully connected layer maps the low-dimensional vector back to the same dimension as the original number of feature channels, finally yielding the channel weight vector. Its mathematical expression is: ,in Represents the channel weight vector. This represents the Sigmoid activation function. and These represent the weight matrices of the two fully connected layers, Represents the ReLU activation function. Represents the global statistical vector of the channel. This represents the bias vector of the second fully connected layer. Finally, the channel weight vector is multiplied channel by channel with the primary multi-scale visual feature map to obtain the attention-weighted feature map, thereby enhancing the emotion-related feature channels.

[0063] After completing the channel attention-weighted processing, a global self-attention mechanism can be used to perform global context aggregation on the attention-weighted feature map. This involves calculating the correlation between each spatial location in the attention-weighted feature map and all other spatial locations to capture long-distance dependencies, thus comprehensively representing the emotional semantic information of the oil painting image. For example, the attention-weighted feature map can be converted into a sequence of feature vectors, with one feature vector corresponding to each spatial location. Subsequently, query vectors, key vectors, and value vectors are generated. The attention weight matrix is ​​obtained by calculating the dot product of the query vector and the key vector. This matrix reflects the degree of influence of each spatial location on other locations. By weighted summing the attention weight matrix and the value vector, the globally context-aggregated feature vector, i.e., the global semantic description vector, can be obtained. This vector integrates the emotional semantic information of the entire feature map, comprehensively reflecting the emotional tendency conveyed by the oil painting. Finally, a structure consisting of a fully connected layer followed by a Dropout layer and an activation function layer can be used to perform the first dimensionality reduction on the global semantic description vector. The input dimension of the fully connected layer is the dimension of the global semantic description vector, and the output dimension is preset according to the model training requirements and feature expression efficiency. The dropout probability of the Dropout layer is set to a preset fixed value to randomly block some neuron connections and prevent the model from overfitting. The activation function is the ReLU6 function, which enhances the numerical stability of the feature vector by limiting the output value to the range of [0,6]. Finally, through this dimensionality reduction structure, a sentiment semantic feature vector with uniform dimension and strong expressive power can be output.

[0064] Specifically, in the visual texture feature extraction branch, multi-scale feature extraction can be performed on the primary multi-scale visual feature map based on its hierarchical characteristics. Different levels of feature maps correspond to different granularities of texture information in the oil painting image. Shallow feature maps mainly represent low-level texture details such as edges and color blocks; mid-level feature maps represent mid-level texture information such as brushstroke direction and local texture; and deep feature maps represent high-level texture information such as overall brushstroke style and overall image quality. Therefore, the primary multi-scale visual feature map can be divided into multiple feature subsets of different levels according to the output order and channel dimensions of the feature maps in the backbone network. Each level of feature map retains its original spatial structure and channel information, ensuring that no texture details at any granularity are lost, thereby achieving comprehensive capture of multi-granular texture information in the oil painting image.

[0065] Subsequently, Gram matrix calculations can be performed on feature maps at different levels to obtain multiple sets of texture statistical descriptions. Traditional full-channel Gram matrix calculations include all channels, leading to interference between texture information with different semantics, and the computational complexity increases sharply with a high number of channels. Therefore, this embodiment uses a channel grouping strategy to divide the channels of the feature map into multiple consecutive groups based on semantic relevance, calculating the correlation between channels only within each group. This preserves the local semantic consistency of texture information while significantly reducing computational redundancy, thus significantly improving the recognizability and computational efficiency of texture features. (Illustrative example)

[0066] For each level of feature map, the number of groups can be determined first based on the total number of channels and the granularity requirements of the texture information. The channel dimension of the feature map at that level is then uniformly divided into a preset number of continuous channel groups. The channel range of the k-th channel group in the l-th level feature map is defined by the following formula, such as:

[0067]

[0068] in, Indicates the first The original feature map of the hierarchy, and These are the start and end channel indices for the group, ensuring that each group contains consecutive and semantically similar channels. Then, a Reshape operation is used to generate the 3D feature map (dimension 1) for each channel group. Convert to a two-dimensional feature matrix Its dimensions are This involves flattening all pixels in the spatial dimension to the sample dimension, retaining the channel dimension as the number of channels within a group, thereby achieving format adaptation for calculating inter-channel correlation. Subsequently, the two-dimensional feature matrix... Calculate the product of the matrix and its transpose to obtain the correlation matrix of channels within the group. Then, normalize the correlation matrix, with the normalization coefficient being the number of channels within the group. For example, its calculation formula is:

[0069]

[0070] Using the above formula, we can obtain the first... The first in the hierarchical feature map Gram matrix corresponding to each channel group The matrix has dimensions of Each element in the group represents the response correlation between the two channels within the group across all spatial pixels, accurately capturing the local texture statistics corresponding to that group. Compared to traditional full-channel Gram matrices, this grouping strategy avoids invalid correlation calculations across semantic channels. For example, channels representing "brushstroke texture" and "color texture" can be grouped into different groups, allowing each Gram matrix to focus on only a single type of texture information, thereby improving the discriminative power of texture features. Simultaneously, after grouping, the dimension of each Gram matrix is ​​reduced from that of the full-channel matrix. Down to This significantly reduces computational complexity and memory usage, and improves the efficiency of model training and inference.

[0071] By performing the above operations sequentially on all channels of each level, multiple sets of texture statistical descriptions can be obtained. These statistical descriptions correspond to local texture information of different levels and different semantics, laying the feature foundation for subsequent aggregation to obtain a joint texture description vector.

[0072] Furthermore, a weighted summation method can be used to aggregate multiple sets of texture statistical descriptions to obtain a joint texture description vector. For example, the weight coefficients of each texture statistical description are first adaptively learned through a training process. The magnitude of the weight coefficient is positively correlated with the discriminative power of the corresponding texture statistical description; that is, the higher the discriminative power of the texture statistical description, the larger the corresponding weight coefficient. Then, each set of texture statistical descriptions is multiplied by its corresponding weight coefficient. By summing all the product results element-wise, a joint texture description vector can be obtained. This vector integrates multi-scale and multi-group texture information, and can comprehensively and selectively represent the visual texture features of the oil painting image.

[0073] Finally, a structure combining a 1×1 convolutional layer and a batch normalization layer can be used to perform a second dimensionality reduction on the joint texture description vector to obtain the visual texture feature vector. The 1×1 convolutional layer, by adjusting the number of output channels, can compress the dimensionality of the joint texture description vector. Its kernel size is 1×1, effectively reducing dimensionality and improving computational efficiency without altering the feature space structure. The batch normalization layer normalizes the output of the 1×1 convolutional layer, eliminating dimensional differences between different feature dimensions, accelerating model training convergence, and improving feature generalization ability, ultimately outputting a visual texture feature vector with uniform dimensions.

[0074] In one embodiment, a contrastive learning loss is constructed based on the emotional semantic feature vector and the visual texture feature vector, including:

[0075] Obtain a pre-defined training dataset of oil painting images, which contains oil painting images created by multiple different painters; randomly sample multiple oil painting images from the training dataset to construct a training batch; each oil painting image in the training batch is used as an original oil painting image;

[0076] Random data augmentation is performed on each original oil painting image in the training batch to obtain the augmented oil painting image corresponding to each original oil painting image; using a dual-branch structure style recognition model, dual-branch feature extraction is performed on each augmented oil painting image to obtain the augmented sentiment semantic feature vector and augmented visual texture feature vector corresponding to each augmented oil painting image.

[0077] For the target original oil painting image in the training batch, the sentiment semantic feature vector corresponding to the target original oil painting image is used as the first anchor point sample, and the enhanced sentiment semantic feature vector of the enhanced oil painting image corresponding to the target original oil painting image is used as the first positive sample. The first anchor point sample and the first positive sample are combined to form the first positive sample pair. The sentiment semantic feature vectors corresponding to non-target oil paintings (excluding the target original oil painting image) and the enhanced sentiment semantic feature vectors of the enhanced oil paintings corresponding to the non-target oil paintings are used as the first negative sample set. Based on the first anchor point sample and each sample in the first negative sample set, multiple first negative sample pairs are formed. The InfoNCE loss function is used to calculate and process each first positive sample pair and each first negative sample pair to obtain the sentiment-content alignment contrast loss corresponding to the target original oil painting image.

[0078] The visual texture feature vector corresponding to the original target oil painting image is used as the second anchor point sample. In the same training batch, oil paintings belonging to the same painter as the original target oil painting image are considered as homogeneous oil paintings, and the visual texture feature vectors corresponding to the homogeneous oil paintings are used as the second positive samples. The second anchor point samples and each second positive sample are used to form a second positive sample pair. The visual texture feature vectors corresponding to oil paintings belonging to different painters than the original target oil painting image in the training batch are used as the second negative sample set. The second anchor samples and each sample in the second negative sample set are used to form a second negative sample pair. The InfoNCE loss function is used to calculate the texture-style invariant contrast loss corresponding to the original target oil painting image.

[0079] The global sentiment-content alignment contrast loss is obtained by averaging the sentiment-content alignment contrast loss corresponding to each original oil painting image in the training batch; the global texture-style invariant contrast loss is obtained by averaging the texture-style invariant contrast loss corresponding to each original oil painting image in the training batch; and the contrastive learning loss is obtained by combining the global sentiment-content alignment contrast loss and the global texture-style invariant contrast loss.

[0080] Specifically, the pre-defined oil painting image training dataset encompasses oil paintings by multiple painters with distinct stylistic characteristics, each painting featuring different themes, compositions, and color expressions. A predetermined number of oil painting images are then selected from this dataset using random sampling without replacement to construct training batches. The distribution of painters within each batch must satisfy a uniformity constraint to avoid model bias caused by an excessively high proportion of samples from a single painter. Furthermore, random data augmentation processing can be performed on each original oil painting image in the training batch. Augmentation operations include, but are not limited to, random center cropping, color and brightness fine-tuning, Gaussian noise injection, and horizontal flipping. All augmentation operations are performed without compromising the core emotional semantics of the oil painting; for example, color fine-tuning is only adjusted within the overall tone and emotional tendency, ensuring that the augmented image has a consistent emotional expression with the original image. The augmented images are then input into a dual-branch style recognition model. Through the same feature extraction process, enhanced emotional semantic feature vectors and enhanced visual texture feature vectors are obtained, providing paired samples for subsequent comparative learning.

[0081] Specifically, the original and enhanced versions of the same image should have consistent emotional semantic expression. Even if the image undergoes local transformations, its emotional tone should remain stable. Therefore, for the target original oil painting image in the training batch, its emotional semantic feature vector can be used as the first anchor sample, and the enhanced emotional semantic feature vector of its corresponding enhanced oil painting image can be used as the first positive sample. The two together constitute the first positive sample pair. Furthermore, the emotional semantic feature vectors of all other non-target oil painting images in the training batch, along with their corresponding enhanced emotional semantic feature vectors, can be used together as the first negative sample set. These samples have significant differences in emotional semantics from the target original image; for example, an oil painting expressing sadness and an oil painting expressing joy are negative samples for each other. Subsequently, the sentiment-content alignment contrast loss corresponding to the target original image can be calculated based on the InfoNCE loss function. Its mathematical expression can be:

[0082]

[0083] in, For the first Sentiment-content alignment contrast loss for each target original image. The emotional semantic feature vector of the target original image. To enhance the emotional semantic feature vector of the image, and These are the sentiment semantic feature vectors of the original non-target image and the enhanced sentiment semantic feature vectors, respectively. This is the cosine similarity calculation function, used to measure the similarity between feature vectors. The temperature parameter is used to adjust the gradient smoothness of the loss function and control the compactness of the feature distribution.

[0084] Indicatively, the visual texture feature vector of the original target oil painting image can be used as the second anchor point sample. Within the same training batch, oil paintings belonging to the same artist as the original target image are selected as homogeneous oil paintings. Although these images differ in content, they possess the artist's unique brushstrokes, color usage, and other textural styles. Therefore, their visual texture feature vectors can be used as the second positive sample, forming a second positive sample pair with the second anchor point sample. Furthermore, the visual texture feature vectors of oil paintings belonging to different artists from the original target image in the training batch can be used as the second negative sample set. These images exhibit significant differences in texture style. Specifically, even if the themes of works by the same artist differ, their texture features still possess high similarity. Therefore, the texture-style invariance contrast loss corresponding to the original target image can then be calculated based on the InfoNCE loss function to guide the model to ignore differences in image content and focus on the artist's unique texture patterns.

[0085] The global sentiment-content alignment contrast loss can be obtained by arithmetically averaging the sentiment-content alignment contrast loss of all original oil painting images in the training batch, and the global texture-style invariant contrast loss can be obtained by arithmetically averaging the texture-style invariant contrast loss of all original oil painting images. Combining these two global losses yields the final contrastive learning loss.

[0086] In one embodiment, the network parameters of the dual-branch style recognition model are optimized based on the contrastive learning loss to obtain an optimized style recognition model, including:

[0087] The global sentiment-content alignment contrast loss and the global texture-style invariance contrast loss are weighted and summed to obtain the total loss function.

[0088] The gradient of the total loss function with respect to the network parameters is obtained by performing gradient calculation on the total loss function using the backpropagation algorithm.

[0089] The gradient descent optimization algorithm is used to iteratively update the network parameters based on the gradient until the total loss function meets the preset convergence condition, thus obtaining the optimized style recognition model.

[0090] Specifically, an adaptive weighting coefficient can be used to weight and sum the global sentiment-content alignment contrast loss and the global texture-style invariance contrast loss to obtain the total loss function. The adaptive weighting coefficient dynamically balances the contribution ratio of the two branch losses, rather than relying on fixed weights set manually, thus effectively addressing the model learning bias caused by differences in the dispersion of the two losses at different training stages. Illustratively, the adaptive weighting coefficient can be calculated from the standard deviation of the two global losses in the initial training stage (the first training epoch), and its mathematical expression is:

[0091]

[0092] in Represents the total loss function. For adaptive weighting coefficients, For global sentiment-content alignment contrast loss, For global texture-style invariant contrast loss. The calculation logic is the ratio of the standard deviation of the global sentiment-content alignment contrast loss in the initial stage to the sum of the standard deviations of the two global losses. In this way, when the dispersion of a certain loss is high, its corresponding weight coefficient will be increased accordingly, ensuring that the learning intensity of the feature of that branch is dynamically adjusted during the training process, thereby improving the overall optimization effect of the model.

[0093] Subsequently, the gradient of the total loss function can be calculated using the backpropagation algorithm. Illustratively, the backpropagation process is based on the chain rule, starting from the total loss function and sequentially calculating the partial derivatives of the loss with respect to the outputs and parameters of each network layer, forming a complete gradient propagation path. The network parameters can encompass all trainable parameters in the shared convolutional neural network backbone, the sentiment semantic feature extraction branch, and the visual texture feature extraction branch, including the weight matrices and bias vectors of convolutional layers, the weight matrices and bias vectors of fully connected layers, and learnable parameters in the attention mechanism. Furthermore, to avoid gradient explosion during gradient propagation, after calculating the gradients of each parameter, a gradient clipping operation can be performed, limiting the L2 norm of the gradient vector to a preset threshold range. When the L2 norm of the gradient vector exceeds this threshold, the gradient is scaled proportionally to ensure that the gradient value remains within a stable range, guaranteeing the convergence of model training.

[0094] Specifically, a gradient descent optimization algorithm can be used to iteratively update network parameters based on gradients. The Adam algorithm can be selected as the optimal gradient descent algorithm, as it combines momentum gradient descent with an adaptive learning rate adjustment mechanism, effectively improving the stability and convergence speed of parameter updates. For example, the Adam algorithm can independently adjust the update step size for each network parameter by maintaining momentum and adaptive learning rate terms. The momentum term, based on the exponential moving average of historical gradients, reduces oscillations during parameter updates, while the adaptive learning rate term, based on the exponential moving average of the squares of historical gradients, allows for a larger learning rate for sparse gradient parameters and a smaller learning rate for dense gradient parameters, achieving personalized parameter updates. During parameter updates, each network parameter is iteratively updated according to the rule of "current parameter value minus the product of the learning rate and the gradient," with real-time monitoring of the total loss function's trend. Furthermore, preset convergence conditions can include two dimensions: first, the total loss function remains stable on the validation set for multiple consecutive training epochs, i.e., the loss difference between adjacent epochs is less than a preset threshold; second, a preset maximum number of training epochs is reached. Parameter updates stop when either condition is met. The iteration can be stopped when one of the above conditions is met. The resulting dual-branch style recognition model is the optimized style recognition model. This model has stable emotional semantics and visual texture feature extraction capabilities, and can provide reliable support for the style feature expression of the target oil painting image.

[0095] Based on the same inventive concept, this application also provides a system for expressing the style features of oil paintings based on contrastive learning, which is used to implement the aforementioned method for expressing the style features of oil paintings based on contrastive learning. The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the system for expressing the style features of oil paintings based on contrastive learning provided below can be found in the limitations of the method for expressing the style features of oil paintings based on contrastive learning described above, and will not be repeated here.

[0096] In one exemplary embodiment, such as Figure 3 As shown, a system 300 for representing the style features of oil painting images based on contrastive learning is provided, including:

[0097] The feature extraction module 301 is used to perform bi-branch feature extraction processing on the original oil painting image using a preset bi-branch structure style recognition model to obtain emotional semantic feature vectors and visual texture feature vectors; the bi-branch structure style recognition model includes a shared convolutional neural network backbone, an emotional semantic feature extraction branch and a visual texture feature extraction branch;

[0098] The model optimization module 302 is used to construct a contrastive learning loss based on the emotional semantic feature vector and the visual texture feature vector, and to optimize the network parameters of the dual-branch structure style recognition model according to the contrastive learning loss, so as to obtain the optimized style recognition model.

[0099] The style expression module 303 is used to perform dual-branch feature extraction processing on the target oil painting image based on the optimized style recognition model, and then concatenate the target emotional semantic feature vector and the target visual texture feature vector to obtain the target structured style feature expression.

[0100] In one exemplary embodiment, the present invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the oil painting image style feature representation method based on contrastive learning of this application. A multi-core processor is preferred to improve the parallel processing capability of the system. The memory provides sufficient temporary storage space to support program execution and data processing. The memory capacity should be large enough to accommodate large amounts of data and computational tasks.

[0101] In one exemplary embodiment, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method for expressing the style features of oil painting images based on contrastive learning as described in this application. The computer-readable storage medium may include: a read-only memory, a random access memory, a solid-state drive, or an optical disk, etc.

[0102] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for representing the stylistic features of oil painting images based on contrastive learning, characterized in that, The method includes: Using a pre-defined dual-branch style recognition model, dual-branch feature extraction is performed on the original oil painting image to obtain emotional semantic feature vectors and visual texture feature vectors; the dual-branch style recognition model includes a shared convolutional neural network backbone, an emotional semantic feature extraction branch, and a visual texture feature extraction branch; Based on the emotional semantic feature vector and the visual texture feature vector, a contrastive learning loss is constructed. According to the contrastive learning loss, the network parameters of the dual-branch style recognition model are optimized to obtain the optimized style recognition model. Based on the optimized style recognition model, the target oil painting image is subjected to dual-branch feature extraction processing. The target emotional semantic feature vector and the target visual texture feature vector are then concatenated to obtain the target structured style feature expression.

2. The method according to claim 1, characterized in that, The method utilizes a pre-defined dual-branch style recognition model to perform dual-branch feature extraction on the original oil painting image, obtaining emotional semantic feature vectors and visual texture feature vectors, including: The original oil painting image is input into the shared convolutional neural network backbone for feature extraction processing to obtain a primary multi-scale visual feature map. The primary multi-scale visual feature maps are respectively input into the emotion semantic feature extraction branch and the visual texture feature extraction branch; In the emotional semantic feature extraction branch, the primary multi-scale visual feature map is subjected to channel attention weighting to obtain an attention-weighted feature map; the attention-weighted feature map is subjected to global context aggregation to obtain a global semantic description vector; the global semantic description vector is subjected to a first dimensionality reduction to obtain the emotional semantic feature vector. In the visual texture feature extraction branch, multi-scale feature extraction is performed on the primary multi-scale visual feature map to obtain multiple feature maps at different levels; grouping Gram matrix calculation is performed on each of the different levels of feature maps to obtain multiple sets of texture statistical descriptions; the multiple sets of texture statistical descriptions are aggregated to obtain a joint texture description vector; the joint texture description vector is subjected to a second dimensionality reduction to obtain the visual texture feature vector.

3. The method according to claim 1, characterized in that, The construction of a contrastive learning loss based on the emotional semantic feature vector and the visual texture feature vector includes: Obtain a preset oil painting image training dataset, which contains oil painting images created by multiple different painters; randomly sample multiple oil painting images from the oil painting image training dataset to construct a training batch; each oil painting image in the training batch is used as the original oil painting image; Random data augmentation is performed on each of the original oil painting images in the training batch to obtain the augmented oil painting image corresponding to each original oil painting image; using the dual-branch structure style recognition model, dual-branch feature extraction is performed on each of the augmented oil painting images to obtain the augmented emotional semantic feature vector and the augmented visual texture feature vector corresponding to each augmented oil painting image; For the target original oil painting image in the training batch, the sentiment semantic feature vector corresponding to the target original oil painting image is used as the first anchor point sample, and the enhanced sentiment semantic feature vector of the enhanced oil painting image corresponding to the target original oil painting image is used as the first positive sample. The first anchor point sample and the first positive sample are combined to form a first positive sample pair. The sentiment semantic feature vectors corresponding to non-target oil painting images other than the target original oil painting image in the training batch and the enhanced sentiment semantic feature vectors of the enhanced oil painting images corresponding to the non-target oil painting images are used as the first negative sample set. Based on the first anchor point sample and each sample in the first negative sample set, multiple first negative sample pairs are formed. The InfoNCE loss function is used to calculate and process each first positive sample pair and each first negative sample pair to obtain the sentiment-content alignment contrast loss corresponding to the target original oil painting image. The visual texture feature vector corresponding to the target original oil painting image is used as the second anchor point sample. In the same training batch, oil paintings belonging to the same painter as the target original oil painting image are considered as homogeneous oil paintings, and the visual texture feature vector corresponding to the homogeneous oil paintings is used as the second positive sample. A second positive sample pair is formed based on the second anchor point sample and each of the second positive samples. The visual texture feature vectors corresponding to oil paintings belonging to different painters than the target original oil painting image in the training batch are used as the second negative sample set. A second negative sample pair is formed based on the second anchor sample and each sample in the second negative sample set. The texture-style invariant contrast loss corresponding to the target original oil painting image is obtained by calculating the InfoNCE loss function on each of the second positive sample pairs and each of the second negative sample pairs. The global sentiment-content alignment contrast loss is obtained by averaging the sentiment-content alignment contrast loss corresponding to each original oil painting image in the training batch; the global texture-style invariant contrast loss is obtained by averaging the texture-style invariant contrast loss corresponding to each original oil painting image in the training batch; and the contrastive learning loss is obtained by combining the global sentiment-content alignment contrast loss and the global texture-style invariant contrast loss.

4. The method according to claim 3, characterized in that, The step of optimizing the network parameters of the dual-branch style recognition model based on the contrastive learning loss to obtain the optimized style recognition model includes: The global sentiment-content alignment contrast loss and the global texture-style invariance contrast loss are weighted and summed to obtain the total loss function. The gradient of the total loss function with respect to the network parameters is obtained by performing gradient calculation on the total loss function using the backpropagation algorithm. The gradient descent optimization algorithm is used to iteratively update the network parameters based on the gradient until the total loss function meets the preset convergence condition, thus obtaining the optimized style recognition model.

5. The method according to claim 2, characterized in that, In the emotional semantic feature extraction branch, the primary multi-scale visual feature map is subjected to channel attention weighting processing to obtain an attention-weighted feature map, including: The primary multi-scale visual feature map is subjected to global average pooling to obtain the channel global statistical vector; The channel global statistical vector is input into the preset fully connected network in the sentiment semantic feature extraction branch for nonlinear transformation processing to obtain the channel weight vector; The channel weight vector is multiplied channel by channel with the primary multi-scale visual feature map to obtain the attention-weighted feature map.

6. The method according to claim 2, characterized in that, The step of grouping and calculating the Gram matrix of the feature maps at multiple different levels yields multiple sets of texture statistical descriptions, including: The channel dimension of the feature map at each level is evenly divided into a preset number of channel groups; For each of the channel groups, the correlation matrix between the feature channels within the group is calculated to obtain the Gram matrix corresponding to the channel group; Flatten the Gram matrices corresponding to the feature maps at the same level to obtain the texture description vectors corresponding to the feature maps at the same level; The texture description vectors corresponding to the feature maps at each level are collected to obtain the multiple sets of texture statistical descriptions.

7. The method according to claim 6, characterized in that, The mathematical expression for the Gram matrix corresponding to the channel group is: in, For the first The first in the hierarchical feature map The Gram matrix corresponding to each channel group has a dimension of ; For feature-level indexing, The total number of layers in the multi-scale feature map; k is the channel grouping index, k= ,K K is the quantity at each level; For the first The first in the hierarchical feature map The number of channels, and satisfying , Indicates the total number of channels; For the first The first in the hierarchical feature map Feature matrices of each channel group; This indicates a dimension reshaping operation, used to convert a 3D feature map into a 2D matrix.

8. A system for representing the stylistic features of oil paintings based on contrastive learning, characterized in that, The system includes: The feature extraction module is used to perform bi-branch feature extraction processing on the original oil painting image using a preset bi-branch style recognition model to obtain emotional semantic feature vectors and visual texture feature vectors; the bi-branch style recognition model includes a shared convolutional neural network backbone, an emotional semantic feature extraction branch, and a visual texture feature extraction branch; The model optimization module is used to construct a contrastive learning loss based on the emotional semantic feature vector and the visual texture feature vector, and to optimize the network parameters of the dual-branch style recognition model according to the contrastive learning loss to obtain the optimized style recognition model. The style expression module is used to perform dual-branch feature extraction processing on the target oil painting image based on the optimized style recognition model, and then concatenate the target emotional semantic feature vector and the target visual texture feature vector to obtain the target structured style feature expression.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images