A sketch work rating method, system, electronic device and storage medium

Abstract

The application discloses a sketch work rating method and system, electronic equipment and a storage medium. The four-stage multi-scale visual feature extraction is performed on the to-be-rated sketch work to obtain a third feature map and a fourth feature map. The multi-head attention feature map transformation is performed on the fourth feature map to obtain a position feature map. The circular linked list channel sorting and convolution processing are performed on the channel of the third feature map, and the information aggregation is performed on each channel information of the fourth feature map. The feature fusion is performed according to the processing results of the third feature map and the fourth feature map to obtain a channel feature map. The position feature map and the channel feature map are rated respectively, and the two rating results are weighted and calculated to obtain a final rating result. The embodiment of the application can pay attention to the stroke sequence and channel dependency in the sketch work, efficiently and more accurately rate the highly abstract and high-similarity sketch work, and has a positive and beneficial effect. The application can be widely applied to the field of computer technology.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, system, electronic device, and storage medium for grading sketches. Background Technology

[0002] Grading sketches is a crucial part of art exams and a key step for teachers in evaluating the quality of students' sketches. However, sketches are characterized by high abstraction and a high degree of similarity among works of similar grading. With the development of artificial intelligence, intelligent grading of sketches is a promising and challenging task. Currently, research applying computer technology to sketch grading is scarce. Most existing methods focus on identifying or generating artworks through manual feature extraction or deep convolutional feature extraction.

[0003] Early methods used manual design to extract color or texture features from artworks. For example, related technologies proposed a method for extracting painters' artistic styles based on color features to extract key areas of oil paintings; or used heterogeneous feature sets to obtain artistic style feature information of ink paintings; and some researchers studied color and shape feature extraction algorithms based on the characteristics of traditional Chinese painting.

[0004] Thanks to the development of deep learning, deep convolutional neural networks can effectively identify artworks, thereby extracting more abstract features. To capture the local expressive techniques of traditional Chinese paintings for style classification, a multi-scale convolutional neural network-based approach has been proposed. Other related technologies address the problem that existing oil painting style classification algorithms ignore the influence of the main subject area and overall effect on the artistic style, proposing an oil painting classification algorithm based on multi-feature fusion.

[0005] While the algorithms described above can effectively identify and classify artworks, they also have their own shortcomings, mainly:

[0006] (1) These artificially designed features lack good generalization performance and rely on the designer's prior knowledge.

[0007] (2) These deep learning-based art recognition and classification methods are all based on the characteristics of the works. However, sketches in the same exam are usually highly similar and do not have prominent characteristics. Therefore, these recognition and classification methods have low accuracy in recognizing the grade of sketches. Summary of the Invention

[0008] In view of this, embodiments of the present invention provide a highly accurate method, system, electronic device, and storage medium for grading sketches, aiming to solve the problem of low accuracy in grading highly abstract and similar sketches in the prior art.

[0009] One aspect of this invention provides a method for rating sketches, comprising: performing multi-scale visual feature extraction on the sketch to be rated in four stages to obtain a third feature map and a fourth feature map; wherein the third feature map is the feature extraction result of the third stage; the fourth feature map is the feature extraction result of the fourth stage; performing multi-head attention feature map transformation on the fourth feature map based on the stroke position features of the fourth feature map to obtain a position feature map; performing circular linked list channel sorting and convolution processing on the channels of the third feature map to obtain a channel interaction feature map; performing information aggregation on the channel information of each channel of the fourth feature map based on the channel features of the fourth feature map to obtain a channel attention feature map; performing feature fusion on the channel interaction feature map and the channel attention feature map to obtain a channel feature map; rating the position feature map to obtain a first rating result, and rating the channel feature map to obtain a second rating result; and performing a weighted calculation on the first rating result and the second rating result to obtain the final rating result of the sketch to be rated.

[0010] Optionally, the step of performing multi-scale visual feature extraction on the sketch to be graded in four stages to obtain a third feature map and a fourth feature map includes: downsampling the sketch to be graded to obtain an initial feature map; in a first stage, performing a first convolution on the initial feature map to obtain a first feature map; in a second stage, performing a second convolution on the first feature map to obtain a second feature map; in a third stage, performing a third convolution on the second feature map to obtain a third feature map; and in a fourth stage, performing a fourth convolution on the third feature map to obtain a fourth feature map.

[0011] Optionally, the step of performing multi-head attention feature map transformation on the fourth feature map based on the stroke position features of the fourth feature map to obtain a position feature map includes: performing block separation on the fourth feature map through a sliding window to obtain several first position feature sub-maps; performing self-attention processing on the several first position feature sub-maps to obtain several second position feature sub-maps; and merging the second position feature sub-maps into blocks to obtain a position feature map.

[0012] Optionally, the step of performing circular linked list channel sorting and convolution processing on the channels of the third feature map to obtain a channel interaction feature map includes: dividing the channels of the third feature map into several first channel groups; further dividing each first channel group into several first channel subgroups; arranging and combining the first channel subgroups of each first channel group through a circular linked list to obtain several second channel groups; wherein each second channel group includes three second channel subgroups; the second channel subgroups originate from different first channel groups; and performing a fifth convolution processing on all second channel groups to obtain a channel interaction feature map.

[0013] Optionally, the step of aggregating information from each channel of the fourth feature map to obtain a channel attention feature map based on the channel features of the fourth feature map includes: performing global average pooling on each channel information of the fourth feature map to obtain a first channel vector; inputting the first channel vector into a multilayer perceptron to obtain a first channel feature map; and performing a sixth convolution on the first channel feature map to obtain a channel attention feature map.

[0014] Optionally, in the steps of rating the location feature map to obtain a first rating result and rating the channel feature map to obtain a second rating result, the rating step includes: flattening the location feature map or the channel feature map to obtain a one-dimensional feature vector; inputting the one-dimensional feature vector into a multilayer perceptron for level classification to obtain a target rating result; wherein the target rating result includes either the first rating result or the second rating result.

[0015] Optionally, the method further includes the step of training a target sketch rating model, wherein the step of training the target sketch rating model includes: acquiring a training dataset and performing data preprocessing on the training dataset; constructing an initial sketch rating model; training the initial sketch rating model based on the training dataset to obtain a first sketch rating model; and evaluating the first sketch rating model using a multi-classification loss function to obtain a target sketch rating model.

[0016] Embodiments of the present invention also provide a sketch grading system, comprising: a multi-feature extraction module, used to perform multi-scale visual feature extraction on the sketch to be graded in four stages to obtain a third feature map and a fourth feature map; wherein the third feature map is the feature extraction result of the third stage; and the fourth feature map is the feature extraction result of the fourth stage; a positional attention sub-network module, used to perform multi-head attention feature map transformation on the fourth feature map according to the stroke position features of the fourth feature map to obtain a positional feature map; a channel interaction sub-network module, used to perform circular linked list channel sorting and convolution processing on the channels of the third feature map to obtain a channel interaction feature map; also used to perform information aggregation on the channel information of each channel of the fourth feature map according to the channel features of the fourth feature map to obtain a channel attention feature map; and also used to perform feature fusion on the channel interaction feature map and the channel attention feature map to obtain a channel feature map; a grading module, used to grade the positional feature map to obtain a first grading result, and grade the channel feature map to obtain a second grading result; and also used to perform weighted calculation on the first grading result and the second grading result to obtain the final grading result of the sketch to be graded.

[0017] This invention also provides an electronic device, including a processor and a memory; the memory is used to store a program; the processor executes the program to implement the method described above.

[0018] This invention also provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0019] The embodiments of the present invention have the following beneficial effects: By performing multi-scale visual feature extraction on the sketches to be graded in four stages, a third feature map and a fourth feature map are obtained, which can abstract the visual features of the sketches to be graded and present them in the feature maps for easy subsequent processing; based on the stroke position features of the fourth feature map, a multi-head attention feature map transformation is performed on the fourth feature map to obtain a position feature map, which can extract the position of the strokes in the sketch and represent the stroke order; the channels of the third feature map are sorted by a circular linked list and convolutional to obtain a channel interaction feature map; based on the fourth feature map... The invention utilizes the channel features of the fourth feature map to aggregate information from each channel, resulting in a channel attention feature map. This map focuses on channel interaction information and channel dependencies in the sketch. The channel interaction feature map and the channel attention feature map are then fused to obtain a channel feature map. The position feature map is rated to obtain a first rating result, and the channel feature map is rated to obtain a second rating result. A weighted calculation is performed on the first and second rating results to obtain the final rating result for the sketch. This weighted calculation allows for a more targeted approach to the rating based on specific needs. Overall, this invention focuses on stroke order and channel dependencies in sketches, efficiently and accurately rating highly abstract and similar sketches, demonstrating positive and beneficial effects. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the method steps provided in an embodiment of the present invention;

[0022] Figure 2 This is a structural diagram of the sketch evaluation model provided in this embodiment of the invention;

[0023] Figure 3 This is a flowchart of the sketch grading method provided in an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of data preprocessing provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0026] To address the problem of low accuracy in grading highly abstract and similar sketches in existing technologies, this invention provides a method for grading sketches, referring to... Figure 1 , Figure 1 This is a flowchart of the method steps provided in this embodiment of the invention, including: performing multi-scale visual feature extraction on the sketch to be graded in four stages to obtain a third feature map and a fourth feature map; wherein, the third feature map is the feature extraction result of the third stage; the fourth feature map is the feature extraction result of the fourth stage; based on the stroke position features of the fourth feature map, performing multi-head attention feature map transformation on the fourth feature map to obtain a position feature map; performing circular linked list channel sorting and convolution processing on the channels of the third feature map to obtain a channel interaction feature map; based on the channel features of the fourth feature map, performing information aggregation on the information of each channel of the fourth feature map to obtain a channel attention feature map; fusing the channel interaction feature map and the channel attention feature map to obtain a channel feature map; grading the position feature map to obtain a first grading result, and grading the channel feature map to obtain a second grading result; and performing a weighted calculation on the first grading result and the second grading result to obtain the final grading result of the sketch to be graded.

[0027] Specifically, refer to Figure 2 , Figure 2 This is a structural diagram of the sketch evaluation model provided in this embodiment of the invention. This embodiment can construct a sketch evaluation model mainly composed of three parts: a multi-scale feature extraction module, a positional attention sub-network, and a channel interaction sub-network. The positional attention sub-network can extract the composition, structural proportions, and local features of the sketch, while the channel interaction sub-network can extract the detail depth, local features, and expression methods of the sketch. Based on this sketch evaluation model, this embodiment includes the following steps S100–S700:

[0028] S100. Perform multi-scale visual feature extraction on the sketches to be graded in four stages to obtain the third feature map and the fourth feature map; where the third feature map is the feature extraction result of the third stage; and the fourth feature map is the feature extraction result of the fourth stage.

[0029] Specifically, in this embodiment of the invention, ConvNext-tiny, pre-trained on the ImageNet dataset, can be used as the backbone network to perform convolutional extraction on the input sketch, extracting feature maps X0, X1, ..., X at different scales. nAs a multi-scale visual feature; among which, the ImageNet project is a large visualization database used for research on visual object recognition software. Step S100 includes the following steps S110 to S150:

[0030] S110. Downsample the sketches to be graded to obtain an initial feature map.

[0031] Specifically, for example, a 400×400 (400 pixels wide and 400 pixels high) sketch to be graded is downsampled to obtain an initial feature map X0∈R. 96×100×100 Where X0 represents the initial feature map, R represents the feature space, and R0 represents the feature space. 96×100×100 This represents a feature map with 96 channels and a height and width of 96 pixels.

[0032] S120. In the first stage, the initial feature map is subjected to the first convolution process to obtain the first feature map.

[0033] Specifically, in the first stage, the initial feature map X0 is subjected to a first convolution process to obtain the first feature map X1∈R. 96 ×100×100 .

[0034] S130. In the second stage, the first feature map is subjected to a second convolution process to obtain the second feature map.

[0035] Specifically, in the second stage, the first feature map X1 is subjected to a second convolution process to obtain the second feature map X2∈R. 192 ×50×50 .

[0036] S140. In the third stage, the second feature map is subjected to a third convolution process to obtain the third feature map.

[0037] Specifically, in the third stage, the second feature map X2 is subjected to a third convolution process to obtain the third feature map X3∈R. 384 ×25×25 .

[0038] S150. In the fourth stage, the third feature map is subjected to a fourth convolution to obtain the fourth feature map.

[0039] Specifically, in the fourth stage, the third feature map X3 is subjected to a fourth convolution to obtain the fourth feature map X4∈R. 768 ×12×12 .

[0040] It should be noted that the first convolution process, the second convolution process, the third convolution process, and the fourth convolution process represent convolutions performed using convolution kernels of different lengths.

[0041] S200. Based on the stroke position features of the fourth feature map, perform multi-head attention feature map transformation on the fourth feature map to obtain the position feature map.

[0042] Specifically, in the position attention subnetwork, this embodiment of the invention first processes the multi-scale visual feature map X∈R c ×h×w As the input to the input location attention subnetwork, where c represents the number of channels, h represents the height, w represents the width, and R... c×h×w This represents a feature map with channel c, height h pixels, and width w pixels. X can be the feature map output of the fourth feature layer of the backbone network, i.e., the fourth feature map X4∈R. 768×12×12 Step S200 includes the following steps S210 to S230:

[0043] S210. The fourth feature map is divided into blocks by a sliding window to obtain several first position feature sub-maps.

[0044] Specifically, in the block separation stage, a 2×2 sliding window is constructed in the block separation layer to process the fourth feature map X4. Each small block within the 2×2 sliding window is assigned to four different small feature maps. The sliding window then moves forward two steps until the fourth feature map X4 has been processed. After passing through the block separation layer, the fourth feature map X4 becomes four first-position feature sub-maps F1, F2, F3, and F4. The height and width of each first-position feature sub-map are reduced to half of their original values, while the depth remains unchanged. These four feature maps integrate information from different positions within the fourth feature map X4.

[0045] S220. Perform self-attention processing on several first position feature sub-images to obtain several second position feature sub-images.

[0046] Specifically, after the block separation stage, a self-attention operation is performed on F1, F2, F3, and F4 to obtain F1′, F2′, F3′, and F4′. i ′ can be represented as:

[0047]

[0048] Among them, F i ′ represents the second positional feature subgraph; i∈{1,2,3,4}; Q i It is F i The query, K i It is feature map F i The weight, V i It is feature map F i The value matrix, F i Represents the first positional feature sub-map; d iF1 is the scaling factor; B is the relative position encoding. F1, F2, F3, and F4 are used as the four heads of the self-attention mechanism, and each head shares the relative position encoding. The shared relative position encoding can connect the four first position feature sub-maps, expressing the relative positional relationship of the first position feature sub-maps.

[0049] S230. Merge the second position feature sub-map into blocks to obtain the position feature map.

[0050] Specifically, block merging is the opposite operation of block separation. Given four second positional feature sub-maps F1′, F2′, F3′, and F4′, the corresponding positions of the second positional feature sub-maps F1′, F2′, F3′, and F4′ are concatenated to obtain the positional feature map X′. The expression for X′ is:

[0051] X' = ​​concat(F t (F2,F3,F4)

[0052] Where X′ represents the location feature map, and concat() represents the concatenation operation.

[0053] Step S200 enhances the interaction of information at different locations on the feature map and strengthens the exchange of location information, highlighting the stroke order in sketching.

[0054] S300. Perform circular linked list channel sorting and convolution processing on the channels of the third feature map to obtain the channel interaction feature map.

[0055] Specifically, step S300 includes steps S310 to S340:

[0056] S310. Divide the channels of the third feature map into several first channel groups.

[0057] Specifically, the channels of the third feature map X3 are divided into n first channel groups.

[0058] S320. Each first channel group is further divided into several first channel subgroups.

[0059] Specifically, each first channel group is further divided into n first channel subgroups. It should be noted that the number of first channel subgroups within each first channel group is the same and equal to the number of first channel groups.

[0060] S330. Arrange and combine the first channel subgroups of each first channel group through a circular linked list to obtain several second channel groups; wherein each second channel group includes three second channel subgroups; the second channel subgroups come from different first channel groups.

[0061] Specifically, a circular linked list is constructed. Each first channel group selects each first channel subgroup according to the head pointer of the circular linked list, forming a second channel group. It can be understood that after arranging and combining channels through the circular linked list, each second channel group includes three second channel subgroups, which originate from different first channel groups. Each second channel group is not only related to its corresponding first channel group but also to other first channel groups, strengthening the information flow between channel groups.

[0062] S340. Perform a fifth convolution on all second channel groups to obtain the channel interaction feature map.

[0063] Specifically, 3×3 group convolutions can be used to aggregate the information of all second channel groups to obtain the channel interaction feature map N.

[0064] S400. Based on the channel features of the fourth feature map, information aggregation is performed on the information of each channel of the fourth feature map to obtain the channel attention feature map.

[0065] Specifically, step S400 includes the following steps S410 to S430:

[0066] S410. Perform global average pooling on the information of each channel of the fourth feature map to obtain the first channel vector.

[0067] Specifically, the information from each channel in the aggregated feature map X is used, where feature map X can be a fourth feature map X4. More specifically, global average pooling (GAP) is performed on the information from each channel of the feature map to obtain the first channel vector U, whose expression is:

[0068]

[0069] Where U is the first channel vector, H is the height of feature map X, and W is the width of feature map X.

[0070] S420. Input the first channel vector into the multilayer perceptron to obtain the first channel feature map.

[0071] Specifically, the first channel vector U is input into a multilayer perceptron (Mlp), and processed using the sigmoid activation function σ in the fully connected layer to obtain the intermediate channel feature map U′. ​​The expression for U′ is:

[0072] U′=σ(Mlp(U))

[0073] Where U′ is the intermediate channel feature map; σ represents the activation function; and Mlp() represents the multilayer perceptron.

[0074] To highlight the importance of each channel, the intermediate channel feature map U′ is multiplied and weighted channel by channel onto the original feature map X to obtain the first channel feature map U″. The expression for the first channel feature map U″ is as follows:

[0075] U″=X·U'

[0076] Where U″ represents the first channel feature map; X represents the input feature map X, which can be the fourth feature map X4.

[0077] S430. Perform a sixth convolution on the first channel feature map to obtain the channel attention feature map.

[0078] Specifically, the first channel feature map U″ is subjected to a sixth convolution with a kernel of 1×1, and U″ is aggregated to obtain the channel attention feature map M.

[0079] Through the channel feature aggregation process in step S400, the channel signal of each channel of the feature map can be taken into account, thus expressing the channel dependency of the feature map of the sketch.

[0080] S500. The channel interaction feature map and the channel attention feature map are fused to obtain the channel feature map.

[0081] Specifically, the channel interaction feature map N and the channel attention feature map M are fused using the add feature fusion method to obtain the channel feature map Z. The expression for Z is:

[0082]

[0083] Where Z represents the channel feature map, M i Represents the channel attention feature map, N i This represents the channel interaction feature map.

[0084] It should be noted that `add` is a feature fusion method, which mainly increases the amount of information in each dimension of the image features while keeping the dimensionality unchanged. In other embodiments, the `concat` feature fusion method can be used for feature fusion.

[0085] The channel feature map obtained by feature fusion removes redundant information from the channels and has stronger semantic information features.

[0086] S600: The location feature map is rated to obtain the first rating result, and the channel feature map is rated to obtain the second rating result.

[0087] Specifically, step S600 includes the following steps S610 to S620:

[0088] S610. Flatten the position feature map or channel feature map to obtain a one-dimensional feature vector.

[0089] Specifically, flattening involves expanding a two-dimensional feature map into a one-dimensional feature vector by rows or columns, which facilitates subsequent calculations and processing.

[0090] S620. Input the one-dimensional feature vector into the multilayer perceptron for hierarchical classification to obtain the target rating result; wherein, the target rating result includes either the first rating result or the second rating result.

[0091] S700: The first and second rating results are weighted and calculated to obtain the final rating result of the sketch to be rated.

[0092] Specifically, for example, the weight value of both can be set to 0.5, which indicates that the first rating result and the second rating result have equal roles in the rating process.

[0093] Embodiments of the present invention may further include step S800: training a target sketch evaluation model.

[0094] S800, training target sketch model.

[0095] Specifically, refer to Figure 3 , Figure 3 This is a flowchart of the sketch grading method provided in this embodiment of the invention. Step S800 includes the following steps S810 to S840:

[0096] S810. Obtain the training dataset and perform data preprocessing on the training dataset.

[0097] Specifically, refer to Figure 4 , Figure 4 This is a schematic diagram of data preprocessing provided in an embodiment of the present invention. The embodiment of the present invention expands the dataset by performing at least one of the following methods on the original image: horizontal flipping, vertical flipping, color jittering, Gaussian blurring, random grayscale transformation, and random erasure, so as to eliminate the imbalance of the training dataset.

[0098] S820. Construct an initial sketch grading model.

[0099] Specifically, the initial sketch grading model works as described in steps S100 to S700 above.

[0100] S830. Train the initial sketch rating model based on the training dataset to obtain the first sketch rating model.

[0101] Specifically, using the training dataset as input, the initial sketch grading model is trained on a single GPU (Graphics Processing Unit) for 10 mini-batch iterations, with the model generated in each iteration serving as the first sketch grading model.

[0102] S840. The rating model of the first sketch is evaluated by a multi-classification loss function to obtain the rating model of the target sketch.

[0103] Specifically, during the training process, a multi-class loss function is used as the training loss function to evaluate the first sketch rating model. Backpropagation algorithm is used as the algorithm for iteratively updating parameters with a learning rate of 0.01. Adadelta is used as the model optimizer to optimize the first sketch rating model. The model is trained for 600 iterations, and the last 5 iterations are used as the final result. The best performing result is selected as the target sketch rating model.

[0104] The trained target sketch rating model can efficiently and accurately rate the input sketches through steps such as S100 to S700.

[0105] This invention also provides a sketch grading system, comprising: a multi-feature extraction module for performing multi-scale visual feature extraction on the sketch to be graded in four stages to obtain a third feature map and a fourth feature map; wherein the third feature map is the feature extraction result of the third stage; and the fourth feature map is the feature extraction result of the fourth stage; a positional attention sub-network module for performing multi-head attention feature map transformation on the fourth feature map based on the stroke position features of the fourth feature map to obtain a positional feature map; a channel interaction sub-network module for performing circular linked list channel sorting and convolution processing on the channels of the third feature map to obtain a channel interaction feature map; further configured to perform information aggregation on the channel information of each channel of the fourth feature map based on the channel features of the fourth feature map to obtain a channel attention feature map; further configured to perform feature fusion on the channel interaction feature map and the channel attention feature map to obtain a channel feature map; and a grading module for grading the positional feature map to obtain a first grading result and grading the channel feature map to obtain a second grading result; and further configured to perform weighted calculation on the first grading result and the second grading result to obtain the final grading result of the sketch to be graded.

[0106] This invention also provides an electronic device, including a processor and a memory; the memory is used to store a program; the processor executes the program to implement the method described above.

[0107] This invention also provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0108] The embodiments of the present invention have the following beneficial effects:

[0109] 1. The embodiments of the present invention can extract the stroke position features and channel interaction relationships in sketches, focus on the stroke order and channel dependence in sketches, and efficiently and more accurately rate highly abstract and similar sketches.

[0110] 2. The target sketch evaluation model constructed and trained in the embodiments of the present invention has the beneficial effects of simple structure, strong generalization ability and robust model, and can better evaluate sketches.

[0111] The technical effects of the embodiments of the present invention are described below with reference to experimental data:

[0112] First, 2242 sketches from the same batch of exams were collected to construct a training dataset. This dataset included 671 sketches of grade A, 708 of grade B, 745 of grade C, and 125 of grade D. Due to the imbalance in the number of sketches across each grade in this training dataset, and specifically addressing the limited number of grade D sketches, this embodiment of the invention employs data augmentation techniques for preprocessing the samples. These data augmentations include horizontal flipping, vertical flipping, color jittering, and Gaussian blur. Horizontal and vertical flipping expand the training sample size, resulting in a more balanced number of sketches across each grade and a more robust target model. Color jittering reduces the target model's sensitivity to color and brightness, while Gaussian blurring enhances the model's generalization ability.

[0113] Referring to Table 1, Table 1 is a comparison table of the performance of the embodiments of the present invention with the ResNet (Residual Network) series of methods, DenseNet series of methods, ConvNeXt series of methods, ResNeSt series of methods, ReNeXt series of methods, ViT series of methods and Swin series of methods in the recognition of sketch artwork grades.

[0114] Table 1

[0115]

[0116]

[0117] The various methods listed in Table 1, excluding those in the embodiments of the present invention, are all classic deep learning methods. Models were built based on these methods, using the same training set. These models were then used for sketch recognition and compared with the methods in the embodiments of the present invention. As can be seen from Table 1, the embodiments of the present invention have superior performance in accuracy, recall, and F1 score compared to other methods in sketch recognition.

[0118] Furthermore, to verify the effectiveness of different components of the method in this embodiment of the invention, ablation experiments were conducted according to the method shown in Table 2, which is a table of ablation experiment results for different components in this embodiment of the invention. Specifically, the neural network selected by the multi-scale feature extraction module in this embodiment of the invention was changed, or the combination of the selected neural network with the position attention sub-network processing and channel interaction sub-network processing in this embodiment of the invention was changed.

[0119] Table 2

[0120]

[0121] The component ablation test results in Table 2 show that ConvNeXt-T achieves an accuracy similar to ResNet50 and is 1% higher than Swin-T. Since the network structure of ConvNeXt-T is simpler than that of Swin-T (Swin-Transformer), this invention uses ConvNeXt-T as the preferred backbone network, i.e., the network of the multi-scale feature extraction module.

[0122] In addition, Table 2 shows that the positional attention subnetwork and the channel interaction subnetwork can effectively improve the model's performance across various metrics. When these two types of subnetworks are embedded into the backbone network ConvNeXt-T, accuracy, recall, and F1 score are significantly improved. In particular, the method in this embodiment of the invention, which includes the positional attention subnetwork and the channel interaction subnetwork, outperforms the backbone network in terms of accuracy, recall, and F1 score, demonstrating the effectiveness of this embodiment.

[0123] Referring to Table 3, which shows the ablation experiment results for different methods in the model structure provided in the embodiments of the present invention:

[0124] Table 3

[0125]

[0126] Table 3 presents ablation experiments conducted on different implementations of the model structure. First, under different positional encoding methods, the positional attention subnetwork with relative positional encoding achieved +3.8% precision, +3.2% recall, and +3.7% F1 score compared to no positional encoding, while achieving +2.0% precision, +1.8% recall, and +2.6% F1 score compared to absolute positional encoding. Second, in the circular linked list channel sorting, the experiment compared the effects of different numbers of groups. The results showed that when the number of groups increased from 2 to 4, the precision decreased by 0.7%; when the number of groups increased to 6, all indicators reached their optimal levels, around 80%. This indicates that the circular linked list channel sorting in this embodiment is effective, and increasing the number of groups can enhance the communication of channel information, improve the accuracy of sketch grading, and strengthen the robustness of the model. Finally, the performance of different feature fusion methods was verified in the channel interaction sub-network. It can be seen that the add feature fusion method outperforms the concat feature fusion method in terms of accuracy. Therefore, the embodiment of the present invention uses the add feature fusion method as the optimal feature fusion method, which increases the amount of information in each dimension and highlights the ability to express the details of sketches in depth during the sketch evaluation process.

[0127] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0128] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0129] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0130] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0131] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0132] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for grading sketches, characterized in that, include: The sketches to be graded undergo multi-scale visual feature extraction in four stages to obtain a third feature map and a fourth feature map; wherein, the third feature map is the feature extraction result of the third stage; and the fourth feature map is the feature extraction result of the fourth stage. Based on the stroke position features of the fourth feature map, a multi-head attention feature map transformation is performed on the fourth feature map to obtain a position feature map; The channels of the third feature map are sorted by a circular linked list and then convolved to obtain a channel interaction feature map. Based on the channel features of the fourth feature map, information is aggregated for each channel of the fourth feature map to obtain a channel attention feature map; The channel interaction feature map and the channel attention feature map are fused to obtain the channel feature map; The location feature map is rated to obtain a first rating result, and the channel feature map is rated to obtain a second rating result; The first rating result and the second rating result are weighted and calculated to obtain the final rating result of the sketch to be rated; The step of performing multi-head attention feature map transformation on the fourth feature map based on the stroke position features of the fourth feature map to obtain a position feature map includes: The fourth feature map is divided into blocks by a sliding window to obtain several first position feature sub-maps; Self-attention processing is performed on several first position feature sub-images to obtain several second position feature sub-images; The second location feature sub-map is merged into blocks to obtain the location feature map; The process of sorting and convolving the channels of the third feature map using a circular linked list to obtain a channel interaction feature map includes: The channels of the third feature map are divided into several first channel groups; Each of the first channel groups is further divided into several first channel subgroups; By arranging and combining the first channel subgroups of each first channel group using a circular linked list, several second channel groups are obtained; each second channel group includes three second channel subgroups; the second channel subgroups originate from different first channel groups; A fifth convolution is performed on all the second channel groups to obtain the channel interaction feature map.

2. The method for grading sketches according to claim 1, characterized in that, The sketches to be graded undergo a four-stage multi-scale visual feature extraction process to obtain the third and fourth feature maps, including: The sketches to be graded are downsampled to obtain initial feature maps; In the first stage, the initial feature map is subjected to a first convolution process to obtain a first feature map; In the second stage, the first feature map is subjected to a second convolution process to obtain a second feature map; In the third stage, the second feature map is subjected to a third convolution process to obtain a third feature map; In the fourth stage, the third feature map is subjected to a fourth convolution process to obtain a fourth feature map.

3. The method for grading sketches according to claim 1, characterized in that, The step of aggregating information from each channel of the fourth feature map based on its channel features to obtain a channel attention feature map includes: Global average pooling is performed on each channel information of the fourth feature map to obtain the first channel vector; The first channel vector is input into the multilayer perceptron to obtain the first channel feature map; The first channel feature map is subjected to a sixth convolution to obtain the channel attention feature map.

4. The method for grading sketches according to claim 1, characterized in that, The step of rating the location feature map to obtain a first rating result and rating the channel feature map to obtain a second rating result includes the following steps: Flatten the location feature map or the channel feature map to obtain a one-dimensional feature vector; The one-dimensional feature vector is input into a multilayer perceptron for classification to obtain a target rating result; wherein the target rating result includes either a first rating result or a second rating result.

5. The method for grading sketches according to claim 1, characterized in that, The method further includes the step of training a target sketch rating model, the step of training the target sketch rating model includes: Obtain the training dataset and perform data preprocessing on the training dataset; Construct an initial sketch evaluation model; The initial sketch rating model is trained based on the training dataset to obtain the first sketch rating model. The target sketch rating model is obtained by evaluating the first sketch rating model using a multi-classification loss function.

6. A system for implementing the sketch grading method as described in any one of claims 1-5, characterized in that, include: The multi-feature extraction module is used to perform multi-scale visual feature extraction on the sketches to be graded in four stages, resulting in a third feature map and a fourth feature map; wherein, the third feature map is the feature extraction result of the third stage; and the fourth feature map is the feature extraction result of the fourth stage. The positional attention sub-network module is used to perform multi-head attention feature map transformation on the fourth feature map based on the stroke position features of the fourth feature map to obtain the positional feature map; The channel interaction sub-network module is used to perform circular linked list channel sorting and convolution processing on the channels of the third feature map to obtain a channel interaction feature map; it is also used to aggregate information of each channel information of the fourth feature map according to the channel features of the fourth feature map to obtain a channel attention feature map; and it is also used to fuse the channel interaction feature map and the channel attention feature map to obtain a channel feature map. The rating module is used to rate the location feature map to obtain a first rating result and to rate the channel feature map to obtain a second rating result; it is also used to perform a weighted calculation on the first rating result and the second rating result to obtain the final rating result of the sketch to be rated.

7. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 5.

Images