Artwork evaluation method and device based on structural deformation, equipment and medium

Abstract

The application provides an art work evaluation method and device based on structural deformation, equipment and medium, belonging to the technical field of education evaluation, the method comprises the following steps: extracting the structural features of the first image of the art work; comparing the structural features with the standard structure template to obtain the first structural deformation features; inputting the first structural deformation features and the visual features of the first image into a multi-modal large language model to obtain a fusion feature vector; the loss function of the multi-modal large language model comprises a structural deformation loss; generating an evaluation result of the art work based on the fusion feature vector and outputting the evaluation result. By comparing the structural features of the art work with the standard structure template, the structural deformation features are obtained, and the evaluation is carried out based on the fusion feature vector of the structural deformation features and the visual features, so that the structural deformation features are used as the professional basis in the evaluation process, the core art problems are identified, and the accuracy of the evaluation is improved.

Description

Technical Field

[0001] This application relates to the field of educational assessment technology, and in particular to a method, apparatus, equipment and medium for evaluating artworks based on structural deformation. Background Technology

[0002] With the rapid application of Artificial Intelligence (AI), Computer Vision (CV), and Multimodal Large Language Model (MLLM) in the field of educational assessment, the marking of art exams in the National College Entrance Examination (NCEE) is gradually transforming towards intelligentization.

[0003] Current intelligent art exam marking systems rely on traditional convolutional neural networks (CNNs) or general multimodal large language models. They have not designed a dedicated architecture for the professional characteristics of art exam papers. Therefore, they can only capture the surface visual features of the works and cannot identify the core art problems in the works, resulting in a lack of professional basis for scoring and low scoring accuracy. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, and medium for evaluating artworks based on structural deformation, aiming to solve the problem that current intelligent art grading systems can only capture the surface visual features of artworks and cannot identify the core art problems in the artworks, resulting in a lack of professional basis for scoring and low scoring accuracy.

[0005] Firstly, this application provides a method for evaluating artworks based on structural deformation, including: Extracting the structural features of the first image of an artwork; The structural features are compared with a standard structural template to obtain the first structural deformation features; the standard structural template is the structural reference corresponding to the artwork. Visual features of the first image are extracted, and the first structural deformation features and the visual features are input into a multimodal large language model to obtain a fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the second structural deformation features of artwork samples in the training dataset; An evaluation result for the artwork is generated based on the fused feature vector, and the evaluation result is output.

[0006] As one embodiment, comparing the structural features with a standard structural template to obtain the first structural deformation feature includes: Align the structural features with the key points in the standard structural template; Based on the alignment results, at least one structural deformation is evaluated, and the corresponding structural deformation index is determined. All structural deformation indices are combined into the first structural deformation feature. The at least one structural deformation includes perspective deformation, proportional deformation, and gray-black deformation.

[0007] As one embodiment, generating the evaluation result of the artwork based on the fused feature vector includes: Extract the fused semantic representation from the fused feature vector; Using a score prediction head, the initial comprehensive score and initial sub-scores of the artwork samples are obtained based on the fused semantic representation. Each initial sub-score is determined based on the structural deformation index associated with the initial sub-score. Using a comment generation head, an initial comment on the artwork is obtained based on the fused semantic representation, wherein at least a portion of the initial comment is obtained based on the at least one structural deformation index; The evaluation result of the artwork is generated based on the initial overall score, the initial sub-scores, and the initial comments.

[0008] As one embodiment, the evaluation result of the artwork is generated based on the initial overall score, the initial sub-scores, and the initial comments, including: Extract defect keywords from the initial comments and map the defect keywords to the sub-item mapping scores of the target structural deformation corresponding to the defect keywords; If there is a logical conflict between the sub-item mapping score of the target structural deformation and the initial sub-item score and the initial comprehensive score of the target structural deformation, the defect keyword, the initial sub-item score and the initial comprehensive score shall be adjusted to be logically consistent. The final overall score, final sub-scores, and final comments are generated based on the adjustment results. The final overall score, the final sub-scores, and the final comments are combined to form the evaluation result of the artwork.

[0009] As one embodiment, the score prediction head includes a shared fully connected layer, a score mapping layer, and a regression branch corresponding to each of the structural deformation indices; Using a score prediction head, based on the fused semantic representation, the initial comprehensive score and initial sub-scores of the artwork samples are obtained, including: The fused semantic representation is input into the shared fully connected layer for dimensionality reduction processing to obtain a dimensionality-reduced semantic representation. The dimensionality-reduced semantic representation is input into the regression branch corresponding to each of the structural deformation indices to obtain the individual scores corresponding to the structural deformation indices. Each of the individual scores is input into the score mapping layer, and the score mapping layer is used to map the individual scores to the initial sub-scores. The initial comprehensive score is obtained based on each of the initial sub-scores.

[0010] As one example, the initial evaluation includes descriptions of advantages, descriptions of defects, and suggestions for improvement; Using the comment generation head, initial comments on the artwork are obtained based on the fused semantic representation, including: Extract structural deformation indices that exceed the deformation threshold from the fused semantic representation as the first target structural deformation index; extract structural deformation indices that do not exceed the deformation threshold from the fused semantic representation as the second target structural deformation index. The glossary of art terms is invoked to map the deformation index of the first target structure to defect description information, and improvement suggestions are generated based on the defect description information. The art terminology glossary is invoked to map the second target structural deformation index into advantage description information; The advantages, the defects, and the improvement suggestions are combined to form the initial evaluation.

[0011] As one embodiment, the initial evaluation includes descriptions of advantages, descriptions of defects, and suggestions for improvement; the evaluation generation head includes a multi-head attention layer and a decoding layer; Using the comment generation head, initial comments on the artwork are obtained based on the fused semantic representation, including: Based on the relationship between each of the structural deformation indices and the deformation threshold of the structural deformation indices, the guiding semantics of each of the structural deformation indices are determined, and the guiding semantics are concatenated with the fused semantic representation to form an input sequence; The input sequence is input into the multi-head attention layer to obtain multiple attention scores; The multiple attention scores are input into the decoding layer, which then calls the art terminology glossary to generate the advantage description information, the defect description information, and the improvement suggestions. The decoding layer then calls the comment structure template to generate the initial comment based on the advantage description information, the defect description information, and the improvement suggestions.

[0012] As one embodiment, training the multimodal large language model includes: Obtain the training dataset, wherein each artwork sample in the training dataset includes a second image of the artwork sample, the second structural deformation feature corresponding to the second image, a score label, and a comment label; The structural deformation loss of the artwork sample is determined based on the second structural deformation feature and the standard structural template corresponding to the artwork sample. The artwork sample is input into an initial multimodal large language model to obtain the predicted score and predicted comment output by the initial multimodal large language model. The score prediction loss is obtained based on the predicted score and the score label, and the text generation loss is obtained based on the predicted comment and the comment label. A total loss function is constructed based on the structural deformation loss, the score prediction loss, and the text generation loss. The initial multimodal large language model is iteratively trained with the goal of minimizing the total loss function to obtain the multimodal large language model base. The multimodal large language model base is fine-tuned based on the mapping relationship between the second structural deformation features of each artwork sample and the comment annotations to obtain a trained multimodal large language model.

[0013] Secondly, this application also provides an art evaluation device based on structural deformation, comprising: The structural feature extraction module is used to extract the structural features of the first image of an artwork. The structural deformation feature acquisition module is used to compare the structural features with a standard structural template to obtain the first structural deformation feature; the standard structural template is the structural reference corresponding to the artwork. The fusion module is used to extract the visual features of the first image and input the first structural deformation features and the visual features into the multimodal large language model to obtain the fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the first structural deformation features of the artwork samples in the training dataset; The evaluation module is used to generate an evaluation result for the artwork based on the fused feature vector, and output the evaluation result.

[0014] Thirdly, this application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the aforementioned methods for evaluating artworks based on structural deformation.

[0015] Fourthly, this application also provides a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the aforementioned methods for evaluating artworks based on structural deformation.

[0016] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements any of the aforementioned methods for evaluating artworks based on structural deformation.

[0017] This application provides a method, apparatus, device, and medium for evaluating artworks based on structural deformation. By comparing the structural features of the artwork with a standard structural template, structural deformation features are obtained. Evaluation is then conducted based on the fusion feature vector of structural deformation features and visual features. In this way, the structural deformation features serve as a professional basis for the evaluation process, identifying core art problems and improving the accuracy of the evaluation. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the art evaluation method based on structural deformation provided in this application; Figure 2 This is the second flowchart of the art evaluation method based on structural deformation provided in this application; Figure 3 This is a flowchart illustrating the process of comparing the structural features with a standard structural template to obtain structural deformation features and structural deformation loss, as provided in this application. Figure 4 This is a schematic diagram of the training process for the multimodal large language model provided in this application; Figure 5 This is a schematic diagram of the process for generating the evaluation result of the artwork based on the fused feature vector provided in this application; Figure 6 This is a structural schematic diagram of the art evaluation device based on structural deformation provided in this application; Figure 7 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0022] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0023] The application scenarios of this application include, but are not limited to, grading of mock college entrance examination art tests, teaching assistance for art training institutions, self-assessment of art students' practice, and assistance in grading college entrance examination art papers.

[0024] The following description, in conjunction with the accompanying drawings, describes the method, apparatus, equipment, and medium for evaluating artworks based on structural deformation provided in this application.

[0025] It should be noted that the art evaluation method based on structural deformation provided in this application embodiment is implemented using an art evaluation device based on structural deformation. The art evaluation method based on structural deformation can output scores and professional comments simultaneously, reducing the burden of manual grading. At the same time, it breaks the black-box modeling, provides concrete guidance, and helps art learners understand their own shortcomings and clarify the direction for improvement.

[0026] This application describes the art evaluation method based on structural deformation using an art evaluation device based on structural deformation as an example.

[0027] Figure 1 This is one of the flowcharts illustrating the art evaluation method based on structural deformation provided in this application. For example... Figure 1 As shown, the art evaluation method based on structural deformation provided in this application includes: S110: Extract the structural features of the first image of the artwork.

[0028] The first image refers to the digital image of the artwork to be evaluated, which can be obtained through an image input interface, such as by inputting an image of an art exam paper through a document camera or scanner.

[0029] Understandably, such as Figure 2 As shown, before extracting the structural features of the first image, the first image is preprocessed to adapt to the characteristics of the artwork, providing a stable and reliable input for the subsequent accurate extraction of structural features.

[0030] In one possible implementation, the preprocessing involves three sub-steps. First, image normalization is performed, scaling first images of different sizes and resolutions to a fixed size (e.g., 448×448 pixels) and performing brightness normalization and contrast normalization. This normalization process eliminates interference from uneven lighting and tilted angles during shooting, ensuring the comparability of features across different first images. Second, noise removal is performed using algorithms such as Gaussian filtering to remove shooting noise and paper texture noise from the first images, while preserving the core lines, colors, and light and shadow features of the artwork, preventing noise from misleading subsequent structural recognition. Finally, scene adaptation is performed, using a lightweight classifier to identify the artwork category, which includes sketches, color paintings, and quick sketches. Scene adaptation provides a category basis for subsequent standard structural template matching; for example, sketch images will be guided to an analysis path focusing on perspective and proportion. After the above preprocessing, the structural features of the image are more easily captured accurately.

[0031] After preprocessing, the Scale-Invariant Feature Transform (SIFT) algorithm is used to extract key points from the preprocessed first image, and the coordinates and connections of these key points are encoded as structural features. These key points form the basis of the structural features and include perspective key points, human skeleton points, and light-dark boundary line key points.

[0032] S120: Compare the structural features with the standard structural template to obtain the first structural deformation feature, wherein the standard structural template is the structural reference corresponding to the artwork.

[0033] like Figure 2As shown, the standard structure templates are pre-built professional benchmark data that conform to art grading standards and are stored in the standard template library. The standard template library is specifically designed for artworks, discarding irrelevant general image templates and containing only three main categories of standard templates. Specifically, the standard structure templates for sketching include standard structures for portrait and still life sketches, covering standard images from different angles (front view, three-quarter view), key point coordinates, black, white, and gray distribution standards, and perspective standards. The standard structure templates for color include standard color distribution, light and dark contrast standards, and warm and cool relationship standards for gouache and watercolor works. The standard structure templates for sketching include standards for human proportions, center of gravity, and line expression for figure and scene sketches. Each standard structure template is associated with the specific structural requirements of the three levels of art grading standards: excellent, good, and satisfactory.

[0034] In this step, the corresponding standard structural template is retrieved from the locally stored standard template library according to the work category. After obtaining the corresponding standard structural template, the structural features of the first image obtained above are compared with the structure in the standard structural template. The first structural deformation feature is quantified based on the deviation between the structural features of the first image and the structure of the standard structural template.

[0035] In one possible implementation, the first structural deformation feature is a multi-dimensional vector that encompasses core indicators such as perspective deformation, proportional deformation, and grayscale deformation. The value of the first structural deformation feature directly reflects the gap between the artwork and professional standards.

[0036] S130: Extract the visual features of the first image, and input the first structural deformation feature and the visual features into the multimodal large language model to obtain the fused feature vector output by the multimodal large language model. The loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the second structural deformation feature of the artwork samples in the training dataset.

[0037] Visual features are characteristics that reflect the surface information of an artwork, including line quality, color distribution, contrast, and texture. A visual encoder (such as ResNet or ViT) can be used to extract features from the first image, generating a high-dimensional visual feature vector.

[0038] Subsequently, the first structural deformation feature is concatenated with the visual feature vector to construct a fused input vector, which contains both the surface visual information and the deep structural information of the artwork.

[0039] In this embodiment, the multimodal large language model is obtained through iterative training on a training dataset used for art evaluation. Each artwork sample in the training dataset is labeled with real structural deformation features. The structural deformation loss is obtained by calculating the difference (e.g., mean squared error) between the structural deformation features predicted by the model and the real structural deformation features labeled in the samples. Through the backpropagation algorithm, this structural deformation loss is used to adjust the model parameters, forcing the model to focus on and accurately reflect structural deformation information when generating fused feature vectors.

[0040] When the fused input vector of the artwork to be evaluated is input, the fused feature vector output by the model is a high-level abstraction of the artwork's semantics, naturally containing precise structural analysis information.

[0041] S140: Generate an evaluation result for the artwork based on the fused feature vector, and output the evaluation result.

[0042] In one possible implementation, the Support Vector Regression (SVR) algorithm is used to process the fused feature vectors to achieve joint text and image review, such as... Figure 2 As shown.

[0043] Specifically, the SVR model is pre-trained using a large number of pre-labeled artwork samples to establish a non-linear mapping relationship between the fused feature vector and the artwork quality score. During the inference phase, the fused feature vector is input into the trained SVR model, and the model outputs a score value, which is the core component of the evaluation result.

[0044] In addition to numerical scoring, qualitative evaluations can also be generated based on rule-based logical judgment methods using the structural deformation components in the fused feature vector. For example, the processor sets a series of thresholds: when the perspective deformation index in the fused feature vector exceeds the first threshold, the work is judged to have a perspective problem; when the black-and-white deformation index exceeds the second threshold, the work is judged to have disordered light and dark relationships. The results of these logical judgments are then combined into textual evaluation opinions.

[0045] Finally, the numerical scores and textual evaluation opinions are packaged into the final evaluation result and output through the output interface.

[0046] This application's embodiments compare the structural features of artworks with standard structural templates to obtain structural deformation features. Evaluation is then performed based on a fused feature vector of structural deformation features and visual features. This allows for the identification of core art problems using structural deformation features as a professional basis during the evaluation process, thus improving the accuracy of the evaluation. According to some examples, the evaluation results of this application accurately match art scoring standards, achieving a consistency of over 0.92 with the scores of senior art examination experts.

[0047] In one possible implementation, such as Figure 3 As shown, in S120, comparing the structural features with a standard structural template to obtain the first structural deformation feature includes: S1210: Align the structural features with the key points in the standard structural template.

[0048] First, extract key points from the structural features of the first image (such as joints of a person, outlines of still objects, etc.) and the corresponding key points in the standard structural template, and then perform key point alignment.

[0049] Keypoint alignment can be rigid or non-rigid. In rigid alignment, the optimal rotation matrix, translation vector, and scaling factor are calculated, and the keypoint coordinates of the first image are mapped to the coordinate system of the standard template through an affine transformation. Rigid alignment eliminates differences in overall position, rotation angle, and scale between the images, placing them on a comparable reference plane. For example, for a sketch image, regardless of its size and angle within the picture, after rigid alignment, the line connecting the top of the head and the chin will maintain the same vertical direction and proportional reference as the standard template.

[0050] Because artworks may inherently possess perspective distortion or proportional inconsistencies, rigid alignment alone cannot perfectly match local features. Therefore, non-rigid alignment is necessary. In non-rigid alignment, the Thin Plate Spline (TPS) interpolation algorithm is employed. By constructing a smooth two-dimensional interpolation function, local elastic deformation is applied to the rigidly aligned image, enabling precise point-to-point matching between the keypoints of the first image and the keypoints of the standard template. Non-rigid alignment not only ensures the overlap of keypoints but also establishes a one-to-one mapping relationship between the pixels of the first image and the pixels of the standard structural template, laying the foundation for subsequent structural deformation calculations.

[0051] S1220: Evaluate at least one structural deformation based on the alignment results, determine the corresponding structural deformation index, and combine all structural deformation indices into the first structural deformation feature; wherein, the at least one structural deformation includes perspective deformation, proportional deformation, and gray-black deformation.

[0052] Perspective distortion, proportional distortion, and gray-tone distortion are three structural distortion indicators that quantify the compliance of a work from three levels: geometric structure, proportional relationship, and light and dark distribution.

[0053] Perspective distortion is used to evaluate the perspective accuracy of three-dimensional forms in a sketch or drawing on a two-dimensional plane. In one possible implementation, the perspective distortion index is determined by calculating the Euclidean distance between the coordinates of perspective key points in a first image and the coordinates of corresponding key points in a standard structural template. Perspective Distortion Index The calculation formula is: (1); in, Indicates the first image i Coordinates of key perspective points (such as the intersection of building edges, the convergence point of the extension lines of the human torso); In the standard structural template, the first i Coordinates of key perspective points; This indicates the total number of perspective key points involved in the calculation; This represents the L2 norm (i.e., the straight-line distance between two points). The smaller the value, the more accurate the perspective of the image; conversely, the larger the value, the more obvious the perspective error, such as vanishing point shift or abnormal line convergence.

[0054] Proportional deformation is used to assess whether the length ratios between different parts of a form meet professional standards (such as the head-to-body ratio of a human body, or the height-to-width ratio of a still object). Specifically, proportional deformation indices... The calculation formula is: (2); in, Indicates the first image j The actual length of each structural segment (e.g., the length of the upper body or the length of the forearm). Indicates the first in the standard template j The nominal length of each structural segment; This indicates the number of structural segments. The formula quantifies the degree of disproportion by calculating the average of the absolute values ​​of the deviations between the actual length and the standard length. The closer the value is to 0, the more standard the proportion. The larger the value, the more severe the distortion of the proportion.

[0055] Gray-to-white distortion is primarily used to evaluate the logical consistency of light and shadow relationships in sketches or color works. Specifically, the image is divided into several specific regions (such as highlights, mid-tones, shadows, and cast shadows), and the average gray value of each region is calculated. Gray-to-white distortion index The calculation formula is: (3); in, Indicates the first image k The measured gray values ​​of each region (the value range is usually 0-255); This represents the reference grayscale value for the corresponding area in the standard structural template. This represents the total number of grayscale evaluation areas. The black-and-white distortion index reflects the difference between the actual light and dark distribution and the standard lighting logic. The closer the value is to 0, the more natural the transition between black, white, and gray, and the better it conforms to the laws of light and shadow. The larger the value, the more blurred the boundary between light and dark or the confused inherent colors.

[0056] The perspective distortion index, proportional distortion index, and grayscale distortion index obtained from the above calculations are combined into a multi-dimensional vector in a predetermined order. This vector is the first structural distortion feature. The first structural distortion feature comprehensively covers the deviations of the artwork in the core professional dimensions, providing a precise quantitative basis for comprehensive evaluation.

[0057] This application's embodiments construct a deformation comparison basis through key point alignment and quantify structural deformation characteristics through multi-dimensional deformation indicators. By accurately mapping from image space to quantified feature space, the judgment process that relies on subjective experience is transformed into a traceable and verifiable mathematical measurement process, enabling the accurate identification of structural defects in artworks.

[0058] Existing general-purpose multimodal large language models are not adapted for the fine arts profession and cannot identify structural problems (such as perspective errors) in artwork images and convert them into corresponding semantics.

[0059] Based on the above considerations, in one possible implementation, such as Figure 4 As shown, training the multimodal large language model includes: S410: Obtain the training dataset. Each artwork sample in the training dataset includes a second image of the artwork sample, the second structural deformation feature corresponding to the second image, a score label, and a comment label.

[0060] The training dataset consists of a large number of artwork samples, each containing complete annotation information to ensure that the model can learn professional art evaluation logic. Specifically, each artwork sample includes a second image of the artwork sample, the second structural deformation features corresponding to the second image, score annotations, and comment annotations.

[0061] The second image is a standardized image obtained after preprocessing the artwork sample. The second structural deformation feature is obtained in advance using the same method as the first structural deformation feature, serving as a supervisory signal for the model to learn structural patterns. The score annotation consists of a comprehensive score and sub-dimension scores given by senior art exam grading experts. The comment annotation consists of professional comments written by experts that conform to the college entrance examination art teaching standards, including descriptions of strengths, identification of weaknesses, and suggestions for improvement. These labeled data together constitute the gold standard for model training.

[0062] S420: Determine the structural deformation loss of the artwork sample based on the second structural deformation feature and the standard structural template corresponding to the artwork sample.

[0063] In this embodiment, structural constraints are explicitly introduced into the optimization objective of the model.

[0064] In one possible implementation, a standard structural template corresponding to the second image is first retrieved. Then, the difference between the structural deformation index currently predicted by the model or output by the model's hidden layer and the corresponding structural deformation index in the second structural deformation feature is calculated, and the structural deformation loss is calculated using the differences in each structural deformation index.

[0065] In another possible implementation, refer to Figure 3 As shown, the structural deformation loss is obtained by directly weighting and summing the various structural deformation indices in the second structural deformation feature. The calculation formula is as follows: (4); in, , , These are the weighting coefficients for perspective distortion, proportional distortion, and grayscale distortion, respectively. The weighting coefficients are determined based on the type of artwork. For example, a sketch emphasizes perspective and proportion, and would be set to... , , Color-themed works emphasize the relationship between black, white, and gray, and are set as follows: , , Sketches emphasize proportion and movement, setting... , , By minimizing The model is forced to learn to recognize structural deviations, thereby establishing sensitivity to structure in the internal feature representation.

[0066] S430: Input the artwork sample into the initial multimodal large language model, obtain the predicted score and predicted comment output by the initial multimodal large language model, obtain the score prediction loss based on the predicted score and the score label, and obtain the text generation loss based on the predicted comment and the comment label.

[0067] Compared to general multimodal large language models, this embodiment improves the input layer of the initial multimodal large language model by adding a structural deformation feature vector input channel. The 256-dimensional second structural deformation feature vector is concatenated with a 512-dimensional visual feature vector to form a 768-dimensional fused feature, which is then input into the model. The model performs forward propagation based on this fused feature and outputs a predicted score and predicted comments.

[0068] To measure the quality of the model output, the score prediction loss is calculated after obtaining the model output. and text generation loss Specifically, the mean squared error loss function can be used to calculate the error between the predicted score and the expert-annotated score, thereby determining the score prediction loss. The cross-entropy loss function can be used to calculate the negative log-likelihood between the predicted comment sequence generated by the model and the comment annotation sequence annotated by experts, thereby determining the text generation loss. It is used to measure the accuracy of comments in terms of vocabulary, grammar and terminology.

[0069] S440: Construct a total loss function based on the structural deformation loss, the score prediction loss, and the text generation loss, and iteratively train the initial multimodal large language model with the goal of minimizing the total loss function to obtain the multimodal large language model base.

[0070] The three types of losses mentioned above are weighted and fused to construct a total loss function to guide the updating of model parameters.

[0071] In one possible implementation, the total loss function The calculation formula is: (5).

[0072] During training, the backpropagation algorithm is used to calculate gradients and update all parameters of the model using stochastic gradient descent or its variants (such as AdamW). Through multiple rounds of iterative training with massive amounts of samples, the model gradually learns to integrate visual representations with structural essence, forming a large language model foundation with professional art knowledge.

[0073] S450: Based on the mapping relationship between the second structural deformation features of each artwork sample and the comment annotation, the multimodal large language model base is fine-tuned to obtain a trained multimodal large language model.

[0074] After obtaining a general multimodal large language model foundation, further fine-tuning is performed to enhance the model's performance on specific tasks. The fine-tuning phase continues to use the same training dataset as mentioned above, but focuses on mining the fine-grained mapping relationship between second structural deformation features (such as "perspective deformation value > 5") and comment annotations (such as "obvious perspective error").

[0075] Specifically, a smaller learning rate can be used, or even some of the model's underlying parameters can be frozen, with only fine-tuning done on the connection and decoding layers. Through this fine-tuning phase, the model can more accurately convert quantized structured data into natural language descriptions that conform to art teaching standards, ultimately forming a well-trained multimodal large language model.

[0076] This application embodiment constructs a multi-task joint training mechanism that includes structural deformation loss, and strengthens fine-tuning training based on the mapping relationship between structural deformation features and comments. Through the direct constraint of structural deformation loss, the fused feature vector output by the model includes explicit structural deformation semantics.

[0077] Existing art evaluation systems can only output numerical scores or comments, and cannot establish a logical connection between scores and comments, resulting in logical contradictions between scores and comments.

[0078] Based on the above considerations, in one possible implementation, such as Figure 5 As shown, in S140, generating the evaluation result of the artwork based on the fused feature vector includes: S1410: Extract the fusion semantic representation from the fusion feature vector.

[0079] The fused feature vector is the output of the last hidden layer of the multimodal large language model, containing global semantic information after deep nonlinear transformation. In this step, linear projection or pooling operations (such as global average pooling) are performed on the fused feature vector to extract a more compact and semantically focused fused semantic representation. This representation not only includes surface visual information such as lines, colors, and brightness, but also deeply encodes deep structural information such as perspective accuracy, proportional harmony, and the logic of black, white, and gray, serving as the core semantic foundation shared by the score prediction head and the comment generation head dual-branch tasks.

[0080] S1420: Using a score prediction head, obtain the initial comprehensive score of the artwork sample and the initial sub-score of the artwork based on the fused semantic representation, wherein each initial sub-score is determined based on the structural deformation index associated with the initial sub-score.

[0081] The score prediction head is an independent branch attached to the fused semantic representation, responsible for mapping abstract semantics to specific quantitative scores. The design of the score prediction head strictly adheres to art professional scoring standards, achieving a strong correlation between sub-scores and structural deformation indicators. Specifically, sub-scores directly correspond to structural deformation indicators such as perspective deformation, proportional deformation, and gray-to-white deformation. For example, the perspective score is obtained through regression calculation from the perspective-related feature components in the fused semantic representation, and it is negatively correlated with the perspective deformation indicator; that is, the smaller the perspective deformation indicator, the higher the perspective score.

[0082] Specifically, different initial sub-scores are assigned according to the category of the artwork: for sketches, the output scores are perspective, proportion, black and white, form, and texture; for quick sketches, the output scores are proportion, dynamics, line, and structure; and for color works, the output scores are light and shadow, hue, saturation, warmth and coolness, and space.

[0083] Finally, the scores of each initial sub-item are weighted and summed according to the preset weight coefficients of each initial sub-item score (such as perspective accounting for 0.5, proportion accounting for 0.3, and black, white and gray accounting for 0.2 in sketching), and an initial comprehensive score between 0 and 100 is obtained. This correlation mechanism ensures that the score directly reflects the structural accuracy of the work and gives the scoring results interpretability.

[0084] The score prediction head establishes a quantitative correlation between sub-item scores and structural deformation indicators, ensuring that the scoring results have a solid professional basis.

[0085] S1430: Using the comment generation head, an initial comment on the artwork is obtained based on the fused semantic representation, wherein at least a portion of the initial comment is obtained based on the at least one structural deformation index.

[0086] The comment generation head is another independent branch attached to the fused semantic representation. It usually adopts a Transformer-based decoder architecture and is responsible for transforming abstract semantics into professional comments that conform to natural language habits.

[0087] When generating text, the comment generation head deeply couples structural deformation index data to ensure consistency between the comment content and the structural deformation index. First, it extracts the numerical values ​​of the structural deformation index contained in the fused semantic representation. If a certain index (such as the perspective deformation index) exceeds a preset threshold, it indicates a significant defect in that dimension. During the decoding process, the comment generation head is guided to generate a description addressing this defect. For example, when a large black-and-white deformation index is detected, the model's generated statement will include professional expressions such as "chaotic black-and-white relationship" or "unnatural transition of light and dark boundaries."

[0088] The initial evaluation strictly follows a fixed three-part structure: "strengths description + defect identification + improvement suggestions". The strengths description is generated based on structural deformation indicators that do not exceed the corresponding deformation threshold, such as "perspective is basically accurate"; the defect identification is generated based on structural deformation indicators that exceed the deformation threshold, such as "the head-to-body ratio of the figure is unbalanced"; the improvement suggestions provide concrete guidance for the defects, such as "it is recommended to adjust the head-to-body ratio to the standard ratio of 1:7".

[0089] In addition, the comment generation head calls a pre-stored glossary of art terms (such as "vanishing point", "three planes and five tones", "inherent color", etc.) during decoding to constrain the output vocabulary, ensuring that the comments are standardized and professional, and avoiding unprofessional expressions.

[0090] By deeply coupling structural deformation data, the comment generation head can accurately locate defects in specific dimensions and generate concrete improvement suggestions, significantly enhancing the professional value and guiding significance of the comments.

[0091] S1440: Generate an evaluation result for the artwork based on the initial overall score, the initial sub-scores, and the initial comments.

[0092] The initial overall score, initial sub-scores, and initial comments are packaged and formatted to generate a structured evaluation result data package. This data package clearly displays the quantitative score and qualitative analysis of the artwork, together forming a comprehensive and professional evaluation. This evaluation result can be used for self-assessment by students and as a reference for teachers' instruction.

[0093] The embodiments of this application perform independent score prediction and comment generation based on shared feature encoding, and both generate corresponding results based on structural deformation features, realizing information sharing and logical coordination between score prediction and comment generation, and solving the technical drawback of the separation between scores and comments.

[0094] In one possible implementation, the score prediction head includes a shared fully connected layer, a score mapping layer, and a regression branch corresponding to each of the structural deformation indices; Based on the above, in S1420, using the score prediction head, the initial comprehensive score and initial sub-scores of the artwork samples are obtained based on the fused semantic representation, including: S1421: Input the fused semantic representation into the shared fully connected layer for dimensionality reduction processing to obtain a dimensionality-reduced semantic representation.

[0095] The score prediction head receives a fused semantic representation from a shared feature encoding layer, which typically has high dimensionality (e.g., 768 dimensions). To reduce computation and extract the features most relevant to the score, this representation is first fed into a shared fully connected layer of the score prediction head. This fully connected layer is equipped with a ReLU activation function, which maps the 768-dimensional fused semantic representation to a lower-dimensional space (e.g., 128 dimensions). The resulting dimensionality-reduced semantic representation, through linear transformation and non-linear activation, filters out redundant information irrelevant to the score and strengthens feature responses related to core dimensions such as perspective, proportion, and grayscale, laying the data foundation for subsequent itemized regression.

[0096] S1422: Input the dimensionality reduction semantic representation into the regression branch corresponding to each of the structural deformation indices to obtain the individual scores corresponding to the structural deformation indices.

[0097] This step employs a multi-branch parallel regression architecture to ensure that each sub-score accurately corresponds to a specific structural deformation index. Specifically, firstly, the corresponding regression branch is activated based on the category of the artwork. For example, for a sketch, three independent regression sub-branches are activated in parallel: the perspective score regression branch, the proportion score regression branch, and the grayscale score regression branch. Subsequently, each regression branch obtains a single score based on its corresponding structural deformation index. This one-to-one mapping relationship ensures that the perspective score regression branch is specifically responsible for analyzing perspective-related information in the fused features, while the proportion score regression branch is specifically responsible for analyzing proportion-related information, thus achieving a strong correlation between sub-scores and structural deformation indices. For example, if the perspective deformation information contained in the fused features is strong, the output value of the perspective score regression branch will be correspondingly lower, and vice versa.

[0098] S1423: Input each of the individual scores into the score mapping layer, use the score mapping layer to map the individual scores to the initial sub-scores, and obtain the initial comprehensive score based on each of the initial sub-scores.

[0099] Specifically, after collecting the individual scores from each regression branch, they are input into the score mapping layer.

[0100] The score mapping layer includes a Sigmoid activation function, which compresses the individual scores of each regression branch output to between 0 and 1, and then maps them to an initial sub-score of 0 to 100 after a linear transformation.

[0101] The initial scores for each sub-item are then input into a weighted summation module. The weighting coefficients within this module strictly adhere to art scoring standards and are dynamically adjusted based on the artwork category; that is, the weighting coefficients for the same structural deformation index differ between sketches, color works, and quick sketches. Through weighted summation, the score mapping layer outputs the initial overall score, along with the scores for each initial sub-item.

[0102] This step, through weighting, highlights the primary and secondary importance of different professional dimensions in the comprehensive evaluation, ensuring the scientific rigor and authority of the scoring results.

[0103] This application embodiment effectively extracts core scoring features by sharing a fully connected layer, and obtains individual scores independently through multiple regression branches. Finally, it obtains initial sub-scores and initial comprehensive scores through score mapping and weighted summation. The independent regression branches strengthen the correlation between the scoring results and the actual structural quality of the work, and the weighted summation is performed with weights corresponding to the work category to ensure that the comprehensive score is consistent with the evaluation focus of the work and to ensure the accuracy of the comprehensive score.

[0104] The scoring process of existing AI grading systems is opaque. They cannot explain the evaluation criteria for scoring, point out the strengths and weaknesses of the works, or provide suggestions for improvement. This black-box modeling method not only reduces the credibility of the scoring results, but also fails to meet the needs of art education to promote learning through assessment.

[0105] Based on the above considerations, in one possible implementation, the initial comments include descriptions of advantages, descriptions of defects, and suggestions for improvement.

[0106] Based on the above, in S1430, using the comment generation head, an initial comment on the artwork is obtained based on the fused semantic representation, including: S14311: Extract structural deformation indicators that exceed the deformation threshold from the fused semantic representation as the first target structural deformation indicator; extract structural deformation indicators that do not exceed the deformation threshold from the fused semantic representation as the second target structural deformation indicator.

[0107] Specifically, semantic representations are analyzed and integrated to extract structural deformation indicators such as perspective deformation, proportional deformation, and gray-black deformation.

[0108] The system has a pre-stored deformation threshold system corresponding to the art scoring criteria (for example, the perspective deformation threshold is set to 5, the scale deformation threshold is set to 0.15, and the grayscale deformation threshold is set to 0.3). Each structural deformation index is compared with its corresponding threshold: structural deformation indices exceeding the threshold are classified as first-target structural deformation indices, representing obvious defects in the work; structural deformation indices whose values ​​do not exceed the threshold are classified as second-target structural deformation indices, representing relative strengths in the work.

[0109] S14312: Call the art terminology glossary, map the first target structure deformation index to defect description information, and generate improvement suggestions based on the defect description information.

[0110] The art terminology glossary establishes a mapping relationship between deformation indicators and professional terms and improvement suggestions. For example, when the transparency deformation indicator exceeds the deformation threshold, the glossary maps the professional term "obvious perspective error, vanishing point positioning deviation" to the relevant terminology. Subsequently, based on the suggestion library associated with this professional term, corresponding improvement suggestions are generated, such as "perspective practice needs to be strengthened, with a focus on mastering the principle of near objects appearing larger and distant objects appearing smaller."

[0111] By iterating through all primary target indicators, the generated defect descriptions and improvement suggestions are logically assembled to form the defect description and improvement suggestion sections of the evaluation. Because a lookup table mapping method is used, the generated text is strictly limited to technical terminology, avoiding unprofessional expressions.

[0112] S14313: Call the art terminology glossary and map the second target structural deformation index into advantage description information.

[0113] For example, when the proportional deformation index does not exceed the deformation threshold, the professional term "the figure's proportions are harmonious, and the head-to-body ratio meets the standard" is mapped according to the art terminology glossary. The processor collects all the mapped descriptions of advantages, refines and splices them together to form the descriptions of advantages in the comments.

[0114] By using a glossary of art terminology, quantitative structural deformation indicators are directly mapped to standardized professional comments, achieving a high degree of transparency in the evaluation process. This ensures that the comments are based on evidence and effectively solves the illusion problem that may exist in generative models. It is particularly suitable for large-scale marking scenarios with high requirements for real-time performance and reliability.

[0115] S14314: Combine the advantages description information, the defects description information, and the improvement suggestions into the initial comment.

[0116] Specifically, the generated text fragments are combined according to a fixed logical order: descriptions of strengths, model information of weaknesses, and suggestions for improvement. For example, the final generated initial comment format is: "Strengths: The overall black, white, and gray relationships in the image are clear. Weaknesses: The right arm of the figure is slightly short in proportion, and there is a slight tilt in perspective. Suggestions: Strengthen the observation of body proportions, and pay attention to the relationship between the elbow and the waist." This template-based combination method ensures the standardization of the comment structure, and because the content is generated entirely based on quantitative indicators, it guarantees a high degree of logical consistency between the comments and the scoring results.

[0117] This application embodiment determines advantages and disadvantages through threshold judgment, and realizes term mapping and template matching through an art terminology glossary. This makes the obtained initial comments not only include the advantages and disadvantages of the work and suggestions for improvement, but also explain the reasons for the score through advantages and disadvantages. Moreover, the output text is standardized and professional, which improves the credibility of the evaluation results.

[0118] In one possible implementation, the initial comments include descriptions of advantages, descriptions of defects, and suggestions for improvement; the comment generation head includes a multi-head attention layer and a decoding layer.

[0119] Based on the above, in S1430, using the comment generation head, an initial comment on the artwork is obtained based on the fused semantic representation, including: S14321: Determine the guiding semantics of each of the structural deformation indices based on the relationship between each of the structural deformation indices and the deformation threshold of the structural deformation indices, and concatenate the guiding semantics with the fused semantic representation into an input sequence.

[0120] First, we analyze the various structural deformation indices in the fusion semantic representation. Based on the comparison between the structural deformation indices and the deformation threshold, we generate semantic labels describing the state of each index as guiding semantics. For example, if the perspective deformation index exceeds the deformation threshold, we generate the guiding semantic "severe perspective deviation"; if the structural deformation index does not exceed the deformation threshold, we generate the guiding semantic "good black-and-white gray relationship".

[0121] Subsequently, these discrete semantic labels are arranged in a preset order to form a structured guiding semantic sequence. Finally, this guiding semantic sequence is concatenated with the fused semantic representation along the sequence dimension to construct an input sequence. This concatenation operation is equivalent to providing the decoder with a clear outline of comments, forcing it to pay attention to these key structural issues during the comment generation process.

[0122] S14322: Input the input sequence into the multi-head attention layer to obtain multiple attention scores.

[0123] The concatenated input sequence is fed into the multi-head attention layer of the comment generation head. This multi-head attention mechanism allows the model to focus on different parts of the input sequence simultaneously. During this process, the semantic labels in the guiding semantic sequence act as queries, interacting with the feature vectors in the fused semantic representation (acting as key / value pairs). The model identifies feature dimensions in the fused semantic representation that are highly correlated with "perspective distortion" by calculating attention scores and assigns them higher weights. Simultaneously, the self-attention mechanism within the fused semantic representation continues to operate, capturing the complex relationships between lines, colors, and structures. Through this dual attention mechanism, the model can establish a strong correlation between guiding semantics and deep visual features, thereby understanding why perspective distortion occurs, rather than simply remaining at the level of the existence of distortion.

[0124] S14323: Input the multiple attention scores into the decoding layer, so that the decoding layer calls the art terminology glossary to generate the advantage description information, the defect description information and the improvement suggestions, and calls the comment structure template to generate the initial comment based on the advantage description information, the defect description information and the improvement suggestions.

[0125] The output of the multi-head attention layer is fed into a cascaded decoding layer. The decoding layer typically generates words one by one using an autoregressive approach. At each generation step, the decoding layer not only decodes based on the attention score but is also constrained by two factors: first, an art terminology vocabulary; when predicting the next word, the decoding layer prioritizes professional terms in the vocabulary (such as "vanishing point" and "tone line") to improve the professionalism of the text; second, a comment structure template; the decoding layer is trained to prioritize generating text that conforms to the "advantages-disadvantages-suggestions" three-part structure.

[0126] Ultimately, the decoding layer outputs a fluent, natural, and structurally complete initial comment, which retains the generative capabilities of a large language model while being strictly constrained by structurally deformed data.

[0127] This application embodiment uses the initial splicing of fused semantic representation and guiding semantics as input for multi-head attention, and constrains it at the decoding layer through an art terminology vocabulary and comment structure template. This transforms the fused semantic representation into insightful and standardized initial comments, which can meet the high standards of evaluation quality required in high-end art teaching assistance and professional marking scenarios.

[0128] In existing evaluation systems, there is a logical contradiction between score prediction and comment generation, such as the serious structural problems mentioned in comments for high-scoring works.

[0129] Based on the above considerations, in one possible implementation, such as Figure 5As shown, in S1440, the evaluation result of the artwork is generated based on the initial overall score, the initial sub-scores, and the initial comments, including: S1441: Extract defect keywords from the initial comments and map the defect keywords to the sub-item mapping scores of the target structural deformation corresponding to the defect keywords.

[0130] First, natural language processing is performed on the initial comments, calling upon a pre-stored defect keyword lexicon. Using string matching or semantic role labeling, key phrases describing the work's shortcomings are extracted from the initial comments, such as "obvious perspective errors," "proportion imbalance," or "chaotic black, white, and gray tones." Then, a mapping logic is established to associate these defect keywords with specific target structural deformation indicators. For example, "obvious perspective errors" is mapped to the perspective deformation indicator (a sub-item mapping score for target structural deformation), and "proportion imbalance" is mapped to the proportion deformation indicator (a sub-item mapping score for target structural deformation).

[0131] Furthermore, based on the severity of the defect keywords (such as "obvious", "minor", "serious"), the defect keywords are mapped to sub-item mapping scores for the target structural deformation, i.e., a sub-item score range. For example, if "obvious perspective error" is extracted, a lower score range (such as 0-60 points) is mapped. This step essentially quantifies the content of the comments, converting it into numerical values ​​that can be directly compared with the initial sub-item scores.

[0132] S1442: If there is a logical conflict between the sub-item mapping score of the target structural deformation and the initial sub-item score and the initial comprehensive score of the target structural deformation, adjust the defect keyword, the initial sub-item score and the initial comprehensive score to be logically consistent.

[0133] First, a logical comparison is performed to determine if there is a conflict between the mapped scores of each item and the actual initial scores of each item (and the initial overall score). The logic for determining conflicts is based on common sense in art scoring: descriptions of serious defects should correspond to lower scores. For example, if the comment mentions "obvious perspective errors" (mapped score 0-60 points), but the actual output initial perspective score is higher than 80 points, or the actual initial overall score is higher than 85 points, then a logical conflict is determined to exist.

[0134] Once a conflict is detected, an automatic correction mechanism is activated. Two correction strategies are employed: first, modifying the comment by using a thesaurus or rewriting the model to weaken the "obvious perspective error" to "generally accurate perspective, details need further refinement," thus aligning it with a higher score; second, modifying the score by proportionally lowering the initial perspective sub-score and the corresponding initial overall score to match the description of a serious defect. The option with the smallest correction is prioritized to maintain the stability of the evaluation results. If a conflict cannot be resolved through simple numerical adjustments (e.g., the comment and score are completely contradictory), the evaluation result is flagged and a manual review mechanism is triggered for final expert adjudication.

[0135] S1443: Generate the final overall score, final sub-scores, and final comments based on the adjustment results. (Refer to...) Figure 2 As shown.

[0136] The adjusted data (including any potentially revised scores and comments) are then solidified. The adjusted scores are confirmed as the final overall score and final sub-scores, and the adjusted comments are confirmed as the final comments, ensuring that there are no logical contradictions among these three sets of data. For example, a high score might correspond to a minor deficiency, while a low score might correspond to a serious defect. This solidification process guarantees the rigor of the output data.

[0137] S1444: Combine the final overall score, the final sub-scores, and the final comments to form the evaluation result of the artwork.

[0138] The final confirmed overall score, final sub-scores, and final comments are packaged according to a standard format to generate the final evaluation result data package. This data package includes not only quantitative indicators but also logically validated qualitative analysis; the two corroborate each other, forming a logically rigorous and complete evaluation system. The evaluation result is then output to a storage module or user terminal.

[0139] This application extracts defect keywords from the initial comments, maps them to sub-item mapping scores, and adjusts the comments and scores based on the logical consistency comparison results between the sub-item mapping scores and the structural deformation index. This ensures the logical consistency between the scores and comments in the evaluation results and improves the professional credibility of the evaluation results.

[0140] In one possible implementation, the evaluation method for artworks based on structural deformation also includes: Assign a unique identifier to the artwork, establish the association between the first image, the first structural deformation feature, the fusion feature vector, the evaluation result, and the unique identifier, and store the association, the first image, the first structural deformation feature, the fusion feature vector, and the evaluation result.

[0141] The globally unique identifier can be a combination of the exam number, admission ticket number, and timestamp, or a universally unique identifier automatically generated by the system. This unique identifier serves as the primary key for this evaluation data link, spanning the entire lifecycle from image input, feature extraction, model inference to result output. It is used to index and associate all data entities generated during the evaluation process, ensuring data traceability.

[0142] When establishing relationships, for example in a relational database, a data table is created with a unique identifier as the primary key, and other data is stored as fields or related tables. This relational storage mechanism ensures that all related information about the artwork, from the original image to the final review, can be retrieved with a single click by simply entering the unique identifier, achieving full transparency in the evaluation process.

[0143] refer to Figure 2 As shown, the stored data includes: Raw data: including the original image before preprocessing (e.g., the first image before preprocessing). Figure 2 The data includes exam paper data, preprocessed images, and basic information such as the identified work categories.

[0144] Intermediate data includes the first structural deformation features, key point coordinates, and fused feature vectors. This data serves as the direct basis for the scoring process and is also crucial evidence for subsequent review.

[0145] Output data includes the final evaluation results, specifically the final overall score, the final sub-scores for each item, the final comments (including advantages, disadvantages, and improvement suggestions), and the consistency verification record of the text and images.

[0146] Supporting data includes static resources such as a standard template library for art exam papers, a glossary of art terms, and model training parameters.

[0147] To safeguard personal privacy and educational data security, strict security strategies are implemented for the storage process. On one hand, encrypted storage is employed, utilizing algorithms such as Advanced Encryption Standards (AES) to encrypt sensitive data containing personal information and exam paper content, preventing unauthorized theft or tampering at the physical storage level. On the other hand, a data backup mechanism is in place, employing a hybrid storage architecture of solid-state drives (SSDs) and cloud storage. Local SSDs are used for high-speed read / write of recent data, while cloud storage is used for off-site disaster recovery backup of historical data. The backup strategy supports both full and incremental backups, ensuring no data loss and rapid business recovery in the event of hardware failure or natural disasters. Simultaneously, a strict access control list is set up for the processor, authorizing only personnel with specific roles (such as marking team leaders and review experts) to access the original data. Ordinary personnel can only view the anonymized evaluation results, fully complying with educational data security regulations.

[0148] This embodiment achieves closed-loop management and traceability of art evaluation data throughout the entire process by constructing an associated storage system based on unique identifiers, effectively solving the problems of data silos and lack of evidence in traditional storage methods.

[0149] Based on the above, this application also provides an art evaluation device based on structural deformation. The art evaluation device based on structural deformation and the aforementioned art evaluation method based on structural deformation can be referred to and correspond to each other.

[0150] As an example, such as Figure 6 As shown, the artwork evaluation device based on structural deformation includes: The structural feature extraction module 610 is used to extract the structural features of the first image of the artwork. The structural deformation feature acquisition module 620 is used to compare the structural features with a standard structural template to obtain a first structural deformation feature, wherein the standard structural template is the structural reference corresponding to the artwork. The fusion module 630 is used to extract the visual features of the first image and input the first structural deformation features and the visual features into the multimodal large language model to obtain the fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the first structural deformation features of the artwork samples in the training dataset; Evaluation module 640 is used to generate an evaluation result of the artwork based on the fused feature vector and output the evaluation result.

[0151] This application embodiment compares the structural features of an artwork with a standard structural template to obtain structural deformation features, and evaluates the artwork based on the fusion feature vector of structural deformation features and visual features. In this way, the structural deformation features are used as a professional basis to identify core art problems and improve the accuracy of the evaluation.

[0152] Figure 7 This is a schematic diagram of the structure of the electronic device provided in this application, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions from the memory 730 to execute a method for evaluating artworks based on structural deformation. This method includes: Extracting the structural features of the first image of an artwork; The structural features are compared with a standard structural template to obtain the first structural deformation features, wherein the standard structural template is the structural reference corresponding to the artwork. Visual features of the first image are extracted, and the first structural deformation features and the visual features are input into a multimodal large language model to obtain a fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the second structural deformation features of artwork samples in the training dataset; An evaluation result for the artwork is generated based on the fused feature vector, and the evaluation result is output.

[0153] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] On the other hand, this application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the art evaluation method based on structural deformation provided in the above embodiments. The method includes: Extracting the structural features of the first image of an artwork; The structural features are compared with a standard structural template to obtain the first structural deformation features, wherein the standard structural template is the structural reference corresponding to the artwork. Visual features of the first image are extracted, and the first structural deformation features and the visual features are input into a multimodal large language model to obtain a fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the second structural deformation features of artwork samples in the training dataset; An evaluation result for the artwork is generated based on the fused feature vector, and the evaluation result is output.

[0155] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the art evaluation method based on structural deformation provided in the above embodiments, the method comprising: Extracting the structural features of the first image of an artwork; The structural features are compared with a standard structural template to obtain the first structural deformation features, wherein the standard structural template is the structural reference corresponding to the artwork. Visual features of the first image are extracted, and the first structural deformation features and the visual features are input into a multimodal large language model to obtain a fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the second structural deformation features of artwork samples in the training dataset; An evaluation result for the artwork is generated based on the fused feature vector, and the evaluation result is output.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for evaluating artworks based on structural deformation, characterized in that, include: Extracting the structural features of the first image of an artwork; The structural features are compared with a standard structural template to obtain the first structural deformation features; The standard structural template is the structural benchmark corresponding to the artwork; Visual features of the first image are extracted, and the first structural deformation features and the visual features are input into a multimodal large language model to obtain a fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the second structural deformation features of artwork samples in the training dataset; An evaluation result for the artwork is generated based on the fused feature vector, and the evaluation result is output.

2. The method for evaluating artworks based on structural deformation according to claim 1, characterized in that, The step of comparing the structural features with a standard structural template to obtain the first structural deformation feature includes: Align the structural features with the key points in the standard structural template; Based on the alignment results, at least one structural deformation is evaluated, and the corresponding structural deformation index is determined. All structural deformation indices are combined into the first structural deformation feature. The at least one structural deformation includes perspective deformation, proportional deformation, and gray-black deformation.

3. The method for evaluating artworks based on structural deformation according to claim 2, characterized in that, The evaluation result of the artwork generated based on the fused feature vector includes: Extract the fused semantic representation from the fused feature vector; Using a score prediction head, the initial comprehensive score and initial sub-scores of the artwork samples are obtained based on the fused semantic representation. Each initial sub-score is determined based on the structural deformation index associated with the initial sub-score. Using a comment generation head, an initial comment on the artwork is obtained based on the fused semantic representation, wherein at least a portion of the initial comment is obtained based on the at least one structural deformation index; The evaluation result of the artwork is generated based on the initial overall score, the initial sub-scores, and the initial comments.

4. The method for evaluating artworks based on structural deformation according to claim 3, characterized in that, The evaluation result of the artwork is generated based on the initial overall score, the initial sub-scores, and the initial comments, including: Extract defect keywords from the initial comments and map the defect keywords to the sub-item mapping scores of the target structural deformation corresponding to the defect keywords; If there is a logical conflict between the sub-item mapping score of the target structural deformation and the initial sub-item score and the initial comprehensive score of the target structural deformation, the defect keyword, the initial sub-item score and the initial comprehensive score shall be adjusted to be logically consistent. The final overall score, final sub-scores, and final comments are generated based on the adjustment results. The final overall score, the final sub-scores, and the final comments are combined to form the evaluation result of the artwork.

5. The method for evaluating artworks based on structural deformation according to claim 3, characterized in that, The score prediction head includes a shared fully connected layer, a score mapping layer, and a regression branch corresponding to each of the structural deformation indices; Using a score prediction head, based on the fused semantic representation, the initial comprehensive score and initial sub-scores of the artwork samples are obtained, including: The fused semantic representation is input into the shared fully connected layer for dimensionality reduction processing to obtain a dimensionality-reduced semantic representation. The dimensionality-reduced semantic representation is input into the regression branch corresponding to each of the structural deformation indices to obtain the individual scores corresponding to the structural deformation indices. Each of the individual scores is input into the score mapping layer, and the score mapping layer is used to map the individual scores to the initial sub-scores. The initial comprehensive score is obtained based on each of the initial sub-scores.

6. The method for evaluating artworks based on structural deformation according to claim 3, characterized in that, The initial comments include descriptions of strengths, descriptions of weaknesses, and suggestions for improvement; Using the comment generation head, initial comments on the artwork are obtained based on the fused semantic representation, including: Extract structural deformation indices that exceed the deformation threshold from the fused semantic representation as the first target structural deformation index; extract structural deformation indices that do not exceed the deformation threshold from the fused semantic representation as the second target structural deformation index. The glossary of art terms is invoked to map the deformation index of the first target structure to defect description information, and improvement suggestions are generated based on the defect description information. The art terminology glossary is invoked to map the second target structural deformation index into advantage description information; The advantages, the defects, and the improvement suggestions are combined to form the initial evaluation.

7. The method for evaluating artworks based on structural deformation according to claim 3, characterized in that, The initial comments include descriptions of strengths, descriptions of weaknesses, and suggestions for improvement; the comment generation head includes a multi-head attention layer and a decoding layer. Using the comment generation head, initial comments on the artwork are obtained based on the fused semantic representation, including: Based on the relationship between each of the structural deformation indices and the deformation threshold of the structural deformation indices, the guiding semantics of each of the structural deformation indices are determined, and the guiding semantics are concatenated with the fused semantic representation to form an input sequence; The input sequence is input into the multi-head attention layer to obtain multiple attention scores; The multiple attention scores are input into the decoding layer, which then calls the art terminology glossary to generate the advantage description information, the defect description information, and the improvement suggestions. The decoding layer then calls the comment structure template to generate the initial comment based on the advantage description information, the defect description information, and the improvement suggestions.

8. The method for evaluating artworks based on structural deformation according to claim 1 or 2, characterized in that, Training the multimodal large language model includes: Obtain the training dataset, wherein each artwork sample in the training dataset includes a second image of the artwork sample, the second structural deformation feature corresponding to the second image, a score label, and a comment label; The structural deformation loss of the artwork sample is determined based on the second structural deformation feature and the standard structural template corresponding to the artwork sample. The artwork sample is input into an initial multimodal large language model to obtain the predicted score and predicted comment output by the initial multimodal large language model. The score prediction loss is obtained based on the predicted score and the score label, and the text generation loss is obtained based on the predicted comment and the comment label. A total loss function is constructed based on the structural deformation loss, the score prediction loss, and the text generation loss. The initial multimodal large language model is iteratively trained with the goal of minimizing the total loss function to obtain the multimodal large language model base. The multimodal large language model base is fine-tuned based on the mapping relationship between the second structural deformation features of each artwork sample and the comment annotations to obtain a trained multimodal large language model.

9. An art evaluation device based on structural deformation, characterized in that, include: The structural feature extraction module is used to extract the structural features of the first image of an artwork. The structural deformation feature acquisition module is used to compare the structural features with a standard structural template to obtain the first structural deformation feature; The standard structural template is the structural benchmark corresponding to the artwork; The fusion module is used to extract the visual features of the first image and input the first structural deformation features and the visual features into the multimodal large language model to obtain the fused feature vector output by the multimodal large language model; the loss function of the multimodal large language model includes structural deformation loss, which is obtained based on the first structural deformation features of the artwork samples in the training dataset; The evaluation module is used to generate an evaluation result for the artwork based on the fused feature vector, and output the evaluation result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the art evaluation method based on structural deformation as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium, wherein a computer program is stored on the non-transitory computer-readable storage medium, characterized in that, When the computer program is executed by the processor, it implements the art evaluation method based on structural deformation as described in any one of claims 1 to 8.

Images