An artistic work intelligent evaluation method for hearing-impaired students

Abstract

The application provides an art work intelligent evaluation method for hearing-impaired students, which comprises the following steps: feature extraction of a standardized art work image by using a convolutional neural network to obtain image features; color distribution of three channels of hue, saturation and brightness is extracted from the image features in an HSV color space to evaluate the color harmony of the art work image; the direction of each line in the art work image is predicted by a direction prediction mechanism based on the image features, and the direction is matched to a highly abstract graphic semantic to evaluate the graphics of the art work image; the center of gravity of the art work image is located by using a spatial attention mechanism based on the image features to evaluate the composition rationality of the art work image; and the evaluation results of different dimensions are converted into multi-modal feedback results suitable for the understanding of hearing-impaired students. The method has high automation, provides various understandable evaluation methods, has high evaluation efficiency, realizes process evaluation, and reduces the work burden of teachers.

Description

Technical Field

[0001] This application belongs to the field of intelligent teaching technology, and in particular relates to an intelligent evaluation method for artworks of students with hearing impairments. Background Technology

[0002] Currently, in design courses for students with hearing impairments, feedback on student work primarily relies on manual feedback from teachers. A search reveals that existing related technologies mainly include CN114822148A, "A Teaching Board Suitable for Art Education for the Deaf," which visualizes teacher guidance through speech-to-text conversion and uses grids and positioning points to assist students in imitating drawing techniques. However, its limitation lies in only converting teacher speech to text and not in automatically analyzing and evaluating the content of student work.

[0003] In addition, the following problems exist in evaluating the artwork of students with hearing impairments: Firstly, general image recognition technologies (such as conventional CNN classification models) are mainly geared towards natural image recognition, and their accuracy in recognizing artworks by hearing-impaired students is low. Because the designs of hearing-impaired students tend to be simplified, highly generalized in shape, or partially abstract, general models are prone to misclassifying them as random scribbles and cannot accurately extract key features such as composition, color, and form to assess their design abilities.

[0004] Secondly, existing teaching evaluation systems mostly focus on the final work score, lacking continuous recording and tracking of students' implicit abilities such as creative ideas and problem-solving during the creative process. The evaluation presents a result-oriented characteristic and cannot objectively reflect the learning and growth trajectory of hearing-impaired students.

[0005] Third, there is a lack of dedicated feedback channels for hearing-impaired students. The existing system does not convert comments into a multimodal form that students can understand independently (such as visual annotations + sign language animation). Students still need to rely on teachers or sign language teachers for translation, resulting in poor feedback timeliness and serious information loss. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide an intelligent evaluation method for the artworks of hearing-impaired students, which aims to solve the technical problem of the lack of automated and understandable evaluation methods for the artworks of hearing-impaired students, improve the recognition accuracy of the artworks of hearing-impaired students, present evaluation feedback in a multimodal form that hearing-impaired students can understand independently, and construct process evaluation recording methods.

[0007] This application provides an intelligent evaluation method for artworks created by students with hearing impairments, including: By using convolutional neural networks to extract features from standardized artwork images, image features containing implicit information about color texture, graphic structure, and spatial layout are obtained. In the HSV color space, the color distribution of the three channels of hue, saturation and brightness are extracted from the image features to evaluate the color harmony of the artwork image. Based on image features, the direction of each line in the artwork image is predicted through a direction prediction mechanism and matched to a highly abstract graphic semantic to evaluate the expressive accuracy of the artwork image. Based on image features, spatial attention mechanism is used to locate the center of gravity of an artwork image in order to evaluate the compositional rationality of the artwork image; The evaluation results from different dimensions are transformed into multimodal feedback results that are easy for hearing-impaired students to understand.

[0008] The intelligent evaluation method for art works of hearing-impaired students provided in this application has the following beneficial effects: 1) It transforms the traditional evaluation method that relies on auditory and textual communication into a multimodal feedback method in the form of visual annotations and sign language animations that hearing-impaired students can understand independently, truly achieving barrier-free communication in the evaluation process; 2) It automatically identifies works and generates feedback, reducing the evaluation time for a single work from 5-10 minutes for teachers to less than 30 seconds, significantly reducing the burden of repetitive evaluations for teachers; 3) It automatically records data for each work, constructing a visual growth portfolio and learner profile, making up for the evaluation deficiency of traditional art teaching that emphasizes results over process; 4) It effectively identifies the characteristics of art works of hearing-impaired students, avoiding the misjudgment problems of general image recognition technology. Attached Figure Description

[0009] Figure 1 A flowchart of the intelligent evaluation method for artworks of hearing-impaired students provided in an embodiment of this application is shown; Figure 2 A schematic diagram of the multimodal feedback results provided in an embodiment of this application is shown; Figure 3 A schematic diagram of the student terminal display interface provided in an embodiment of this application is shown; Figure 4 A schematic diagram of a procedural learner profile provided in an embodiment of this application is shown. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this technical solution clearer, the following detailed description, in conjunction with specific embodiments, further illustrates this technical solution. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this technical solution.

[0011] Example 1: Please see as follows Figure 1 The flowchart shown is for an intelligent evaluation method for artwork created by students with hearing impairments. Figure 1 As shown, the method includes: S101. Use convolutional neural networks to extract features from standardized artwork images to obtain image features that contain information about color texture, graphic structure, and spatial layout.

[0012] In this step, the convolutional neural network includes an input layer, a convolutional feature extraction layer, a feature encoding layer, and an output layer. It uses ResNet as the backbone network and extracts image features that contain implicit color and texture information, graphic structure information, and spatial layout information from the artwork image layer by layer through multiple convolutional layers and residual modules.

[0013] As an example, the input artwork image is represented as follows: (1) In the formula, This indicates that the input is an image of an artwork. and These represent the height and width of the image, respectively; 3 represents the number of RGB color channels.

[0014] The image features of an artwork image output by a convolutional neural network are represented as follows: (2) In the formula, Represents a convolutional neural network. This represents the image features output by the convolutional neural network.

[0015] It is important to note that convolutional neural networks need to be trained beforehand. During the training phase, the teacher's evaluation results are used as supervisory labels to train the convolutional neural network. Let the number of training samples be M, and the teacher's evaluation value corresponding to the i-th training sample be... The network prediction evaluation value is The training loss function is then expressed as: (3) In the formula, L represents the training loss; M represents the number of training samples; This represents the human evaluation value corresponding to the i-th sample; This represents the network prediction evaluation value.

[0016] By continuously updating the parameters of the convolutional neural network, the loss function gradually converges, thereby obtaining a convolutional neural network capable of recognizing the features of artworks.

[0017] Furthermore, before using a convolutional neural network to extract features from standardized artwork images, the method also includes: Step 201: Obtain images of artworks designed by hearing-impaired students using one or more devices, such as electronic drawing tablets, tablets, or mobile phones.

[0018] Step 202: Denoise the artwork image using an adaptive median filtering algorithm.

[0019] Step 203: Enhance the contrast of the denoised image using histogram equalization.

[0020] Step 204: Perform size normalization processing on the enhanced image to unify the size to 512×512 pixels.

[0021] S102. Extract the color distribution of the three channels of hue, saturation and brightness from the image features in the HSV color space to evaluate the color harmony of the artwork image.

[0022] In practice, the color harmony of an artwork's image is evaluated using the following methods: Step 1021: Based on the color distribution of the three channels of hue, saturation, and brightness, construct a color histogram and extract the set of dominant hues: (4) In the formula, Indicates the first Primary color.

[0023] Step 1022: Calculate the theme color percentage based on the number of pixels corresponding to each primary color tone. (5) In the formula, and They represent the first and The number of pixels for each color.

[0024] Step 1023: Calculate the hue distance between any two primary hues based on the hue values ​​in the HSV color space: (6) In the formula, and They represent the first and the The hue value of the primary color tone.

[0025] Step 1024: Determine whether the main color tones satisfy any of the pre-set color matching rules of complementary colors, adjacent colors, or similar colors based on the hue distance.

[0026] In this step, corresponding color matching rules and theme color application standards are pre-designed according to different themes and stored in the database for easy retrieval at any time.

[0027] Step 1025: When the color matching rules are met, assign a higher color harmony evaluation value to the image; otherwise, assign a lower color harmony evaluation value to the image.

[0028] As an example, in artworks themed around "the changing seasons," if the main subject of the painting uses green and blue to represent spring and summer, yellow and orange to represent autumn, and white to represent winter, then the use of the theme color is considered reasonable. If the entire work uses only a single color system such as red and yellow, then the color expression of the work is considered to have a low degree of matching with the theme content, and the color feature evaluation score is reduced accordingly.

[0029] S103. Based on image features, predict the direction of each line in the artwork image through a direction prediction mechanism, and match it to a highly abstract graphic semantic to evaluate the expressive accuracy of the artwork image.

[0030] In this step, the matching relationship between the direction of the lines and the semantics of the graphics is as follows: simple broken lines - mountains, circles with lines - people, multiple wavy lines - rivers, radial lines - the sun, long rectangles combined with multiple vertical lines - buildings, ellipses combined with tree trunks - trees, and horizontal lines combined with curved roads.

[0031] Furthermore, the accuracy of the artwork's imagery is evaluated by judging the accuracy of the graphic's semantic expression based on excellent exemplary works on the corresponding theme, which are pre-stored in the teaching standards database.

[0032] S104. Based on image features, use spatial attention mechanisms to locate the center of gravity of an artwork image in order to evaluate the compositional rationality of the artwork image.

[0033] In practice, the compositional rationality of an artwork's image is evaluated using the following methods: The known image features are represented as follows: (7) In the formula, Indicating the first in an image of an artwork Feature vectors corresponding to each spatial location This represents the total number of spatial locations in the feature.

[0034] Step 1041: Calculate the corresponding spatial attention weights based on the importance of the feature vectors corresponding to each spatial location in the image. (8) In the formula, Indicates the first Spatial attention weights corresponding to each spatial location This represents the learnable weight parameters in the attention mechanism. Indicates the first The importance score of each spatial location This represents an exponential function.

[0035] here, The denominator of the exponential function represents the sum of importance scores for all spatial locations, used to normalize the attention weights to satisfy: (9) Furthermore, image features can be weighted and fused based on spatial attention weights to obtain enhanced image features: (10) In the formula, This represents the image features enhanced by the spatial attention mechanism. The greater the attention weight, the higher the contribution of the corresponding spatial location to the final image features.

[0036] Step 1042: Calculate the centroid coordinates of the main image region based on the attention weights corresponding to each spatial location. (11) (12) In the formula, and These represent the x and y coordinates of the centroid of the main image region, respectively. and They represent the first The x-coordinate and y-coordinate corresponding to each spatial location.

[0037] Step 1043: Calculate the composition offset based on the positional relationship between the centroid of the main image region and the geometric center of the image. (13) In the formula, This represents the Euclidean distance between the centroid of the main image region and the geometric center of the image. and These represent the x-coordinate and y-coordinate of the geometric center of the image, respectively.

[0038] Step 1044: Evaluate the compositional rationality of the artwork image based on the compositional offset and the pre-set compositional rules.

[0039] In this step, the pre-set composition rules can be the golden ratio, the rule of thirds, central composition, or other methods. This application does not impose any restrictions on these methods.

[0040] S105. Transform evaluation results from different dimensions into multimodal feedback results that are suitable for hearing-impaired students to understand.

[0041] In practice, the evaluation results from different dimensions are transformed into multimodal feedback results that are easy for hearing-impaired students to understand through the following methods: Step 1051: Mark the evaluation results of different dimensions in the corresponding areas of the artwork image using one or more of the following visual forms: highlighting, arrow indicators, reference lines, and text.

[0042] And / or, in step 1052, the text comments of the evaluation results of different dimensions are segmented into a word sequence, the corresponding sign language animation segment for each word is retrieved according to the word-to-sign language animation comparison table, and then the sign language animation segments are spliced ​​together according to the time sequence to generate a complete sign language animation video.

[0043] And / or, step 1053, based on the evaluation results of different dimensions, improve the artwork image according to the corresponding evaluation criteria, and generate a reference image containing illustrations for comparison of each dimension.

[0044] Furthermore, the method also includes obtaining the comprehensive evaluation result through the following means: Step 1054: Weight and fuse the evaluation results from the three dimensions to obtain the comprehensive evaluation value of the artwork image: (14) In the formula, T represents the comprehensive evaluation characteristic; This indicates the evaluation value for color harmony; This indicates the accuracy rating of the expression; This indicates the evaluation value for the rationality of the composition; Let each represent a corresponding evaluation weight, and satisfy the following conditions: (15) When the color coordination evaluation value is below the preset threshold, corresponding color matching examples are recommended; when the expression evaluation value is below the preset threshold, excellent demonstration works on the corresponding theme are recommended; when the composition rationality evaluation value is below the preset threshold, students are prompted to adjust the main layout based on the location of the main area and composition rules, and corresponding composition examples are recommended. All of the above examples are pre-stored in the teaching standard database.

[0045] Furthermore, after transforming the evaluation results from different dimensions into multimodal feedback results suitable for hearing-impaired students to understand, the method also includes: Step 301: Automatically record images of the artworks of hearing-impaired students, as well as evaluation results from different dimensions, multimodal feedback results, and comprehensive evaluation results.

[0046] Step 302: Generate the ability development trends of hearing-impaired students in different dimensions based on historical data, and generate corresponding teaching suggestions for teachers.

[0047] Here, the comprehensive evaluation results, evaluation results from different dimensions, and feedback information are stored in the student's historical evaluation database to establish a student's artistic ability growth record. Based on the evaluation results of each evaluation, the database generates the student's artistic ability development trend, providing a basis for teachers to carry out personalized teaching.

[0048] Furthermore, the intelligent evaluation method for artwork created by hearing-impaired students proposed in this application can be directly deployed on a dedicated teaching terminal (including an all-in-one machine with a drawing tablet, display screen, and processor). Students can complete their creations directly on the terminal and receive immediate feedback without having to upload photos via their mobile phones. The terminal can also integrate a vibration motor to alert students to the feedback generated through vibration (suitable for scenarios where attention is diverted).

[0049] Example 2: Real-time classroom feedback scenario: In a design class at a special education school, after students complete a design project themed "The Changing Seasons," they take a photo of it using a tablet and upload it to a system equipped with the method proposed in this application. The evaluation process is as follows: (2.1) Perform noise reduction, contrast enhancement, and standardization processing on the photos; (2.2) The main subject of the picture is centered (composition score 82 points, qualified), only red and yellow are used in the color (color richness score 55 points, unqualified), and the lines are smooth (shape score 78 points, qualified). (2.3) After comparing with the standard library, the specific problems of the color matching dimension are: lack of use of cool colors and insufficient expression of the theme of the four seasons; (2.4) Highlight the area in blue on the original image: the area that needs to be added with cool colors, and mark it: it is recommended to add blue and green tones; In addition, generate sign language virtual human animations, with the following comments: It is recommended to add blue and green tones to represent the spring and summer seasons; In addition, we will push out a picture and text card containing examples of color matching for the four seasons; Please refer to the following section: Figure 2 The diagram showing the multimodal feedback results and as shown below Figure 3 The diagram shows the student terminal display interface.

[0050] (2.5) The evaluation data will be automatically saved into the student's personal file.

[0051] Example 3: Process Assessment and Instructional Decision-Making Scenarios After a student uses this system 8 times consecutively, the system automatically generates a learner profile for that student. Please refer to the example below. Figure 4 The diagram shows a process learner profile.

[0052] Composition ability: improved from 62 points to 85 points (significant improvement); Color matching: Improved from 50 points to 58 points (slow progress, intervention required); Modeling ability: Improved from 70 to 78 points (normal progress).

[0053] The system sent a suggestion to the teacher: This student needs to strengthen their color perception; it is recommended to provide training resources on blue-green tones and add basic color composition exercises. Based on this, the teacher adjusted the teaching plan and assigned targeted training tasks to the student.

[0054] The above content is only a preferred embodiment of the present invention. For those skilled in the art, many changes can be made in the specific implementation and application scope based on the ideas of the present invention. As long as these changes do not depart from the concept of the present invention, they all fall within the protection scope of the present invention.

Claims

1. A method for intelligent evaluation of artworks by students with hearing impairments, characterized in that, The method includes: By using convolutional neural networks to extract features from standardized artwork images, image features containing implicit information about color texture, graphic structure, and spatial layout are obtained. In the HSV color space, the color distribution of the three channels of hue, saturation and brightness are extracted from the image features to evaluate the color harmony of the artwork image. Based on image features, the direction of each line in the artwork image is predicted through a direction prediction mechanism and matched to a highly abstract graphic semantic to evaluate the expressive accuracy of the artwork image. Based on image features, spatial attention mechanism is used to locate the center of gravity of an artwork image in order to evaluate the compositional rationality of the artwork image; The evaluation results from different dimensions are transformed into multimodal feedback results that are easy for hearing-impaired students to understand.

2. The method as described in claim 1, characterized in that, Before performing feature extraction on standardized artwork images, the method further includes: Obtain images of artworks designed by hearing-impaired students using one or more devices such as electronic drawing tablets, tablets, and mobile phones; Denoising of artwork images is achieved using an adaptive median filtering algorithm; Histogram equalization is used to enhance the contrast of denoised images. The enhanced image is then normalized to a size of 512×512 pixels.

3. The method as described in claim 1, characterized in that, Evaluate the color harmony of an artwork image using the following methods: Based on the color distribution of the three channels of hue, saturation, and brightness, a color histogram is constructed, and the set of dominant hues is extracted. The proportion of the theme color is calculated based on the number of pixels corresponding to each main color. Calculate the hue distance between any two primary hues based on the hue values ​​in the HSV color space; Determine whether the main color tones satisfy any of the pre-defined color matching rules of complementary colors, adjacent colors, or similar colors based on the hue distance; When the color matching rules are met, the image is given a higher color harmony rating; otherwise, the image is given a lower color harmony rating.

4. The method as described in claim 1, characterized in that, Evaluate the compositional rationality of an artwork's image using the following methods: Calculate the corresponding spatial attention weights based on the importance of the feature vectors corresponding to each spatial location in the image; Calculate the centroid coordinates of the main image region based on the corresponding attention weights at each spatial location; The composition offset is calculated based on the positional relationship between the centroid of the main image region and the geometric center of the image. The compositional rationality of the artwork image is evaluated based on the compositional offset and pre-set compositional rules.

5. The method as described in claim 1, characterized in that, The process of converting multi-dimensional performance levels into multimodal feedback results suitable for hearing-impaired students to understand includes: The evaluation results from different dimensions are marked in the corresponding areas of the artwork image using one or more of the following visual forms: highlighting, arrow indicators, reference lines, and text. And / or, the text comments of the evaluation results of different dimensions are segmented into a word sequence, the corresponding sign language animation clips for each word are retrieved according to the word-to-sign language animation comparison table, and then the sign language animation clips are spliced ​​together according to the time sequence to generate a complete sign language animation video. And / or, based on the evaluation results of different dimensions, improve the artwork image according to the evaluation criteria to generate a reference image containing comparative illustrations of each dimension.

6. The method as described in claim 1, characterized in that, After converting the multi-dimensional performance levels into multimodal feedback results suitable for hearing-impaired students to understand, the method further includes: The system automatically records images of artwork created by hearing-impaired students, as well as evaluation results from different dimensions, multimodal feedback results, and comprehensive evaluation results; wherein the comprehensive evaluation result is obtained by weighted fusion of the evaluation results from the three dimensions. Based on historical data, the system generates developmental trends of hearing-impaired students' abilities in different dimensions and provides teachers with corresponding teaching suggestions.

Images