Online curriculum homework scoring system for multi-mode multi-level feature joint learning

Abstract

The invention discloses an online curriculum homework scoring system based on multi-modal multi-level feature joint learning, belongs to the technical field of information processing, and solves the problems that most of current mainstream homework automatic scoring methods are based on text content, and results obtained by the homework automatic scoring methods may be incomplete in evaluation and poor in homework scoring efficiency. And the learning motivation of students is reduced. Comprising a text feature extraction module, a multi-modal feature matching module, a multi-level feature fusion module, a job analysis module and a learning behavior application module, and the output end of the text feature extraction module and the output end of the multi-modal feature matching module are connected with the input end of the multi-level feature fusion module. The output end of the multi-level feature fusion module is connected with the input end of the homework analysis module, and the homework analysis module is connected with the learning behavior application module; the text feature extraction module is used for extracting feature vectors of text content from different levels. The method supports more comprehensive formats, and the obtained homework score is more comprehensive and more humanized.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to an online course assignment grading system for multi-modal and multi-level feature joint learning. Background Art

[0002] Automatic homework grading is an important and widely applicable task in the field of natural language processing. Its core goal is to achieve automatic grading of student homework by using machine technology to replace manual graders. This technology can efficiently and accurately evaluate a large number of homework in a short period of time, greatly reducing the time and labor costs required for manual grading. The automatic homework grading system, especially in the field of education, shows extremely broad application prospects and great potential. It can not only improve the work efficiency of teachers, allowing teachers to devote more time and energy to teaching and student tutoring, but also provide effective technical support for large-scale online education platforms.

[0003] In addition, under the premise of ensuring that the scoring is scientific, reasonable and standardized, the use of machines for scoring has obvious advantages over manual scoring. Machine scoring can eliminate the subjectivity in the human scoring process and avoid possible deviations and errors in scoring, thereby evaluating students' homework performance more fairly and objectively. Through highly accurate algorithm models, the automatic homework scoring system can conduct a comprehensive analysis of each homework according to the predetermined scoring criteria, thereby ensuring the accuracy and fairness of the scoring results and maximizing the fairness of the scoring. Therefore, the automatic homework scoring technology can not only improve the quality of educational assessment, but also provide important technical support for educators and promote the modernization of educational assessment methods.

[0004] Currently, most mainstream automatic homework grading methods are based on text content, ignoring other forms of information in the homework, such as multimodal information such as pictures, resulting in incomplete and objective grading. In addition, students' usual learning behaviors are not included in the grading criteria. The results obtained in this way may lead to problems such as incomplete evaluation and decreased student learning motivation.

[0005] Therefore, an online course assignment grading system based on multimodal and multi-level feature joint learning is proposed to solve or alleviate the above problems. Summary of the invention

[0006] The purpose of the present invention is to solve the shortcomings existing in the prior art, and proposes an online course homework grading system with multimodal and multi-level feature joint learning. This system not only takes into account the text content in the homework, but also takes into account the content in the form of pictures. At the same time, a large language model is used to obtain the characteristic information of the homework from different levels such as vocabulary, paragraphs, and documents, and the students' usual learning behavior is also included in the grading criteria. Using this system, not only can the students' homework be comprehensively evaluated, but also the accuracy and objectivity of the grading can be improved.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] An online course homework grading system for multimodal and multi-level feature joint learning, comprising a text feature extraction module, a multimodal feature matching module, a multi-level feature fusion module, a homework analysis module, and a learning behavior application module, wherein the output ends of the text feature extraction module and the multimodal feature matching module are both connected to the input end of the multi-level feature fusion module, the output end of the multi-level feature fusion module is connected to the input end of the homework analysis module, and the homework analysis module is connected to the learning behavior application module;

[0009] The text feature extraction module is used to extract feature vectors of text content from different levels, the multimodal feature matching module is used to perform feature matching on the text and image of the assignment, the multi-level feature fusion module is used to fuse features at different levels, the assignment analysis module is used to evaluate the logic and expressiveness of the assignment content from multiple levels, and the learning behavior application module is used to combine student data.

[0010] Preferably, the text feature extraction module includes a large language model GPT-4, and the text feature extraction module is used to extract feature vectors of text content from different levels, including the following steps:

[0011] Use the large language model GPT-4 to extract information features from the vocabulary,

[0012]

[0013] Use the large language model GPT-4 to extract information features from the paragraphs.

[0014]

[0015] Using the large language model GPT-4 to extract information features of tasks from documents, Vector 文档 = GPT-4(doc), where m and n respectively indicate that the task has m paragraphs and n words, and w i is the i-th word, p i For the jth paragraph of homework, GPT-4(wi ) is w i The feature vector obtained after GPT-4, GPT-4 (p i ) is paragraph p i The feature vector obtained after GPT-4, GPT-4 (doc) is the feature vector obtained after the document doc passes through GPT-4, and finally the vocabulary-level feature vector Vector is obtained. 词汇 , Vector 段落 is the paragraph-level feature vector, Vector 文档 is the document-level feature vector.

[0016] Preferably, the multimodal feature matching module includes a CLIP model, and the multimodal feature matching module is used to perform feature matching on the text and image of the job, including the following steps:

[0017] Use the CLIP model to perform feature matching between the text and image of the assignment;

[0018] The CLIP model is sim(T,I)=cos(CLIP(T)+CLIP(I)), where T is text, I is image, CLIP(T) and CLIP(I) are the text feature vector and image feature vector in the same feature space after CLIP encoding, respectively, and the value range of sim(T,I) is between [-1,1].

[0019] Preferably, the multi-level feature fusion module is used to fuse features at different levels, including the following steps:

[0020] Standardize the features at different levels and unify the scale. 词汇 = normalize(Vector 词汇 ), Vector 段落 = normalize(Vector 段落 ), Vector 文档 = normalize(Vector 文档 );

[0021] Using feature concatenation for feature fusion, Vector 融合 =(Vector 词汇 +Vector 段落 +Vector 文档 ) / 3, where normalize is used for normalization.

[0022] Preferably, the homework analysis module is used to evaluate the logic and expression ability of the homework content from multiple levels, including the following steps:

[0023] The task fusion feature vector and the standard answer feature vector obtained by the multi-level feature fusion module;

[0024] Calculate the cosine similarity between the task fusion feature vector and the standard answer feature vector;

[0025] Evaluate the degree of match between student work and standard answers through similarity;

[0026] Calculate the student's score based on the set similarity weight.

[0027] Preferably, the step of calculating the cosine similarity between the task fusion feature vector and the standard answer feature vector comprises the following steps:

[0028] The cosine similarity between the task fusion feature vector and the standard answer feature vector is calculated using the similarity formula. Among them, Vector 融合 is the task fusion feature vector, answer is the standard answer feature vector, and the value range of C is between [-1,1].

[0029] Preferably, the learning behavior application module is used to combine student data, including the following steps:

[0030] Importing student data, including study time, grade distribution, and homework completion time;

[0031] According to the set behavior score standards and behavior score weights, combined with student data, the learning behavior score is obtained;

[0032] Import the scores of the homework analysis module and the learning behavior scores into the scoring model to calculate the final homework scores.

[0033] Preferably, the scoring model includes Among them, S a , T a , L a , I a They are the frequency of students submitting homework, the time to complete homework, the duration of online learning and the number of interactions, S p , T p , L p , I p They are the preset student homework submission frequency, homework completion time, online learning time and interaction times to get full marks, ω s ,ω t ,ω l ,ω i are the weights of students’ homework submission frequency, homework completion time, online learning time and interaction times, respectively. 作业represents the score of the student's submitted homework, ω o ,ω b are the weights of homework scores and learning behavior scores respectively.

[0034] Preferably, a loss function is also included, and the loss function is used to calculate the score of the learned assignment so as to make the assignment scoring of the system similar to the manual scoring.

[0035] Preferably, the loss function is Among them, y i and They represent the predicted score and label of the i-th composition respectively, and N is the number of compositions.

[0036] The present invention has the following beneficial effects:

[0037] 1. Propose a text feature extraction module, which uses the GPT model to generate text embedding vectors, including deep semantics and logical relationships, and the information content obtained is more comprehensive and detailed. These features not only contain basic grammatical and lexical information, but also capture deeper semantics and logical relationships, thereby improving the accuracy and comprehension of scoring;

[0038] 2. A multimodal feature matching module is proposed. The CLIP model is used to achieve feature matching between text and image modalities. The similarity between the feature vectors of text and image is calculated to evaluate the matching degree between text description and image content, which is used as an important dimension for scoring. This cross-modal feature matching can provide a more comprehensive understanding of the assignment content and improve the accuracy and comprehensiveness of scoring.

[0039] 3. A multi-level feature fusion module is proposed to extract information from the three levels of vocabulary, paragraph and document. First, the features of each level are standardized, and then the features are fused using weighted average, feature concatenation or deep learning network. Finally, the fused features are mapped to the scoring space to improve the comprehensiveness and accuracy of the scoring method.

[0040] 4. Propose a learning behavior application module, which dynamically adjusts parameters based on students' learning behavior data, such as study time, homework submission status, grade distribution, etc., combined with the homework scoring model. Such dynamic adjustment can better reflect students' learning progress and understanding level, which can not only accurately reflect students' true level, but also stimulate students' enthusiasm for learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is a structural block diagram of the present invention;

[0043] Figure 2 It is a flowchart of the present invention;

[0044] Figure 3 It is a picture related to the work in the embodiment of the present invention.

[0045] 1. Text feature extraction module; 2. Multimodal feature matching module; 3. Multi-level feature fusion module; 4. Job analysis module; 5. Learning behavior application module. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0049] In the description of the present invention, it should be understood that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the product of the invention is conventionally placed when in use, or are the orientations or positional relationships conventionally understood by those skilled in the art. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0050] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0051] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0052] An online course assignment grading system based on joint learning of multi-modal and multi-level features, such as Figure 1 and Figure 2 As shown, it includes a text feature extraction module 1, a multimodal feature matching module 2, a multi-level feature fusion module 3, a job analysis module 4, and a learning behavior application module 5, the output ends of the text feature extraction module 1 and the multimodal feature matching module 2 are connected to the input end of the multi-level feature fusion module 3, the output end of the multi-level feature fusion module 3 is connected to the input end of the job analysis module 4, and the job analysis module 4 is connected to the learning behavior application module 5;

[0053] The text feature extraction module 1 is used to extract feature vectors of text content from different levels, the multimodal feature matching module 2 is used to perform feature matching on the text and image of the assignment, the multi-level feature fusion module 3 is used to fuse features at different levels, the assignment analysis module 4 is used to evaluate the logic and expression ability of the assignment content from multiple levels, and the learning behavior application module 5 is used to combine student data.

[0054] The text feature extraction module 1 can extract information features of the task from multiple levels. First, analyze the entire document to capture the theme, main idea and overall structure. Secondly, go deep into the paragraph level to identify the logical relationship, turning points and changes in themes between paragraphs. In addition, it can also perform refined analysis at the vocabulary level to identify keywords, phrases and sentence structures to better understand the meaning and connotation of the text. This meticulous feature extraction process enables the model to fully understand the content of the text and extract key information from it, providing strong support for subsequent tasks and applications. Through this advanced technical means, text data can be processed more efficiently and accurately, providing users with better services and experience;

[0055] The core function of the multimodal feature matching module 2 is to compare the text description with the image content as a scoring dimension. Through the CLIP model, the semantic understanding and association between text and image can be achieved, so as to more accurately evaluate the degree of matching between them.

[0056] The multi-level feature fusion module 3 utilizes the information features of different levels obtained by the text feature extraction module 1. These features include information extracted from different levels such as documents, paragraphs, and vocabulary. In order to make better use of these features and ensure that they are compared at the same scale, these features are first standardized. Standardization can unify the scales between different features and eliminate the problem of unbalanced feature weights caused by different dimensions. Once the features are standardized, the weighted average method can be used for feature fusion. In this process, weights are assigned to features at different levels. These weights can be determined based on their importance in the task, or they can be learned through model training. Through the weighted average method, information features from different levels are organically integrated to form a more comprehensive and comprehensive feature representation. The advantage of this is that it can make full use of information at different levels, thereby improving the performance of the model;

[0057] The homework analysis module 4 uses the homework fusion feature vector and the standard answer feature vector obtained by the multi-level feature fusion module 3. These feature vectors are obtained by deep analysis and feature extraction of the homework text, and can better capture the semantics and logical relationships of the homework. The design of the homework analysis module 4 can effectively help teachers and graders quickly and accurately evaluate the quality of students' homework.

[0058] The learning behavior application module 5 takes into account the students' various learning data. The learning behavior application module 5 can adjust the weight of the score according to the students' learning time, and can also adjust the parameters according to the completion of the students' previous homework and the distribution of grades. Through the dynamic adjustment of the learning behavior application module 5, the model can better adapt to the learning characteristics and needs of different students, provide personalized scoring and feedback, and thus enhance students' learning experience and learning enthusiasm. Such a design can effectively stimulate students' learning motivation, promote their continuous progress and growth, make the scoring method more in line with the students' actual learning situation, and thus enhance students' learning experience and learning enthusiasm.

[0059] Preferably, the text feature extraction module 1 includes a large language model GPT-4, and the text feature extraction module 1 is used to extract feature vectors of text content from different levels, including the following steps:

[0060] Use the large language model GPT-4 to extract information features from the vocabulary,

[0061] Use the large language model GPT-4 to extract information features from the paragraphs.

[0062] Using the large language model GPT-4 to extract information features of tasks from documents, Vector 文档 = GPT-4(doc), where m and n respectively indicate that the task has m paragraphs and n words, and w i is the i-th word, p i For the jth paragraph of homework, GPT-4(w i ) is w i The feature vector obtained after GPT-4, GPT-4 (p i ) is paragraph p i The feature vector obtained after GPT-4, GPT-4 (doc) is the feature vector obtained after the document doc passes through GPT-4, and finally the vocabulary-level feature vector Vector is obtained. 词汇 , Vector 段落 is the paragraph-level feature vector, Vector 文档 is the document-level feature vector.

[0063] The large language model GPT-4 used in the text feature extraction module 1 is a deep learning model with excellent performance in the field of natural language processing. Through GPT-4, it is possible to achieve a deep understanding and analysis of the text content, which goes beyond the limitations of traditional methods. This means that not only the surface grammatical and lexical information in the text is obtained, but also deeper semantic and logical relationships can be explored. That is, the large language model GPT-4 is used to deeply understand and analyze the text content, obtain the basic grammatical and lexical information contained in the text, and the deeper semantic and logical relationships, and extract the information features of the task from different levels such as documents, paragraphs, and vocabulary. Specifically, the text feature extraction module 1 can extract the information features of the task from multiple levels. First, analyze the entire document to capture the theme, main idea, and overall structure. Secondly, go deep into the paragraph level to identify the logical relationship, turning point, and theme change between paragraphs. In addition, it can also perform refined analysis at the vocabulary level to identify keywords, phrases, and sentence structures, so as to better understand the meaning and connotation of the text. This meticulous feature extraction process enables the model to fully understand the content of the text and extract key information from it, providing strong support for subsequent tasks and applications. This advanced technical means can process text data more efficiently and accurately, providing users with better services and experience.

[0064] Preferably, the multimodal feature matching module 2 includes a CLIP model. The multimodal feature matching module 2 is used to perform feature matching on the text and image of the job, including the following steps:

[0065] Use the CLIP model to perform feature matching between the text and image of the assignment;

[0066] The CLIP model is sim(T,I)=cos(CLIP(T)+CLIP(I)), where T is text, I is image, CLIP(T) and CLIP(I) are the text feature vector and image feature vector in the same feature space after CLIP encoding, respectively, and the value range of sim(T,I) is between [-1,1].

[0067] The CLIP model is a model that combines natural language processing and computer vision. It is pre-trained on large-scale data and can convert text and images into vector representations. It can also evaluate the degree of match between text descriptions and image content by calculating the similarity between these vectors. Specifically, the CLIP model encodes text descriptions and images in a unified representation space by learning the contrast relationship between them. This means that both text descriptions and images can be converted into high-dimensional vector representations. Then, by calculating the similarity between these vectors, the degree of match between text descriptions and image content can be evaluated.

[0068] Preferably, the multi-level feature fusion module 3 is used to fuse features at different levels, including the following steps:

[0069] Standardize the features at different levels and unify the scale. 词汇 = normalize(Vector 词汇 ), Vector 段落 = normalize(Vector 段落 ), Vector 文档 = normalize(Vector 文档 );

[0070] Using feature concatenation for feature fusion, Vector 融合 =(Vector 词汇 +Vector 段落 +Vector 文档 ) / 3, where normalize is used for normalization.

[0071] The multi-level feature fusion module 3 utilizes the information features of different levels obtained by the text feature extraction module 1. These features include information extracted from different levels such as documents, paragraphs, and words. In order to better utilize these features and ensure that they are compared at the same scale, these features are first standardized. Standardization can unify the scales between different features and eliminate the problem of unbalanced feature weights caused by different dimensions.

[0072] Preferably, the homework analysis module 4 is used to evaluate the logic and expression ability of the homework content from multiple levels, including the following steps:

[0073] The task fusion feature vector and the standard answer feature vector obtained by the multi-level feature fusion module 3;

[0074] Calculating the cosine similarity between the task fusion feature vector and the standard answer feature vector, calculating the cosine similarity between the task fusion feature vector and the standard answer feature vector, including the following steps,

[0075] The cosine similarity between the task fusion feature vector and the standard answer feature vector is calculated using the similarity formula. Among them, Vector 融合 is the task fusion feature vector, answer is the standard answer feature vector, and the value range of C is between [-1,1];

[0076] Evaluate the degree of match between student work and standard answers through similarity;

[0077] Calculate the student's score based on the set similarity weight.

[0078] The main task in the homework analysis module 4 is to evaluate the similarity between the homework submitted by the students and the standard answer, so as to score the homework. In order to achieve this goal, a commonly used similarity calculation method, namely cosine similarity, is adopted. Cosine similarity measures the angle between two vectors. The closer the value is to 1, the more similar the two vectors are, and the closer the value is to 0, the greater the difference between the two vectors. Specifically, the cosine similarity between the fusion feature vector of the homework submitted by the student and the feature vector of the standard answer is used as another dimension of scoring. By calculating this similarity, the degree of match between the student's homework and the standard answer is evaluated, so as to provide a more objective and accurate score for the homework. By combining the method of multi-level feature fusion and cosine similarity calculation, the text information can be fully utilized to provide a more scientific and reliable basis for homework scoring, thereby improving the fairness and accuracy of the evaluation.

[0079] Preferably, the learning behavior application module 5 is used to combine student data, including the following steps:

[0080] Import student data, including study time, grade distribution, and homework completion time;

[0081] According to the set behavior score standards and behavior score weights, combined with student data, the learning behavior score is obtained;

[0082] The score of homework analysis module 4 and the learning behavior score are imported into the scoring model to calculate the final homework score. The scoring model includes Among them, S a , T a , L a , I a They are the frequency of students submitting homework, the time to complete homework, the duration of online learning and the number of interactions, S p , T p , L p , I p They are the preset student homework submission frequency, homework completion time, online learning time and interaction times to get full marks, ω s ,ω t ,ω l ,ω i are the weights of students’ homework submission frequency, homework completion time, online learning time and interaction times, respectively. 作业 represents the score of the student's submitted homework, ω o ,ω b are the weights of homework scores and learning behavior scores respectively.

[0083] Learning behavior application module 5 comprehensively considers students' various learning data, including study time, completion of previous assignments, grade distribution, etc., to dynamically adjust the parameters of the scoring model. For example, if students spend more time to complete their assignments, the scoring model may be more inclined to consider the quality and depth of the assignments, and the requirements for completion may be relatively high. On the other hand, if students study for a shorter period of time, the scoring model may pay more attention to the basic completion of the assignments to encourage students to spend more time studying. If students have performed well in previous assignments, the scoring model may be more inclined to give high scores and increase the requirements for assignments to motivate students to maintain an excellent learning state. Conversely, if students have lower grades in the past, the scoring model may focus more on providing constructive feedback and support to help students improve their learning outcomes.

[0084] Preferably, a loss function is also included, which is used to calculate the score of the learned assignment so that the system's assignment scoring is similar to the manual scoring. The loss function is Among them, y i and They represent the predicted score and label of the i-th composition respectively, and N is the number of compositions.

[0085] MSE is used as a loss function to assess the rationality of the system's grading of student assignments, so that students who complete their assignments well and behave seriously in class get higher scores than those who complete their assignments poorly. This loss function is then used to calculate the scores of the learned assignments, and by optimizing the parameters, the system's assignment grading is as similar to the manual grading as possible.

[0086] Here is a specific case, the content of the case is as follows:

[0087] The input of the model is the student's text homework content, homework-related pictures, as well as the standard answers set in the background and the student's usual learning behavior.

[0088] (I) Text feature extraction module 1

[0089] This module is mainly used to extract the characteristic information of text assignments, so the required input is the text assignment content of the students. Assume that the text assignment content of the students is

[0090] “Title: Microscopic observation of onion epidermal cells, e.g. Figure 3 As shown,

[0091] Experimental steps:

[0092] 1. Take a piece of fresh onion skin and place it on a glass slide.

[0093] 2. Add iodine solution for dyeing.

[0094] 3. Use a microscope to observe the cell structure and record the cell wall, nucleus and cytoplasm observed.

[0095] Experimental results:

[0096] Under a microscope, onion cells are rectangular, with thick cell walls, clearly visible nuclei, and appear dark blue after staining.

[0097] For vocabulary-level feature extraction of the assignment content, we first need to obtain the keywords and word frequencies in the text, and then perform feature extraction on each keyword and word frequency. The keywords and word frequencies obtained for this text content are:

[0098] "Keyword frequency

[0099] Onion skin 3

[0100] Cell Wall 2

[0101] Nucleus 2

[0102] Microscope 2

[0103] Dyeing 2"

[0104] Then use GPT4 to extract word-level features and get the feature vectors of each word:

[0105] {"Onion skin":[2.5,3.6,4.7,...],

[0106] "Cell wall": [1.4, 2.5, 3.6, ...],

[0107] "Nucleus": [1.4, 2.5, 3.6, ...],

[0108] "Microscope": [1.3, 2.4, 3.5, ...],

[0109] "color":[0.2,1.3,2.4,...]}

[0110] Finally, the feature vectors and word frequency weights of each keyword are combined to obtain the vocabulary-level feature vector [1.38, 2.48, 3.58, ...].

[0111] For paragraph-level feature extraction, we also analyze the structure, logical relationship and main ideas of each paragraph to obtain the characteristics of each paragraph.

[0112] For the experimental steps paragraph, after extracting features through GPT4, the feature vector [3.45, 4.55, 5.65, ...] is obtained.

[0113] For the experimental results paragraph, after GPT4 extracts features, the feature vector [4.50, 5.60, 6.70, ...] is obtained.

[0114] Finally, the features of each paragraph are combined to obtain the paragraph-level feature vector [3.975, 5.075, 6.175, ...].

[0115] For document-level feature extraction, the theme, semantics, and grammatical structure of the entire text are integrated, and after feature extraction by GPT4, a document-level feature vector [4.48, 5.58, 6.68, ...] is obtained.

[0116] Finally, we obtain three levels of feature vectors of the assignment text: vocabulary-level feature vectors [1.38, 2.48, 3.58, ...], paragraph-level feature vectors [3.975, 5.075, 6.175, ...], and document-level feature vectors [4.48, 5.58, 6.68, ...].

[0117] (II) Multimodal feature matching module 2

[0118] The input required by this module is the student's text homework content and homework-related pictures. Assume that the student's text homework content is

[0119] "Title: Microscopic observation of onion epidermal cells

[0120] Experimental steps:

[0121] 1. Take a piece of fresh onion skin and place it on a glass slide.

[0122] 2. Add iodine solution for dyeing.

[0123] 3. Use a microscope to observe the cell structure and record the cell wall, nucleus and cytoplasm observed.

[0124] Experimental results:

[0125] Under a microscope, onion cells are rectangular, with thick cell walls, clearly visible nuclei, and appear dark blue after staining.

[0126] The pictures related to the assignment are Figure 2 As shown,

[0127] Using the CL IP model to extract the feature vector of the text, we get [0.45, 0.55, 0.65, ...], and using the CL IP model to extract the feature vector of the image, we get [0.43, 0.53, 0.63, ...].

[0128] Then the cosine similarity Cos([0.45, 0.55, 0.65, ...], [0.43, 0.53, 0.63, ...]) between the text feature vector and the image feature vector is calculated to obtain a similarity score of 0.95.

[0129] (III) Multi-level feature fusion module 3

[0130] Assume that the vocabulary-level, paragraph-level, and document-level feature vectors obtained in step 1 are [1.38, 2.48, 3.58, ...], [3.975, 5.075, 6.175, ...], and [4.48, 5.58, 6.68, ...], respectively. Standardize these three features to get [0.85, 0.90, 0.95, ...], [0.89, 0.94, 0.99, ...], and [0.87, 0.92, 0.9 7,...], and then weighted average, assuming that the vocabulary weight is 0.3, the paragraph weight is 0.4, and the document weight is 0.3, then the fusion feature obtained after weighted average is [0.3*0.85+0.4*0.89+0.3*0.87,0.3*0.90+0.4*0.94+0.3*0.92,0.3*0.95+0.4*0.99+0.3*0.97,...] = [0.87,0.92,0.97,...]

[0131] (IV) Operation Analysis Module 4

[0132] Assume that the obtained student homework fusion features are [0.87, 0.92, 0.97, ...], the similarity score between text and image is 0.95, and the standard answer is

[0133] "Title: Microscopic observation of plant cells

[0134] Experimental steps:

[0135] 1. Prepare a piece of onion skin and place it on a glass slide.

[0136] 2. Add iodine solution for dyeing.

[0137] 3. Observe under a microscope and record the structural features of the cell wall, nucleus and vacuole.

[0138] Experimental results:

[0139] Under the microscope, onion cells appear polygonal, with clear cell wall and vacuole structures, and the nucleus appears dark blue after staining. "

[0140] Extract the feature vector of the standard answer and get the comprehensive feature vector as [0.75, 0.85, 0.95, ...];

[0141] Calculate the cosine similarity of the feature vector of the student homework fusion feature and the standard answer Cos i ne_Simi lar ity([0.87,0.92,0.97,...],[0.75,0.85,0.95,...])=0.85. Assume that the similarity weight of the text image is 0.3 and the similarity weight of the homework and the answer is 0.7. The final score is (0.7*0.85+0.3*0.95)*100=88.

[0142] (V) Learning Behavior Application Module 5

[0143] Assume that the score after passing the homework analysis module 4 is 88, the student's homework submission frequency is 8 times / month, the homework completion time is 2 hours / time, the online learning time is 10 hours / week, and the number of interactions is 15 times / week. The weights of the above four behaviors are all set to 0.25. At the same time, the student's homework submission frequency is set to 10 times / month, the homework completion time is 3 hours / time, the online learning time is 12 hours / week, and the number of interactions is 20 times / week. When the student can get full marks in this area, the student's behavior score is

[0144] (0.25*(8 / 10)+0.25*(2 / 3)+0.25*(10 / 12)+0.25*(15 / 20)=0.25*0.8+0.25*0.67+0.25*0.83+0.25*0.75)*100=76.

[0145] Assuming that the weight of the student's homework score is 0.8 and the weight of the learning behavior is 0.2, the student's final score is calculated to be 88*0.8+76*0.2=85.6.

[0146] From this we can see that when students do not hand in their homework on time and do not interact with teachers, it will have a certain impact on their final scores.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An online course assignment grading system based on multi-modal and multi-level feature joint learning, characterized in that: The system comprises a text feature extraction module (1), a multimodal feature matching module (2), a multi-level feature fusion module (3), a task analysis module (4), and a learning behavior application module (5), wherein the output ends of the text feature extraction module (1) and the multimodal feature matching module (2) are connected to the input end of the multi-level feature fusion module (3), the output end of the multi-level feature fusion module (3) is connected to the input end of the task analysis module (4), and the task analysis module (4) is connected to the learning behavior application module (5); The text feature extraction module (1) is used to extract feature vectors of text content from different levels, the multimodal feature matching module (2) is used to perform feature matching between the text and image of the homework, the multi-level feature fusion module (3) is used to fuse features at different levels, the homework analysis module (4) is used to evaluate the logic and expression ability of the homework content from multiple levels, and the learning behavior application module (5) is used to combine student data.

2. According to claim 1, a multi-modal multi-level feature joint learning online course assignment grading system is characterized in that: The text feature extraction module (1) includes a large language model GPT-4. The text feature extraction module (1) is used to extract feature vectors of text content from different levels, and includes the following steps: Use the large language model GPT-4 to extract information features from the vocabulary, Use the large language model GPT-4 to extract information features from the paragraphs. Using the large language model GPT-4 to extract information features from documents, Vector 文档 = GPT-4(doc), where m and n respectively indicate that the task has m paragraphs and n words, and w i is the i-th word, p i For the jth paragraph of homework, GPT-4(w i ) is w i The feature vector obtained after GPT-4, GPT-4 (p i ) is paragraph p i The feature vector obtained after GPT-4, GPT-4 (doc) is the feature vector obtained after the document doc passes through GPT-4, and finally the vocabulary-level feature vector Vector is obtained. 词汇 , Vector 段落 is the paragraph-level feature vector, Vector 文档 is the document-level feature vector.

3. According to claim 1, the online course assignment grading system of multi-modal and multi-level feature joint learning is characterized in that: The multimodal feature matching module (2) comprises a CLIP model, and the multimodal feature matching module (2) is used to perform feature matching on the text and image of the job, and comprises the following steps: Use the CLIP model to perform feature matching between the text and image of the assignment; The CLIP model is sim(T,I)=cos(CLIP(T)+CLIP(I)), where T is text, I is image, CLIP(T) and CLIP(I) are the text feature vector and image feature vector in the same feature space after CLIP encoding, respectively, and the value range of sim(T,I) is between [-1,1].

4. The online course assignment grading system of multi-modal and multi-level feature joint learning according to claim 1 is characterized in that: The multi-level feature fusion module (3) is used to fuse features at different levels, and comprises the following steps: Standardize the features at different levels and unify the scale. 词汇 =normalize(Vector 词汇 ), Vector 段落 =normalize(Vector 段落 ), Vector 文档 =normalize(Vector 文档 ); Using feature concatenation for feature fusion, Vector 融合 =(Vector 词汇 +Vector 段落 +Vector 文档 ) / 3, where normalize is used for normalization.

5. The online course assignment grading system of multi-modal and multi-level feature joint learning according to claim 1 is characterized in that: The homework analysis module (4) is used to evaluate the logic and expression ability of the homework content from multiple levels, including the following steps: The task fusion feature vector and the standard answer feature vector obtained by the multi-level feature fusion module (3); Calculate the cosine similarity between the task fusion feature vector and the standard answer feature vector; Evaluate the degree of match between student work and standard answers through similarity; Calculate the student's score based on the set similarity weight.

6. The online course assignment grading system of multi-modal and multi-level feature joint learning according to claim 5 is characterized in that: The calculation of the cosine similarity between the fusion feature vector and the standard answer feature vector comprises the following steps: The cosine similarity between the task fusion feature vector and the standard answer feature vector is calculated using the similarity formula. Among them, Vector 融合 is the task fusion feature vector, answer is the standard answer feature vector, and the value range of C is between [-1,1].

7. The online course assignment grading system of multi-modal and multi-level feature joint learning according to claim 1 is characterized in that: The learning behavior application module (5) is used to combine student data and includes the following steps: Importing student data, including study time, grade distribution, and homework completion time; According to the set behavior score standards and behavior score weights, combined with student data, the learning behavior score is obtained; The score of the homework analysis module (4) and the learning behavior score are imported into the scoring model to calculate the final homework score.

8. The online course assignment grading system of multi-modal and multi-level feature joint learning according to claim 7 is characterized in that: The scoring model includes Among them, S a , T a , L a , I a They are the frequency of students submitting homework, the time to complete homework, the duration of online learning and the number of interactions, S p , T p , L p , I p They are the preset student homework submission frequency, homework completion time, online learning time and interaction times to get full marks, ω s ,ω t ,ω l ,ω i are the weights of students’ homework submission frequency, homework completion time, online learning time and interaction times, respectively. 作业 represents the score of the student's submitted homework, ω o ,ω b are the weights of homework scores and learning behavior scores respectively.

9. The online course assignment grading system of multi-modal and multi-level feature joint learning according to claim 1 is characterized in that: It also includes a loss function, which is used to calculate the score of the learned homework so that the system's homework scoring is similar to the manual scoring.

10. The online course assignment grading system of multi-modal and multi-level feature joint learning according to claim 9 is characterized in that: The loss function is Among them, y i and They represent the predicted score and label of the i-th composition respectively, and N is the number of compositions.

Images