Language learning evaluation method and system based on large language model feedback
By conducting multiple logical, coherent and grammatical analysis of the content of language writing text, combining learning behavior patterns and emotional analysis, the problem of singularity and one-sidedness of language learning evaluation in the existing technology is solved, and a more comprehensive and accurate Language Learning Assessment.
Patent Information
- Application Number
- CN202411001912.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-25
AI Technical Summary
The existing language learning evaluation method based on large language models lacks logical and coherent analysis of the content of writing texts, cannot comprehensively and accurately evaluate the user's language learning effect, and lacks analysis of learning behavior patterns and emotions, resulting in language learning The assessment is too single and one-sided.
By obtaining the content data of language writing text for preprocessing and multiple content analysis feedback, combining preset coding tables and matrix coding for learning behavior patterns, using dual attention mechanisms for emotion analysis, and combining language learning based on these results Evaluate.
It realizes multi-dimensional analysis of language learning, improves the accuracy and comprehensiveness of evaluation, helps users to understand their own problems and emotional changes in language learning, and improves the quality of language learning and evaluation results.
Smart Images

Figure CN119026596A_ABST
Abstract
Description
Technical Field
[0001] The present invention mainly relates to the field of data processing technology, and in particular to a language learning evaluation method and system based on large language model feedback. Background Art
[0002] With the rapid development of artificial intelligence technology, large language models have received widespread attention in the application of foreign language learning and writing, bringing learners a new method of autonomous learning assessment. In the current language learning assessment based on large language models, most of them directly score the content of the writing text through the large language model, and use the scoring results to evaluate the language learning effect. There is a lack of analysis and feedback on the content of the writing text, which makes it impossible for users to clearly know what problems exist in their language learning and writing. Only a small number of people will analyze and feedback the content of the writing text through the large language model, but they only analyze and feedback the grammar of the writing text content, and do not consider the analysis of the logic and coherence of the writing text content, resulting in the inability to comprehensively and accurately evaluate the user's language learning effect. At the same time, in the current language learning assessment, there is a lack of analysis of learning behavior patterns and sentiment analysis of the user's learning process, which makes it impossible to clearly understand the behavior and emotional expression of the language learning process, making the language learning assessment too single. In addition, the lack of analysis of learning behavior patterns and sentiment analysis of the user's learning process will lead to low accuracy in language learning impact analysis, making the language learning assessment one-sided, so that the user's language learning quality cannot be effectively improved, resulting in the language learning assessment failing to achieve the expected effect. Summary of the invention
[0003] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a language learning evaluation method and system based on large language model feedback, which can effectively improve the user's language learning quality and achieve a more ideal effect of language learning evaluation.
[0004] In order to solve the above technical problems, the present invention provides a language learning evaluation method based on large language model feedback, the method comprising:
[0005] Acquiring language writing text content data, and preprocessing the language writing text content data to obtain preprocessed language writing text content data;
[0006] Obtain different input instructions from the user, and perform several writing content analysis and feedback on the preprocessed language writing text content data using a large language model based on the different input instructions to obtain several writing feedback data;
[0007] Acquire reflection log information and feedback record information generated by a number of writing feedback data, and perform learning behavior pattern analysis based on a preset coding table using the reflection log information and feedback record information in combination with matrix coding to obtain a learning behavior pattern analysis result;
[0008] Based on the reflection log information, sentiment analysis is performed using a dual-path attention mechanism to obtain feedback sentiment information;
[0009] Performing a qualitative analysis of the impact of language learning based on the learning behavior pattern analysis results and the feedback emotional information to obtain a qualitative analysis result of the impact of language learning;
[0010] A comprehensive evaluation of language learning is performed based on the learning behavior pattern analysis results, the feedback emotional information and the qualitative analysis results of the language learning impact to obtain a comprehensive evaluation result.
[0011] Optionally, the preprocessing of the language writing text content data to obtain the preprocessed language writing text content data includes:
[0012] Performing a garbled processing on the language writing text content data to obtain the language writing text content data after the garbled processing;
[0013] The language writing text content data after the garbled characters are removed is formatted to obtain the pre-processed language writing text content data.
[0014] Optionally, the method uses a large language model to perform writing content analysis and feedback on the preprocessed language writing text content data several times based on different input instructions to obtain several writing feedback data, including:
[0015] Inputting the preprocessed language writing text content data into a large language model, the large language model performs several content logic analysis feedbacks, content coherence analysis feedbacks and grammatical analysis feedbacks based on different input instructions, and obtains several content logic analysis feedback data, content coherence analysis feedback data and grammatical analysis feedback data, wherein the large language model includes a content logic analysis feedback module, a content coherence analysis feedback module and a grammatical analysis feedback module;
[0016] The content logic analysis feedback module includes a topic score calculation submodule, a content relevance analysis submodule and a logic score calculation submodule;
[0017] The content coherence analysis feedback module includes a sentence vector representation extraction submodule, a chapter overall vector representation extraction submodule and a coherence analysis submodule;
[0018] The grammatical analysis feedback module includes a part-of-speech analysis submodule and a grammatical analysis submodule.
[0019] Optionally, the topic score calculation submodule includes a high-level semantic information extractor, a spatial information adapter based on a convolutional neural network, and a linear classifier. The high-level semantic information extractor includes several double-tower models composed of 7-layer transformers, and the linear classifier includes a binary tree and a support vector machine model based on a radial basis function.
[0020] Optionally, the performing of learning behavior pattern analysis based on a preset coding table using the reflection log information and feedback record information in combination with matrix coding to obtain a learning behavior pattern analysis result includes:
[0021] Based on the preset coding table, node coding and marking are performed on the reflection log information and the feedback record information to obtain a plurality of coding categories;
[0022] Calculate coding frequency distribution based on several coding categories;
[0023] Based on several coding categories, matrix coding and preset project maps are used to obtain the correlation between coding categories;
[0024] Based on the correlation between the coding frequency distribution and the coding categories, the behavior dimension classification is performed to obtain the behavior dimension classification result;
[0025] Based on the behavioral dimension classification results, coding frequency distribution and the correlation between coding categories, a learning behavior pattern analysis is performed to obtain a learning behavior pattern analysis result.
[0026] Optionally, the obtaining of associations between coding categories based on a plurality of coding categories using matrix coding and a preset project map includes:
[0027] Obtaining a node coding matrix of each coding category, and transforming the node coding matrix based on a polynomial clustering algorithm to obtain first hypergraph data;
[0028] Convert the node encoding matrix based on a K-nearest neighbor algorithm to obtain second hypergraph data;
[0029] Based on the fusion of the first hypergraph data and the second hypergraph data, a hypergraph association matrix is obtained;
[0030] Based on the hypergraph association matrix combined with a preset project map, association mining of coding categories is performed to obtain association relationships between coding categories.
[0031] Optionally, performing sentiment analysis based on the reflection log information using a dual-path attention mechanism to obtain feedback sentiment information includes:
[0032] Acquire training log data, perform sentiment orientation annotation on the training log data, obtain sentiment orientation annotated training log data, and build a sentiment orientation classification model based on the sentiment orientation annotated training log data;
[0033] Vectorizing the reflection log information based on a global vector to obtain a first text vector matrix;
[0034] Vectorizing the reflection log information based on the word conversion vector to obtain a second text vector matrix;
[0035] Based on the sentiment tendency classification model, sentiment tendency classification is performed using the first text vector matrix and the second text vector matrix to obtain sentiment tendency information;
[0036] The target modifier is determined based on the preset dictionary using the reflection log information, the sentiment intensity is calculated based on the target modifier using the sentiment tendency information combined with a two-way attention mechanism, and feedback sentiment information is generated based on the sentiment tendency information and the sentiment intensity.
[0037] Optionally, performing a qualitative analysis of the impact of language learning based on the learning behavior pattern analysis result and the feedback emotional information to obtain a qualitative analysis result of the impact of language learning includes:
[0038] Acquire historical language learning impact analysis data, construct a target probability model based on the historical language learning impact analysis data using a logical sequence, and construct a Bayesian network based on the target probability model using a functor and a chi-square validation method;
[0039] Based on the historical language learning impact analysis data, a qualitative analysis topological network is constructed using factor relationship analysis method;
[0040] Based on the Bayesian network and the qualitative analysis topological network, the qualitative analysis of the impact of language learning is performed using the learning behavior pattern analysis results and the feedback emotional information to obtain the qualitative analysis results of the impact of language learning.
[0041] Optionally, the comprehensive evaluation of language learning is performed based on the learning behavior pattern analysis results, the feedback emotional information and the qualitative analysis results of the language learning impact to obtain a comprehensive evaluation result, including:
[0042] Based on the learning behavior pattern analysis results, feedback emotional information and language learning impact qualitative analysis results, a preset evaluation scheme is used to conduct a comprehensive evaluation of language learning, obtain a comprehensive evaluation result, and generate a language learning evaluation report based on the comprehensive evaluation result.
[0043] In addition, the present invention also provides a language learning evaluation system based on large language model feedback, the system comprising:
[0044] Writing content data preprocessing module: used to obtain language writing text content data, and preprocess the language writing text content data to obtain the preprocessed language writing text content data;
[0045] Writing content analysis and feedback module: used to obtain different input instructions from users, and based on different input instructions, use a large language model to perform several writing content analysis and feedback on the preprocessed language writing text content data to obtain several writing feedback data;
[0046] Learning behavior pattern analysis module: used to obtain reflection log information and feedback record information generated by a number of writing feedback data, and to perform learning behavior pattern analysis based on a preset coding table using the reflection log information and feedback record information combined with matrix coding to obtain a learning behavior pattern analysis result;
[0047] Sentiment analysis module: used to perform sentiment analysis based on the reflection log information using a dual-path attention mechanism to obtain feedback sentiment information;
[0048] A language learning impact qualitative analysis module: used to perform a language learning impact qualitative analysis based on the learning behavior pattern analysis results and the feedback emotional information, and obtain a language learning impact qualitative analysis result;
[0049] Language learning comprehensive evaluation module: used to conduct a comprehensive evaluation of language learning based on the learning behavior pattern analysis results, feedback emotional information and language learning impact qualitative analysis results to obtain a comprehensive evaluation result.
[0050] In an embodiment of the present invention, the large language model performs several content logic analysis feedback, content coherence analysis feedback and grammatical analysis feedback on the preprocessed language writing text content data based on different input instructions, analyzes the language writing text content from multiple dimensions, enables users to understand their own problems in language learning and writing more comprehensively and accurately, and promotes the user's self-learning ability. Based on the preset coding table, the reflection log information and feedback record information are combined with matrix coding to perform learning behavior pattern analysis, which can more intuitively understand the user's behavior pattern in the process of receiving and applying language learning feedback. Based on the reflection log information, the dual-way attention mechanism is used for sentiment analysis, which can clearly understand the user's emotional changes in language learning, so that the feedback emotional information obtained is closer to the user's actual emotions. Based on the learning behavior pattern analysis results and feedback emotional information, a qualitative analysis of language learning impact is performed to improve the accuracy of language learning impact analysis and avoid the one-sidedness of language learning evaluation. The user's language learning quality can be effectively improved, and the language learning evaluation can achieve a more ideal effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0052] Figure 1 is a flow chart of a language learning evaluation method based on large language model feedback in an embodiment of the present invention;
[0053] Figure 2 It is a schematic diagram of the structural composition of a language learning evaluation system based on large language model feedback in an embodiment of the present invention. DETAILED DESCRIPTION
[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] Embodiment 1
[0056] See also Figure 1 , Figure 1 : is a flow chart of a language learning evaluation method based on large language model feedback in an embodiment of the present invention, the method comprising:
[0057] S11: acquiring language writing text content data, and preprocessing the language writing text content data to obtain preprocessed language writing text content data;
[0058] In the specific implementation process of the present invention, the preprocessing of the language writing text content data to obtain the preprocessed language writing text content data includes: removing garbled characters from the language writing text content data to obtain the language writing text content data after the garbled characters are removed; formatting the language writing text content data after the garbled characters are removed to obtain the preprocessed language writing text content data.
[0059] Specifically, the language writing text content data input by the user is obtained based on a preset script, and the language writing text content is the content written by the user in a second language. In the input language writing text content data, garbled characters may appear, and the language writing text content data needs to be processed to remove the garbled characters, extract all font features in the language writing text content data, and divide the fonts into normal fonts, garbled fonts and pending fonts according to the font features; extract the dot matrix image of the characters in the pending font, and calculate the similarity between the dot matrix image and the corresponding encoding based on the garbled detection algorithm of the image statistical features, and judge the normal characters or garbled characters in the pending font according to the similarity; perform vertical and horizontal editing and correction on the garbled characters in the pending font and the garbled characters in the garbled font, remove the garbled characters, and obtain the language writing text content data after the garbled characters are removed. The language writing text content data after the garbled characters are formatted, and the preset format parameters are obtained. The language writing text content data after the garbled characters are formatted according to the preset format parameters. After the formatting is completed, the pre-processed language writing text content data can be obtained.
[0060] S12: obtaining different input instructions from the user, and performing several writing content analysis and feedback on the preprocessed language writing text content data using a large language model based on the different input instructions to obtain several writing feedback data;
[0061] In the specific implementation process of the present invention, the preprocessed language writing text content data is subjected to several writing content analysis feedbacks by using a large language model based on different input instructions to obtain several writing feedback data, including: the preprocessed language writing text content data is input into the large language model, and the large language model performs several content logic analysis feedbacks, content coherence analysis feedbacks and grammatical analysis feedbacks based on different input instructions to obtain several content logic analysis feedback data, content coherence analysis feedback data and grammatical analysis feedback data, wherein the large language model includes a content logic analysis feedback module, a content coherence analysis feedback module and a grammatical analysis feedback module; the content logic analysis feedback module includes a topic score calculation submodule, a content relevance analysis submodule and a logic score calculation submodule; the content coherence analysis feedback module includes a sentence vector representation extraction submodule, a paragraph overall vector representation extraction submodule and a coherence analysis submodule; the grammatical analysis feedback module includes a part of speech analysis submodule and a grammatical analysis submodule.
[0062] Furthermore, the topic score calculation submodule includes a high-level semantic information extractor, a spatial information adapter based on a convolutional neural network, and a linear classifier. The high-level semantic information extractor includes several double-tower models composed of 7-layer transformers, and the linear classifier includes a binary tree and a support vector machine model based on a radial basis function.
[0063] Specifically, different input instructions of the user are obtained, and different input instructions are generated according to different prompts. The user can make the large language model analyze and feedback the text writing content according to different input instructions. The input instructions are determined according to the actual needs of the user, and multiple writing content feedback analyses are performed to more clearly understand the content that needs to be modified. If the user is unclear about the writing feedback data, the input instructions can be adjusted to re-analyze and feedback the text writing content. The pre-processed language writing text content data is input into the large language model, and the large language model performs several content logic analysis feedback, content coherence analysis feedback and grammatical analysis feedback based on different input instructions, wherein the large language model includes a content logic analysis feedback module, a content coherence analysis feedback module and a grammatical analysis feedback module. The content logic analysis feedback module includes a topic score calculation submodule, a content relevance analysis submodule and a logic score calculation submodule; the topic score calculation submodule includes a high-level semantic information extractor, a spatial information adapter based on a convolutional neural network and a linear classifier. The high-level semantic information extractor includes a plurality of twin-tower models composed of 7-layer transformers. Transformer is a sequence model based on an attention mechanism. Each layer of transformer adopts a self-attention mechanism. The twin-tower model composed of transformers has a stronger semantic understanding ability and can more accurately capture the deep semantic relevance of language writing text content data. At the same time, compared with traditional models, the twin-tower model is better at processing long text information. Therefore, the semantic understanding effect of language writing text is more reliable. One of the 7-layer transformers is responsible for the instruction information input by the user. Therefore, the high-level semantic information extractor can better extract semantic information and obtain more reliable semantic features. The spatial information adapter based on convolutional neural network has three sections of residual convolutional neural network, which is used to enhance the robustness of the model structure and improve the representation ability of fine-grained information. The linear classifier includes a binary tree and a support vector machine model based on radial basis function. The radial basis function is used as the kernel function of the support vector machine model. The radial basis function has better approximation performance and flexibility and can be applied to a variety of application scenarios. At the same time, the penalty parameter and kernel function of the support vector machine model are optimized by cross-validation. The pre-processed language writing text content data is input into the topic score calculation submodule to calculate the topic score and obtain the topic score. The topic score is the score of the writing text content around the topic; the content relevance analysis submodule is used to perform relevance analysis based on the pre-processed language writing text content data, segment the pre-processed language writing text content data, obtain content segmentation, and perform relevance analysis based on the content segmentation combined with the context to obtain relevance analysis data.The logic score calculation submodule is used to calculate the information entropy based on the preprocessed language writing text content data, convert the information entropy into the text language logic score, and then generate content logic analysis feedback data based on the topic score, association analysis data and text language logic score. The logic analysis feedback data can be used to understand the user's writing logic ability and whether they can express their own understanding in a good way. The content coherence analysis feedback module includes a sentence vector representation extraction submodule, a chapter overall vector representation extraction submodule and a coherence analysis submodule. The sentence vector representation module is used to divide the preprocessed language writing text content data into sentence granularity according to the pretrained autoregressive model and generate sentence vector representations; the chapter overall vector representation extraction submodule is used to generate attention output using the preprocessed language writing text content data according to a convolutional neural network, and obtain the chapter overall vector representation according to the attention output using the weight matrix of the feedforward neural network; the coherence analysis submodule is composed of a three-layer feedforward neural network, calculates the similarity between sentence vector representations, constructs a similarity matrix according to the similarity between sentence vector representations, generates coherence feature parameters according to the similarity matrix combined with the chapter overall vector representation, and uses the feedforward neural network to perform coherence analysis on the writing text content according to the coherence feature parameters to obtain content coherence analysis feedback data. The grammatical analysis feedback module includes a part-of-speech analysis submodule and a grammatical analysis submodule. The part-of-speech analysis submodule is used to perform part-of-speech analysis on the preprocessed language writing text content data according to the pre-trained part-of-speech analysis model. The grammatical analysis submodule is used to establish a tree network according to the grammatical analysis tree, and add a multi-layer perceptron for classification at the root node of the tree network. The multi-layer perceptron adopts a hierarchical attention mechanism, and performs grammatical analysis of the writing text content according to the tree network using the part-of-speech analysis results to obtain grammatical analysis feedback data. The writing feedback data is composed of content logic analysis feedback data, content coherence analysis feedback data and grammatical analysis feedback data. The user can modify the writing text content according to the writing feedback data. After modification, the modified language writing text content data can be input into the large language model again for writing content analysis feedback until the required requirements are met.
[0064] S13: obtaining reflection log information and feedback record information generated by a plurality of writing feedback data, and performing learning behavior pattern analysis based on a preset coding table using the reflection log information and feedback record information in combination with matrix coding to obtain a learning behavior pattern analysis result;
[0065] In the specific implementation process of the present invention, the reflection log information and feedback record information are combined with matrix coding to perform learning behavior pattern analysis based on the preset coding table to obtain learning behavior pattern analysis results, including: node coding and marking the reflection log information and feedback record information based on the preset coding table to obtain several coding categories; calculating the coding frequency distribution based on several coding categories; obtaining the association between coding categories based on several coding categories using matrix coding and preset project maps; classifying behavior dimensions based on the coding frequency distribution and the association between coding categories to obtain behavior dimension classification results; performing learning behavior pattern analysis based on the behavior dimension classification results, the coding frequency distribution and the association between coding categories to obtain learning behavior pattern analysis results.
[0066] Furthermore, the method of obtaining the association relationship between coding categories based on several coding categories using matrix coding and a preset project map includes: obtaining a node coding matrix for each coding category, converting the node coding matrix based on a polynomial clustering algorithm to obtain first hypergraph data; converting the node coding matrix based on a K-nearest neighbor algorithm to obtain second hypergraph data; fusing the first hypergraph data and the second hypergraph data to obtain a hypergraph association matrix; and performing association mining of coding categories based on the hypergraph association matrix combined with a preset project map to obtain the association relationship between coding categories.
[0067] Specifically, the reflection log information and feedback record information generated by several writing feedback data are obtained. The user reflects and gives feedback based on the multiple writing feedback data, thereby generating the reflection log information and feedback record information. The reflection log information may include the feelings of modifying the writing content and the learning behavior adopted in the modification process, and the feedback record information may include the number of modifications of the writing content, the modified content and the modification behavior, etc. The reflection log information and feedback record information are coded and annotated based on the preset coding table. The preset coding table is constructed based on the learning behavior in the historical feedback process. The reflection log information and feedback record information are coded and annotated through the preset coding table to create nodes. Each node represents a coding category, and several coding categories are obtained. Each coding category corresponds to a corresponding behavior dimension. The coding frequency distribution is calculated based on several coding categories, that is, the frequency distribution of the coding categories is statistically analyzed through data statistical tools. The node coding matrix of each coding category is obtained, and the node coding matrix of each coding category is generated through the matrix coding query in the software. The node encoding matrix is transformed based on the polynomial clustering algorithm, that is, the node encoding matrix is clustered by encoding categories, and each category obtained can be used as a hyperedge of the first hypergraph data, and each node in the node encoding matrix is a node in the first hypergraph data, so as to obtain the first hypergraph data. The node encoding matrix is transformed based on the K nearest neighbor algorithm, and the adjacent node matrix of the node encoding matrix is obtained according to the K nearest neighbor algorithm, and several hyperedges are generated according to the adjacent node matrix. The Euclidean distance between each encoding node in the node encoding matrix and the hyperedge is calculated, and several hyperedges whose Euclidean distance reaches a preset threshold are selected, so as to obtain the second hypergraph data. Based on the fusion of the first hypergraph data and the second hypergraph data, the nodes in the first hypergraph data and the hyperedges in the second hypergraph data are constructed into a matrix to obtain the hypergraph association matrix, which can reduce the scale of the generated hypergraph association matrix while ensuring the quality of the hypergraph association matrix. The hypergraph is a data structure, which is usually expressed by a hypergraph association matrix. In the hypergraph data structure, the edge becomes a hyperedge and can connect multiple nodes, so the hypergraph association matrix can obtain the association relationship more naturally and quickly. Based on the hypergraph association matrix combined with the preset project map, the association mining of the coding categories is performed. The preset project map can display the overall structure of the coding categories, and the association relationship between the coding categories can be quickly obtained in combination with the hypergraph association matrix. Based on the coding frequency distribution and the association relationship between the coding categories, the behavior dimension classification is performed to obtain the behavior dimension classification result. The behavior dimension may include seeking feedback information, understanding feedback information, using feedback information, providing feedback information and managing emotions.Based on the association between the behavior dimension classification results, coding frequency distribution and coding categories, learning behavior pattern analysis is performed, that is, the frequency of each behavior dimension of the user is analyzed, and the user's behavior pattern in the learning process is analyzed through the association between the behavior dimension classification results, coding frequency distribution and coding categories to obtain the learning behavior pattern analysis results, so as to more intuitively understand the user's behavior pattern in the process of receiving and applying language learning feedback.
[0068] S14: Perform sentiment analysis based on the reflection log information using a dual-path attention mechanism to obtain feedback sentiment information;
[0069] In the specific implementation process of the present invention, the sentiment analysis is performed based on the reflection log information using a two-way attention mechanism to obtain feedback sentiment information, including: obtaining training log data, annotating the training log data with sentiment tendencies, obtaining the training log data with sentiment tendencies annotated, and constructing a sentiment tendency classification model based on the training log data with sentiment tendencies annotated; vectorizing the reflection log information based on a global vector to obtain a first text vector matrix; vectorizing the reflection log information based on a word conversion vector to obtain a second text vector matrix; based on the sentiment tendency classification model, using the first text vector matrix and the second text vector matrix to perform sentiment tendency classification to obtain sentiment tendency information; based on a preset dictionary, using the reflection log information to determine a target modifier, based on the target modifier, using the sentiment tendency information combined with a two-way attention mechanism to calculate sentiment intensity, and generating feedback sentiment information based on the sentiment tendency information and sentiment intensity.
[0070] Specifically, the training log data, i.e., the experimental log information used for training the model, is obtained, and the training log data is labeled with sentiment tendencies. The corresponding sentiment tendencies can be labeled through labels to obtain the training log data with sentiment tendencies labeled, and a sentiment tendency classification model is constructed based on the training log data with sentiment tendencies labeled, that is, the training log data with sentiment tendencies labeled is input into a deep neural network for training to obtain a convergence model, which is the sentiment tendency classification model. The reflection log information is vectorized based on a global vector, the reflection log information is globally vectorized to obtain a first text vector, a co-occurrence matrix is constructed through the global vector, the first text vector is trained based on the co-occurrence matrix, and a first text vector matrix is obtained. The reflection log information is vectorized based on a word conversion vector to obtain a local context window, and the reflection log information is vectorized based on the local context window using a word conversion vector to obtain a second text vector matrix. Based on the sentiment tendency classification model, sentiment tendency classification is performed using the first text vector matrix and the second text vector matrix, and feature fusion is performed based on the first text vector matrix and the second text vector matrix to obtain target fusion features, and the target fusion features are input into the sentiment tendency classification model to perform sentiment tendency classification to obtain sentiment tendency information, and the sentiment tendency information may include positive emotions such as positive or negative emotions such as negative. Based on the preset dictionary, the reflection log information is used to determine the target modifier, and the preset dictionary contains a large number of polarity words and modifiers and their corresponding sentiment polarity types. Based on the target modifier, the sentiment intensity is calculated using the sentiment tendency information combined with the two-way attention mechanism, and the attention score is calculated based on the target modifier using the sentiment tendency information combined with the two-way attention mechanism, and the sentiment intensity is calculated according to the attention score through the activation function. Feedback sentiment information is formed based on the sentiment tendency information and sentiment intensity.
[0071] S15: performing a qualitative analysis of the impact of language learning based on the learning behavior pattern analysis results and the feedback emotional information to obtain a qualitative analysis result of the impact of language learning;
[0072] In the specific implementation process of the present invention, the qualitative analysis of the impact of language learning is performed based on the learning behavior pattern analysis results and the feedback emotional information to obtain the qualitative analysis results of the impact of language learning, including: obtaining historical language learning impact analysis data, constructing a target probability model based on the historical language learning impact analysis data using a logical sequence, and constructing a Bayesian network based on the target probability model using a functor and a chi-square validation method; constructing a qualitative analysis topological network based on the historical language learning impact analysis data using a factor relationship analysis method; and performing a qualitative analysis of the impact of language learning based on the Bayesian network and the qualitative analysis topological network using the learning behavior pattern analysis results and the feedback emotional information to obtain the qualitative analysis results of the impact of language learning.
[0073] Specifically, historical language learning impact analysis data is obtained, that is, the influence of past learning behavior patterns and emotional management on language learning during the learning process, nodes with labeled attribute values corresponding to the historical language learning impact analysis data are obtained, and the nodes with labeled attribute values are sorted to obtain a logical sequence, and the target probability model is constructed based on the logical sequence using the historical language learning impact analysis data combined with dependent statements. Based on the target probability model, a Bayesian network is constructed using a functor and a chi-square verification method, a root node is constructed according to the target probability model, and the functor of the target probability model is used as the parent node. The chi-square verification method is used to verify the independence of the nodes, and nodes without child nodes are extracted as leaf nodes according to the independence test results, and a Bayesian network is constructed based on the root node, the parent node, and the leaf node. Based on the historical language learning impact analysis data, a qualitative analysis topological network is constructed using a factor relationship analysis method, the historical language learning impact analysis data is digitized to obtain the digitized historical language learning impact analysis data, a feature data matrix is constructed based on the digitized historical language learning impact analysis data, several learning impact factors are generated based on the feature data matrix, several learning impact factors are evaluated to obtain corresponding impact characteristic values, several learning impact factors are analyzed for correlation relationships to obtain factor correlation relationships, and a qualitative analysis topological network is constructed based on the factor correlation relationships and the corresponding impact characteristic values. Based on the Bayesian network and the qualitative analysis topological network, a qualitative analysis of the impact of language learning is performed using the learning behavior pattern analysis results and the feedback emotional information, that is, the learning behavior pattern analysis results and the feedback emotional information are respectively input into the Bayesian network and the qualitative analysis topological network for language learning impact analysis, and the analysis results of the two are combined to obtain the qualitative analysis results of the impact of language learning, and the qualitative analysis of the impact of language learning is to analyze the degree of influence of the user's learning behavior pattern and emotional changes on language learning.
[0074] S16: Perform a comprehensive evaluation of language learning based on the learning behavior pattern analysis results, the feedback emotional information and the qualitative analysis results of the language learning impact to obtain a comprehensive evaluation result.
[0075] In the specific implementation process of the present invention, the comprehensive evaluation of language learning is performed based on the learning behavior pattern analysis results, feedback emotional information and qualitative analysis results of language learning impact to obtain a comprehensive evaluation result, including: based on the learning behavior pattern analysis results, feedback emotional information and qualitative analysis results of language learning impact, a preset evaluation scheme is used to perform a comprehensive evaluation of language learning to obtain a comprehensive evaluation result, and a language learning evaluation report is generated based on the comprehensive evaluation result.
[0076] Specifically, based on the learning behavior pattern analysis results, feedback emotional information and qualitative analysis results of language learning impact, a preset evaluation plan is used to conduct a comprehensive evaluation of language learning. The preset evaluation plan includes a learning evaluation dimension for second language writing to obtain a comprehensive evaluation result. The user can generate a language learning evaluation report based on the comprehensive evaluation result, and can learn about his or her own changes in second language writing learning based on the language learning evaluation report.
[0077] In an embodiment of the present invention, the large language model performs several content logic analysis feedback, content coherence analysis feedback and grammatical analysis feedback on the preprocessed language writing text content data based on different input instructions, analyzes the language writing text content from multiple dimensions, enables users to understand their own problems in language learning and writing more comprehensively and accurately, and promotes the user's self-learning ability. Based on the preset coding table, the reflection log information and feedback record information are combined with matrix coding to perform learning behavior pattern analysis, which can more intuitively understand the user's behavior pattern in the process of receiving and applying language learning feedback. Based on the reflection log information, the dual-way attention mechanism is used for sentiment analysis, which can clearly understand the user's emotional changes in language learning, so that the feedback emotional information obtained is closer to the user's actual emotions. Based on the learning behavior pattern analysis results and feedback emotional information, a qualitative analysis of language learning impact is performed to improve the accuracy of language learning impact analysis and avoid the one-sidedness of language learning evaluation. The user's language learning quality can be effectively improved, and the language learning evaluation can achieve a more ideal effect.
[0078] Embodiment 2
[0079] See also Figure 2 , Figure 2 : is a schematic diagram of the structure of a language learning evaluation system based on large language model feedback in an embodiment of the present invention, the system comprising:
[0080] The writing content data preprocessing module 21 is used to obtain the language writing text content data, and preprocess the language writing text content data to obtain the preprocessed language writing text content data;
[0081] Writing content analysis and feedback module 22: used to obtain different input instructions from the user, and perform several writing content analysis and feedback on the preprocessed language writing text content data using a large language model based on the different input instructions to obtain several writing feedback data;
[0082] The learning behavior pattern analysis module 23 is used to obtain the reflection log information and feedback record information generated by a plurality of writing feedback data, and to perform learning behavior pattern analysis based on a preset coding table using the reflection log information and feedback record information in combination with matrix coding to obtain a learning behavior pattern analysis result;
[0083] Sentiment analysis module 24: used to perform sentiment analysis based on the reflection log information using a dual-path attention mechanism to obtain feedback sentiment information;
[0084] The language learning impact qualitative analysis module 25 is used to perform a language learning impact qualitative analysis based on the learning behavior pattern analysis results and the feedback emotional information to obtain a language learning impact qualitative analysis result;
[0085] The language learning comprehensive evaluation module 26 is used to conduct a comprehensive evaluation of language learning based on the learning behavior pattern analysis results, the feedback emotional information and the language learning impact qualitative analysis results to obtain a comprehensive evaluation result.
[0086] In the specific implementation process of the present invention, the specific implementation method of the system item can refer to the implementation method of the above-mentioned method item, which will not be repeated here.
[0087] In an embodiment of the present invention, the large language model performs several content logic analysis feedback, content coherence analysis feedback and grammatical analysis feedback on the preprocessed language writing text content data based on different input instructions, analyzes the language writing text content from multiple dimensions, enables users to understand their own problems in language learning and writing more comprehensively and accurately, and promotes the user's self-learning ability. Based on the preset coding table, the reflection log information and feedback record information are combined with matrix coding to perform learning behavior pattern analysis, which can more intuitively understand the user's behavior pattern in the process of receiving and applying language learning feedback. Based on the reflection log information, the dual-way attention mechanism is used for sentiment analysis, which can clearly understand the user's emotional changes in language learning, so that the feedback emotional information obtained is closer to the user's actual emotions. Based on the learning behavior pattern analysis results and feedback emotional information, a qualitative analysis of language learning impact is performed to improve the accuracy of language learning impact analysis and avoid the one-sidedness of language learning evaluation. The user's language learning quality can be effectively improved, and the language learning evaluation can achieve a more ideal effect.
[0088] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0089] In addition, the above is a detailed introduction to a language learning assessment method and system based on large language model feedback provided by an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A language learning evaluation method based on large language model feedback, characterized in that: The method comprises: Acquiring language writing text content data, and preprocessing the language writing text content data to obtain preprocessed language writing text content data; Obtain different input instructions from the user, and perform several writing content analysis and feedback on the preprocessed language writing text content data using a large language model based on the different input instructions to obtain several writing feedback data; Acquire reflection log information and feedback record information generated by a number of writing feedback data, and perform learning behavior pattern analysis based on a preset coding table using the reflection log information and feedback record information in combination with matrix coding to obtain a learning behavior pattern analysis result; Based on the reflection log information, sentiment analysis is performed using a dual-path attention mechanism to obtain feedback sentiment information; Performing a qualitative analysis of the impact of language learning based on the learning behavior pattern analysis results and the feedback emotional information to obtain a qualitative analysis result of the impact of language learning; A comprehensive evaluation of language learning is performed based on the learning behavior pattern analysis results, feedback emotional information and language learning impact qualitative analysis results to obtain a comprehensive evaluation result.
2. The language learning evaluation method based on large language model feedback according to claim 1, characterized in that: The preprocessing of the language writing text content data to obtain the preprocessed language writing text content data includes: Performing a garbled processing on the language writing text content data to obtain the language writing text content data after the garbled processing; The language writing text content data after the garbled characters are removed is formatted to obtain the pre-processed language writing text content data.
3. The language learning evaluation method based on large language model feedback according to claim 1, characterized in that: The method uses a large language model to perform several writing content analysis and feedback on the preprocessed language writing text content data based on different input instructions to obtain several writing feedback data, including: Inputting the preprocessed language writing text content data into a large language model, the large language model performs several content logic analysis feedbacks, content coherence analysis feedbacks and grammatical analysis feedbacks based on different input instructions, and obtains several content logic analysis feedback data, content coherence analysis feedback data and grammatical analysis feedback data, wherein the large language model includes a content logic analysis feedback module, a content coherence analysis feedback module and a grammatical analysis feedback module; The content logic analysis feedback module includes a topic score calculation submodule, a content relevance analysis submodule and a logic score calculation submodule; The content coherence analysis feedback module includes a sentence vector representation extraction submodule, a chapter overall vector representation extraction submodule and a coherence analysis submodule; The grammatical analysis feedback module includes a part-of-speech analysis submodule and a grammatical analysis submodule.
4. The language learning evaluation method based on large language model feedback according to claim 3, characterized in that: The topic score calculation submodule includes a high-level semantic information extractor, a spatial information adapter based on a convolutional neural network, and a linear classifier. The high-level semantic information extractor includes a double-tower model composed of several 7-layer transformers, and the linear classifier includes a binary tree and a support vector machine model based on a radial basis function.
5. The language learning evaluation method based on large language model feedback according to claim 1, characterized in that: The method of performing learning behavior pattern analysis based on the preset coding table using the reflection log information and feedback record information in combination with matrix coding to obtain the learning behavior pattern analysis result includes: Based on the preset coding table, node coding and marking are performed on the reflection log information and the feedback record information to obtain a plurality of coding categories; Calculate coding frequency distribution based on several coding categories; Based on several coding categories, matrix coding and preset project maps are used to obtain the correlation between coding categories; Based on the correlation between the coding frequency distribution and the coding categories, the behavior dimension is classified to obtain the behavior dimension classification result; Based on the behavioral dimension classification results, coding frequency distribution and the correlation between coding categories, a learning behavior pattern analysis is performed to obtain a learning behavior pattern analysis result.
6. The language learning evaluation method based on large language model feedback according to claim 5, characterized in that: The method of obtaining the association relationship between the coding categories based on the plurality of coding categories by using the matrix coding and the preset project map includes: Obtaining a node coding matrix of each coding category, and transforming the node coding matrix based on a polynomial clustering algorithm to obtain first hypergraph data; Convert the node encoding matrix based on a K-nearest neighbor algorithm to obtain second hypergraph data; Based on the fusion of the first hypergraph data and the second hypergraph data, a hypergraph association matrix is obtained; Based on the hypergraph association matrix combined with a preset project map, association mining of coding categories is performed to obtain association relationships between coding categories.
7. The language learning evaluation method based on large language model feedback according to claim 1, characterized in that: The sentiment analysis is performed based on the reflection log information using a dual-path attention mechanism to obtain feedback sentiment information, including: Acquire training log data, perform sentiment orientation annotation on the training log data, obtain sentiment orientation annotated training log data, and build a sentiment orientation classification model based on the sentiment orientation annotated training log data; Vectorizing the reflection log information based on a global vector to obtain a first text vector matrix; Vectorizing the reflection log information based on the word conversion vector to obtain a second text vector matrix; Based on the sentiment tendency classification model, sentiment tendency classification is performed using the first text vector matrix and the second text vector matrix to obtain sentiment tendency information; The target modifier is determined based on the preset dictionary using the reflection log information, the sentiment intensity is calculated based on the target modifier using the sentiment tendency information combined with a two-way attention mechanism, and feedback sentiment information is generated based on the sentiment tendency information and the sentiment intensity.
8. The language learning evaluation method based on large language model feedback according to claim 1, characterized in that: The performing of a qualitative analysis of the impact of language learning based on the learning behavior pattern analysis result and the feedback emotional information to obtain a qualitative analysis result of the impact of language learning includes: Acquire historical language learning impact analysis data, construct a target probability model based on the historical language learning impact analysis data using a logical sequence, and construct a Bayesian network based on the target probability model using a functor and a chi-square validation method; Based on the historical language learning impact analysis data, a qualitative analysis topological network is constructed using factor relationship analysis method; Based on the Bayesian network and the qualitative analysis topological network, the qualitative analysis of the impact of language learning is performed using the learning behavior pattern analysis results and the feedback emotional information to obtain the qualitative analysis results of the impact of language learning.
9. The language learning evaluation method based on large language model feedback according to claim 1, characterized in that: The comprehensive evaluation of language learning is performed based on the learning behavior pattern analysis results, the feedback emotional information and the qualitative analysis results of the language learning impact to obtain a comprehensive evaluation result, including: Based on the learning behavior pattern analysis results, feedback emotional information and language learning impact qualitative analysis results, a preset evaluation scheme is used to conduct a comprehensive evaluation of language learning, obtain a comprehensive evaluation result, and generate a language learning evaluation report based on the comprehensive evaluation result.
10. A language learning evaluation system based on large language model feedback, characterized in that: The system comprises: Writing content data preprocessing module: used to obtain language writing text content data, and preprocess the language writing text content data to obtain the preprocessed language writing text content data; Writing content analysis and feedback module: used to obtain different input instructions from users, and based on different input instructions, use a large language model to perform several writing content analysis and feedback on the preprocessed language writing text content data to obtain several writing feedback data; Learning behavior pattern analysis module: used to obtain reflection log information and feedback record information generated by a number of writing feedback data, and to perform learning behavior pattern analysis based on a preset coding table using the reflection log information and feedback record information combined with matrix coding to obtain a learning behavior pattern analysis result; Sentiment analysis module: used to perform sentiment analysis based on the reflection log information using a dual-path attention mechanism to obtain feedback sentiment information; A language learning impact qualitative analysis module: used to perform a language learning impact qualitative analysis based on the learning behavior pattern analysis results and the feedback emotional information, and obtain a language learning impact qualitative analysis result; Language learning comprehensive evaluation module: used to conduct a comprehensive evaluation of language learning based on the learning behavior pattern analysis results, feedback emotional information and language learning impact qualitative analysis results to obtain a comprehensive evaluation result.
Citation Information
Patent Citations
Big data analysis ecosystem for English ubiquitous learning mode
CN110109964A
Text scoring method, device and system
CN111104789A
Cross-scene-oriented English composition automatic scoring system and method
CN116306572A
Diagnostic report generation method, system and equipment
CN117574876A
Ideological and political course learning method based on education element universe and related device
CN118133837A
Cited By
Alzheimer's disease patient behavior monitoring method and system based on data analysis
CN120319474A
