An educational data analysis method based on semantic processing

Abstract

The application relates to the technical field of education data processing, and discloses an education data analysis method based on semantic processing, which comprises the following steps: obtaining education answering data and preprocessing, generating structured education text and student answering records; constructing synonym samples and negative sample fine-tuning RoBERTa to generate a semantic encoding model; obtaining question semantic representation and answer matching results based on semantic encoding; strictly comparing answer values to generate question answering performance values; obtaining potential knowledge dimensions according to initial clustering of question semantic representation; combining group answering vectors to correct the potential knowledge dimensions and generate explanation information; and calculating student mastery degrees and generating mastery degree snapshots according to question answering performance values. The application improves the accuracy of potential knowledge dimension division and the stability of new question mastery degree prediction.

Description

Technical Field

[0001] This invention relates to the field of educational data processing technology, and in particular to an educational data analysis method based on semantic processing. Background Technology

[0002] With the application of online teaching platforms, intelligent grading systems, and learning management systems, educational data such as question texts, reference answers, student answers, student scores, and answer times have accumulated in educational settings. Existing educational data analysis methods typically rely on manually labeled knowledge points or pre-defined knowledge systems to aggregate student scores under corresponding knowledge points in order to analyze students' mastery of those knowledge points. Some methods also utilize text similarity or semantic models to match student answers with reference answers.

[0003] Existing methods rely on manual pre-annotation of knowledge points in questions. When the question bank is large, the question sources are complex, or the knowledge point classification is inconsistent, the annotation cost is high and consistency is poor, making it difficult to stably quantify students' knowledge mastery. Simply clustering based on the semantics of the question text can easily group questions with similar wording but different testing abilities into the same category, and may also overlook questions with significant wording differences but consistent group performance, affecting the accuracy of potential knowledge dimension classification. When student answers contain negative expressions, numerical expressions, and synonyms, traditional semantic matching methods are prone to misjudgment, affecting the calculation results of question performance values ​​and student mastery.

[0004] Therefore, how to provide a semantic processing-based educational data analysis method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an educational data analysis method based on semantic processing. This invention fully utilizes RoBERTa semantic encoding, rigorous numerical comparison, and group response pattern correction, and describes in detail the process of analyzing students' knowledge mastery and predicting the mastery of new questions under the condition of no manual knowledge point annotation. It has the advantages of low annotation cost, high accuracy of dimensional division, and stable mastery calculation.

[0006] An educational data analysis method based on semantic processing according to an embodiment of the present invention includes the following steps: Acquire educational quiz data, perform preprocessing on the educational quiz data, and generate structured educational text and student answer records; Judgment-type statements are extracted from structured educational texts, and negative variation statements are generated based on negative word substitution. Judgment-type statements, negative variation statements, and synonym rewritten statements are input into RoBERTa to perform contrastive learning fine-tuning and generate a semantic encoding model. The structured educational text is input into the semantic encoding model to obtain the semantic representation of the questions. The student's answer and the reference answer are input into the semantic encoding model to obtain the semantic matching result of the answer. Numerical expressions are extracted from student answers and reference answers, the numerical expressions are standardized, the standardized numerical expressions are strictly compared, and the question answer performance value is generated based on the numerical comparison results and the semantic matching results of the answers. Based on the semantic representation of the question, perform initial clustering to generate an initial set of potential knowledge dimensions; Based on students' answer records, construct a group answer vector for questions. Perform secondary merging and splitting on the initial set of potential knowledge dimensions according to the group answer vector for questions to generate a revised set of potential knowledge dimensions. Generate dimension explanation information for the revised set of potential knowledge dimensions. The student mastery level is calculated based on the performance value of answering questions and the modified set of potential knowledge dimensions. Mastery snapshots containing dimension explanation information and student mastery levels are generated according to the analysis batch.

[0007] Optionally, the preprocessing of the educational answer data specifically includes: Perform field integrity checks and abnormal record cleaning on the educational quiz data, and delete records that are missing question identifiers, student identifiers, student answers, or student scores; Perform desensitization mapping on student identifiers to generate desensitized student identifiers; Based on the full score of the question, the student's score is normalized to generate a normalized score; The question text, reference answer, and question analysis are concatenated according to the field identifiers to generate structured educational text; The student's desensitized identifier, question identifier, student's answer, normalized score, and answer time are linked to generate a record of the student's answers.

[0008] Optionally, the comparative learning fine-tuning specifically includes: Sentence segmentation is performed on structured educational texts, and sentences containing judgment predicates are extracted as judgment statements; Replace the judgment predicates in the judgment statements with preset synonym predicates, or replace the words adjacent to the judgment predicates with preset synonyms, automatically generate synonym rewritten statements, and form positive sample pairs with the judgment statements and synonym rewritten statements; Replace the judgment predicate in the judgment statement with a preset negative predicate, or insert a preset negative word before the judgment predicate, automatically generate a negative change statement, and form a negative sample pair with the judgment statement and the negative change statement; Input positive and negative sample pairs into RoBERTa to obtain the sentence vector representation of each sentence; A contrastive learning loss is constructed based on the increasing direction of sentence vector similarity for positive sample pairs and the decreasing direction of sentence vector similarity for negative sample pairs. Based on the contrastive learning loss, the parameters of RoBERTa are fine-tuned to obtain the semantic encoding model.

[0009] Optionally, inputting student answers and reference answers into the semantic coding model specifically includes: Based on the question identifiers in the students' answer records, the corresponding reference answers are extracted from the structured educational text, and the students' answers and the corresponding reference answers are constructed into answer matching text pairs; By inputting the student's answer, the corresponding reference answer, and the answer-matching text pair into the semantic encoding model, we obtain the independent semantic representation of the student's answer, the independent semantic representation of the reference answer, and the joint semantic representation of the answer pair. The overall semantic proximity result is calculated based on the independent semantic representations of the student's answer and the independent semantic representation of the reference answer. The semantic deviation representation of the student's answer relative to the reference answer is extracted based on the joint semantic representation of the answer pair. The overall semantic proximity result and semantic deviation representation are fused to generate the semantic matching result of the answer.

[0010] Optionally, generating the question answer performance value based on the numerical comparison result and the semantic matching result of the answer specifically includes: Perform numerical recognition on student answers and reference answers separately, and extract the numerical sets of student answers and reference answers; Perform format conversion, unit conversion, and precision unification on the numerical expressions in the student answer numerical set and the reference answer numerical set to obtain a standardized student answer numerical set and a standardized reference answer numerical set. Establish a correspondence between the standardized student answer value set and the standardized reference answer value set according to the order of numerical appearance, perform a strict consistency comparison on the corresponding values, and generate numerical comparison results; When no numerical expression is extracted from the reference answer, the semantic matching result of the answer will be used as the performance value of the question answer; When numerical expressions are extracted from the reference answer, the question answer performance value is generated based on the numerical comparison results, the answer semantic matching results, and the normalized score in the student's answer record; the question answer performance value is jointly determined by the normalized score, the answer semantic matching results, and the numerical comparison results.

[0011] Optionally, the initial clustering based on the semantic representation of the question specifically includes: For each question, the semantic representation is vectorized and the semantic distance between any two questions is calculated. Construct a question adjacency graph based on semantic distance, treat questions as nodes, and connect question nodes whose semantic distance meets the preset adjacency conditions as edges; Density-based clustering is performed on the adjacency graph of the questions, and the question nodes whose connectivity density meets the preset density condition are divided into the same initial potential knowledge dimension. Mark question nodes that do not meet the preset density conditions as question nodes to be merged; For each initial potential knowledge dimension, an initial dimension identifier is generated, and the correspondence between the initial dimension identifier and the question identifier is recorded to obtain the initial potential knowledge dimension set.

[0012] Optionally, the step of performing a secondary merging and splitting of the initial latent knowledge dimension set based on the question group's answer vector specifically includes: According to the unified sorting of the desensitized student identifiers, the normalized score corresponding to each question is filled into the vector position in sequence to generate the question group answer vector, and the unanswered positions are filled with missing markers. For any two questions, select the vector positions corresponding to students who gave the same answer and calculate the similarity of their answer patterns. Within the same initial potential knowledge dimension, questions whose semantic distance satisfies the preset adjacency condition and whose answer pattern similarity satisfies the preset retention condition are retained in the same dimension, while questions whose semantic distance satisfies the preset adjacency condition but whose answer pattern similarity does not satisfy the preset retention condition are split into questions to be reorganized. Among different initial potential knowledge dimensions, questions whose semantic distance does not meet the preset adjacency condition but whose answer pattern similarity meets the preset merging condition are added to the candidate merging list; The candidate merge list and the questions to be reorganized are re-clustered based on the similarity of the answering patterns to generate a set of corrected potential knowledge dimensions.

[0013] Optionally, generating dimensional explanation information for correcting the potential knowledge dimension set specifically includes: For each potential knowledge dimension to be corrected, the corresponding question identifier set is read, and the text content of the corresponding question is extracted from the structured educational text; The text content is segmented and stop word filtered. The frequency and distinguishability of words in the current corrected potential knowledge dimension are calculated, and words that meet the preset sorting conditions are selected as dimension keywords. Calculate the semantic similarity between the semantic representation of each question and the center vector of the current modified potential knowledge dimension, and select the question with the highest semantic similarity as the representative question; Pseudo-tags are generated based on dimension keywords and representative titles, and the dimension keywords, representative titles, and pseudo-tags are associated with dimension explanation information.

[0014] Optionally, the calculation of student mastery based on question-answering performance and a modified set of potential knowledge dimensions specifically includes: Read the correction dimension identifiers and corresponding question identifier sets from the correction potential knowledge dimension set; Based on the set of question identifiers, extract the question performance values ​​of the same student on the corresponding question from the student's answer records; The average performance of the same student in answering questions within the same modified latent knowledge dimension is calculated to obtain the student's mastery level in the corresponding modified latent knowledge dimension. When a student does not have a performance value for answering a question within the corresponding modified potential knowledge dimension, the student's mastery level in the corresponding modified potential knowledge dimension is marked as null.

[0015] Optionally, generating a mastery snapshot that includes dimensional explanation information and student mastery levels according to the analysis batch specifically includes: According to the analysis batch, the corrected dimension identifier, dimension explanation information, question identifier set, corrected potential knowledge dimension center vector and student mastery are stored together to generate a mastery snapshot; Obtain the new question text, perform field concatenation on the new question text and input it into the semantic encoding model to obtain the semantic representation of the new question; Calculate the semantic similarity between the semantic representation of the new question and the center vector of each modified latent knowledge dimension in the mastery snapshot, and determine the modified latent knowledge dimension with the highest semantic similarity and that meets the preset matching conditions as the dimension to which the new question belongs; Read the de-identified student identifiers of the target students and retrieve the students' mastery of the target students in the dimension to which the new topic belongs from the mastery snapshot; The retrieved student mastery level is used as the target student's predicted mastery level of the new question. The output includes the dimension to which the new question belongs, the corresponding dimension explanation information, and the predicted mastery level.

[0016] The beneficial effects of this invention are: (1) This invention generates potential knowledge dimensions by performing RoBERTa semantic encoding on the question text and adopting a clustering method that does not require pre-setting the number of clusters. This reduces the reliance on manual knowledge point annotation and pre-set knowledge system. It can automatically form knowledge dimensions that can be used for mastery analysis when the question bank is large and the question sources are complex, thereby reducing the cost of manual annotation and improving the applicability of educational data analysis.

[0017] (2) This invention performs comparative learning and fine-tuning of RoBERTa by automatically constructing negative change statements and synonym rewriting statements, and combines the numerical expressions in the student's answer with the reference answer for strict consistency comparison, so that the semantic matching process of the answer can distinguish between synonymous expressions, negative expressions and numerical errors, reduce misjudgments caused by semantically similar but opposite meanings, and textual similar but numerical errors, and improve the reliability of the answer performance value.

[0018] (3) This invention constructs a group answer vector for questions, performs secondary merging and splitting on the initial potential knowledge dimension, so that questions with similar semantics but different group answer performances can be split, and questions with large textual differences but similar group answer performances can be merged; at the same time, it stores dimension explanation information and student mastery through mastery snapshots, and performs dimension matching and mastery prediction for new questions, thereby improving the stability of student knowledge mastery analysis and subsequent question prediction. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an educational data analysis method based on semantic processing proposed in this invention; Figure 2 This is a schematic diagram illustrating the potential knowledge dimension correction of an educational data analysis method based on semantic processing proposed in this invention; Figure 3 This is a schematic diagram illustrating the mastery snapshot and prediction of an educational data analysis method based on semantic processing proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figures 1-3 A semantic processing-based method for educational data analysis includes the following steps: Acquire educational quiz data, perform preprocessing on the educational quiz data, and generate structured educational text and student answer records; Judgment-type statements are extracted from structured educational texts, and negative variation statements are generated based on negative word substitution. Judgment-type statements, negative variation statements, and synonym rewritten statements are input into RoBERTa to perform contrastive learning fine-tuning and generate a semantic encoding model. The structured educational text is input into the semantic encoding model to obtain the semantic representation of the questions. The student's answer and the reference answer are input into the semantic encoding model to obtain the semantic matching result of the answer. Numerical expressions are extracted from student answers and reference answers, the numerical expressions are standardized, the standardized numerical expressions are strictly compared, and the question answer performance value is generated based on the numerical comparison results and the semantic matching results of the answers. Based on the semantic representation of the question, perform initial clustering to generate an initial set of potential knowledge dimensions; Based on students' answer records, construct a group answer vector for questions. Perform secondary merging and splitting on the initial set of potential knowledge dimensions according to the group answer vector for questions to generate a revised set of potential knowledge dimensions. Generate dimension explanation information for the revised set of potential knowledge dimensions. The student mastery level is calculated based on the performance value of answering questions and the modified set of potential knowledge dimensions. Mastery snapshots containing dimension explanation information and student mastery levels are generated according to the analysis batch.

[0022] In this embodiment, the preprocessing of educational answer data specifically includes: The system performs field integrity checks and anomaly cleaning on educational quiz data, deleting records that are missing question identifiers, student identifiers, student answers, or student scores. Field integrity checks involve checking the existence of fields in each piece of educational quiz data according to a pre-defined field table, which includes question identifiers, student identifiers, student answers, and student scores. Anomaly cleaning involves deleting records where fields are empty, student scores are less than 0, student scores are greater than the maximum score for a question, or the same student has submitted the same question multiple times without being able to determine the final submission result. The student identifiers are anonymized and mapped to generate anonymized student identifiers. Anonymization mapping refers to generating a unique anonymized number for each student identifier and establishing a one-to-one correspondence between student identifiers and anonymized student identifiers. The anonymized student identifiers are represented by fixed-length strings, and only the anonymized student identifiers are used in subsequent calculations. The student scores are normalized based on the full score of the question to generate a normalized score. Score normalization means dividing the student score by the full score of the corresponding question to obtain a normalized score with a value range of 0 to 1. The question text, reference answer, and question analysis are concatenated according to field identifiers to generate structured educational text. Field identifiers are fixed marks used to distinguish different text sources. The structured educational text is generated by concatenating the question field identifier, question text, reference answer field identifier, reference answer, question analysis field identifier, and question analysis in that order. When the reference answer or question analysis is empty, the corresponding field identifier is retained and an empty value is filled in. The student's anonymized identifier, question identifier, student's answer, normalized score, and answer time are linked to generate a student answer record. The student answer record uses the anonymized student identifier and question identifier as a joint index, and writes the student's answer, normalized score, and answer time into the same record. When the same joint index corresponds to multiple records, the record with the latest answer time is selected as the valid student answer record.

[0023] In this embodiment, comparative learning fine-tuning specifically includes: The structured educational text is segmented into sentences, and sentences containing judgment predicates are extracted as judgment statements. The segmentation process divides the structured educational text according to periods, question marks, semicolons, line breaks, and title field identifiers. The judgment predicates are determined by a preset judgment predicate list, which contains words such as "is", "belongs to", "establishes", "correct", and "exists". When a segmented sentence contains at least one judgment predicate from the preset judgment predicate list, the corresponding sentence is identified as a judgment statement. The process involves replacing the predicates in judgment statements with preset synonyms, or replacing adjacent words of the predicates with preset synonyms, to automatically generate synonym rewritten statements. These judgment statements and their synonym rewritten statements are then paired as positive samples. The synonym rewritten statements are generated using semantically preservative masking. Predicates, judgment objects, or judgment complements in judgment statements are used as candidate rewritten fragments. These fragments are replaced with the [MASK] marker. The replaced sentences are then input into RoBERTa to obtain candidate word prediction results. Candidate words with the same part of speech as the candidate rewritten fragments and predicted probabilities ranking in the top 20 are selected from the prediction results and filled back into the [MASK] positions to form candidate rewritten sentences. Semantic similarity is calculated between the candidate rewritten sentences and the judgment statements. Candidate rewritten sentences with a semantic similarity of at least 0.85 and without negative words are retained as synonym rewritten statements. The judgment predicates in the judgment statements are replaced with preset negative predicates, or preset negative words are inserted before the judgment predicates to automatically generate negative change statements. The judgment statements and negative change statements are then paired to form negative sample pairs. Preset negative words include, for example, "not", "non-", "none", "not", "no", and "cannot". When generating negative change statements, if the judgment statement contains, for example, "is", "belongs to", "is valid", "exists", or "can", the corresponding judgment predicates are replaced with "is not", "does not belong to", "is not valid", "does not exist", or "cannot". If the judgment statement contains numerical comparison predicates "greater than", "less than", or "equal to", they are replaced with "not greater than", "not less than", or "not equal to" respectively. Input positive and negative sample pairs into RoBERTa to obtain the sentence vector representation of each sentence; the sentence vector representation is generated by the [CLS] position vector in the last hidden state of RoBERTa. A contrastive learning loss is constructed based on the increasing direction of sentence vector similarity for positive sample pairs and the decreasing direction of sentence vector similarity for negative sample pairs. Based on the contrastive learning loss, RoBERTa parameters are fine-tuned to obtain a semantic encoding model. Sentence vector similarity is calculated using cosine similarity. The contrastive learning loss uses the InfoNCE loss function with a temperature parameter of 0.1. After fine-tuning, the RoBERTa parameters are saved, and the saved RoBERTa is used as the semantic encoding model.

[0024] In this embodiment, inputting student answers and reference answers into the semantic coding model specifically includes: Based on the question identifiers in the student's answer record, the corresponding reference answers are extracted from the structured educational text, and the student's answer and the corresponding reference answer are constructed into an answer matching text pair. The answer matching text pair is constructed according to a fixed field order, with the format "[reference answer]corresponding reference answer[student answer]student answer". The corresponding reference answer is retrieved from the structured educational text by the question identifier. When the same question identifier corresponds to multiple reference answers, the multiple reference answers are concatenated according to a preset separator to form the corresponding reference answer. The student's answer, the corresponding reference answer, and the answer-matching text pair are input into the semantic encoding model to obtain independent semantic representations of the student's answer, independent semantic representations of the reference answer, and joint semantic representations of the answer pair. The independent semantic representations of the student's answer and the reference answer are obtained by encoding a single sentence input using the semantic encoding model, while the joint semantic representation of the answer pair is obtained by encoding the answer-matching text pair using the semantic encoding model. The semantic representation uses the [CLS] position vector in the last hidden state of the semantic encoding model. The overall semantic similarity result is calculated based on the independent semantic representations of the student's answer and the independent semantic representation of the reference answer. The semantic deviation representation of the student's answer relative to the reference answer is extracted based on the joint semantic representation of the answer pair. The overall semantic similarity result is obtained by using the cosine similarity between the independent semantic representations of the student's answer and the independent semantic representation of the reference answer. The semantic deviation representation is obtained by linear transformation of the joint semantic representation of the answer pair. The input of the linear transformation is the joint semantic representation of the answer pair, and the output is a one-dimensional deviation value. The parameters of the linear transformation are updated synchronously when RoBERTa performs contrastive learning fine-tuning. The overall semantic proximity result and the semantic deviation representation are fused to generate the answer semantic matching result; the answer semantic matching result is obtained by subtracting the semantic deviation representation from the overall semantic proximity result; when the semantic deviation representation is less than zero, the semantic deviation representation is set to zero before being included in the calculation; the generated answer semantic matching result is associated with and saved with the question identifier in the student's answer record.

[0025] In this embodiment, generating the question answer performance value based on the numerical comparison result and the semantic matching result of the answer specifically includes: Numerical recognition is performed on both student answers and reference answers to extract numerical sets for student answers and reference answers, respectively. Numerical recognition refers to extracting integers, decimals, fractions, percentages, ratios, scientific notation values, values ​​with units, and interval values ​​from student answers and reference answers through regular expression matching and unit word adjacency recognition. The student answer numerical sets are arranged in the order in which the values ​​appear in the student answers, and the reference answer numerical sets are arranged in the order in which the values ​​appear in the reference answers. The numerical expressions in the student answer set and the reference answer set are processed through format conversion, unit conversion, and precision standardization to obtain a standardized student answer set and a standardized reference answer set. Format conversion includes converting percentages to decimals, fractions to decimals, and scientific notation to decimals. Unit conversion is performed according to a preset unit conversion table, which records the conversion relationships between different units under the same dimension. Precision standardization refers to rounding the student answer values ​​according to the number of decimal places in the reference answer values. Establish a correspondence between the standardized student answer set and the standardized reference answer set according to the order of their appearance. Perform a strict consistency comparison on the corresponding values ​​and generate the numerical comparison results. When the number of elements in the standardized student answer set and the standardized reference answer set are the same, establish a correspondence one by one according to the element index. When the number of elements in the two sets are different, the positions where corresponding elements are missing are marked as numerical inconsistencies. Strict consistency comparison means that the two standardized values ​​in the correspondence are the same in terms of numerical magnitude and unit dimension. The numerical comparison results record the comparison results of each correspondence and the summary results of all correspondences. When no numerical expression is extracted from the reference answer, the semantic matching result of the answer will be used as the performance value of the question answer; When numerical expressions are extracted from the reference answer, a question performance value is generated based on the numerical comparison results, the semantic matching results of the answer, and the normalized score in the student's answer record. The question performance value is jointly determined by the normalized score, the semantic matching results of the answer, and the numerical comparison results. When all numerical comparison results are consistent, the semantic matching results of the answer are retained for calculation. When there are inconsistencies in the numerical comparison results, the semantic matching results of the answer are deducted according to the number of numerical inconsistencies before being used in the calculation to generate the question performance value. The deduction coefficient is set separately according to different subjects.

[0026] In this embodiment, performing initial clustering based on the semantic representation of the question specifically includes: For each question's semantic representation, vector normalization is performed, and the semantic distance between any two questions is calculated. Vector normalization means converting the question's semantic representation into a unit vector. The semantic distance is calculated using cosine distance. Construct a question adjacency graph based on semantic distance, treat questions as nodes, and connect question nodes that meet the preset adjacency conditions as edges; the preset adjacency condition is that the semantic distance is not greater than a distance threshold, which is 0.35; the edges in the question adjacency graph are undirected edges, and the weight of the edge is determined by the semantic distance between the two corresponding questions, and the weight of the edge is denoted as semantic distance. Density-based clustering is performed on the adjacency graph of the questions, and question nodes whose connectivity density meets the preset density conditions are divided into the same initial potential knowledge dimension. The density-based clustering uses the HDBSCAN algorithm. The preset density conditions include the minimum number of cluster samples and the minimum number of core samples. The minimum number of cluster samples is 3 and the minimum number of core samples is 2. The connectivity density is determined by the number of neighboring nodes of a question node within the distance threshold range. Question nodes that do not meet the preset density conditions are marked as question nodes to be merged; question nodes to be merged refer to question nodes that are marked as noise in the output of the HDBSCAN algorithm. Question nodes to be merged retain question identifiers, question semantic representations, and student answer records. For each initial potential knowledge dimension, an initial dimension identifier is generated, and the correspondence between the initial dimension identifier and the question identifier is recorded to obtain the initial potential knowledge dimension set. The initial dimension identifier is generated according to the analysis batch number and cluster sequence number. The initial potential knowledge dimension set is indexed by the initial dimension identifier and records the corresponding question identifier set, question semantic representation set, and question node labeling results to be merged.

[0027] In this implementation, the secondary merging and splitting of the initial potential knowledge dimension set based on the collective answer vectors of the questions specifically includes: Following the unified sorting of anonymized student identifiers, the normalized score corresponding to each question is sequentially filled into the vector position to generate the question group's answer vector. Unanswered positions are filled with missing markers. The unified sorting of anonymized student identifiers is determined according to the lexicographical order of the anonymized student identifiers. The dimension of the question group's answer vector is equal to the number of students participating in the current analysis batch. The normalized score is filled into the vector position corresponding to the anonymized student identifier. Missing markers are marked with null values ​​and are not included in subsequent answer pattern similarity calculations. For any two questions, the similarity of answer patterns is calculated by selecting the vector positions corresponding to students who answered the same questions. Students who answered the same questions are desensitized students whose answer vectors do not contain missing markers in either question. When the number of students who answered the same questions is less than 3, the similarity of answer patterns between the two questions is not calculated. When the number of students who answered the same questions is not less than 3, the normalized scores corresponding to the students who answered the same questions are extracted to form two vectors of equal length, and the similarity of answer patterns is calculated using the Pearson correlation coefficient. Within the same initial potential knowledge dimension, questions whose semantic distance meets the preset adjacency condition and whose answer pattern similarity meets the preset retention condition are retained in the same dimension. Questions whose semantic distance meets the preset adjacency condition but whose answer pattern similarity does not meet the preset retention condition are split into questions to be reorganized. The preset retention condition is that the answer pattern similarity is not lower than the first similarity threshold, which is 0.45. For questions within the same initial potential knowledge dimension, the average answer pattern similarity between each question and other questions within the same initial potential knowledge dimension is calculated. When the average answer pattern similarity is lower than the first similarity threshold, the corresponding question is marked as a question to be reorganized. Among different initial potential knowledge dimensions, questions whose semantic distance does not meet the preset adjacency condition but whose answer pattern similarity meets the preset merging condition are added to the candidate merging list; the preset merging condition is that the answer pattern similarity is not lower than the second similarity threshold, which is 0.7; the candidate merging list records the question identifier pair, the corresponding initial dimension identifier, the semantic distance, and the answer pattern similarity; when there are multiple candidate merging objects for the same question, the candidate merging object with the highest answer pattern similarity is retained; The candidate merging list and the questions to be reorganized are re-clustered based on the similarity of their answering patterns to generate a revised set of potential knowledge dimensions. Re-clustering refers to constructing a response pattern relationship graph using the questions to be reorganized and the questions in the candidate merging list as nodes, and the relationships between question nodes that meet the preset merging conditions as edges. The connected subgraphs in the response pattern relationship graph are used as the revised potential knowledge dimensions, and the questions to be reorganized that do not form a connected relationship are retained as single-question potential knowledge dimensions. The revised potential knowledge dimension set records the revised dimension identifier, the question identifier set, and the source initial dimension identifier.

[0028] In this embodiment, the specific steps for generating dimension explanation information to correct the potential knowledge dimension set include: For each potential knowledge dimension to be corrected, the corresponding question identifier set is read, and the text content of the corresponding question is extracted from the structured educational text. The text content of the corresponding question refers to the question text, reference answer, and question analysis corresponding to the question identifier in the structured educational text. When the question analysis is empty, only the question text and reference answer are extracted. All the question text content contained in the question identifier set together constitute the dimension text set of the current potential knowledge dimension to be corrected. The text content is segmented and stopped word filtered. The frequency and discriminative power of words within the current revised potential knowledge dimension are calculated, and words that meet the preset sorting criteria are selected as dimension keywords. The frequency of occurrence refers to the number of times a word appears in the dimension text set of the current revised potential knowledge dimension. The discriminative power is represented by the TF-IDF value, which is calculated from the word frequency in the current revised potential knowledge dimension and the inverse document frequency of the word in all revised potential knowledge dimensions. The preset sorting criteria are to sort the words from high to low according to their TF-IDF values ​​and select the top 3 to 8 words. Calculate the semantic similarity between the semantic representation of each question and the center vector of the current modified latent knowledge dimension, and select the question with the highest semantic similarity as the representative question; the center vector of the current modified latent knowledge dimension is obtained by averaging the semantic representations of all questions within the current modified latent knowledge dimension; the semantic similarity is calculated using cosine similarity; the number of representative questions is 1 to 3, arranged in descending order of semantic similarity; Pseudo-tags are generated based on dimension keywords and representative titles, and the dimension keywords, representative titles, and pseudo-tags are associated with dimension explanation information. Pseudo-tags are generated by combining at least the two highest-ranking keywords in the dimension keywords. Dimension explanation information includes the corrected dimension identifier, dimension keywords, representative title identifier, and pseudo-tags.

[0029] In this implementation, calculating student mastery based on question-answering performance and a revised set of potential knowledge dimensions specifically includes: Read the correction dimension identifiers and corresponding question identifier sets from the correction potential knowledge dimension set; Based on the set of question identifiers, extract the question answer performance value of the same student on the corresponding question from the student's answer record; when the same student has multiple question answer performance values ​​under the same question identifier, select the question answer performance value with the latest answer time to participate in the calculation; The average performance of the same student in answering questions within the same modified latent knowledge dimension is calculated to obtain the student's mastery level in the corresponding modified latent knowledge dimension. When a student does not have a performance value for answering a question within the corresponding modified potential knowledge dimension, the student's mastery level in the corresponding modified potential knowledge dimension is marked as null.

[0030] In this embodiment, generating a mastery snapshot containing dimensional explanation information and student mastery level according to the analysis batch specifically includes: According to the analysis batch, the corrected dimension identifier, dimension explanation information, question identifier set, corrected potential knowledge dimension center vector and student mastery are stored together to generate a mastery snapshot; The process involves obtaining the text of a new question, performing field-based concatenation on the text, and inputting it into a semantic encoding model to obtain a semantic representation of the new question. The new question text includes at least the question stem. If a reference answer and question analysis exist for the new question, the question stem, reference answer, and question analysis are concatenated according to field identifiers to generate a structured text of the new question. The semantic representation of the new question is obtained by encoding the structured text of the new question using the semantic encoding model. Calculate the semantic similarity between the semantic representation of the new question and the center vector of each modified latent knowledge dimension in the mastery snapshot. The modified latent knowledge dimension with the highest semantic similarity and that meets the preset matching conditions is determined as the dimension to which the new question belongs. The semantic similarity is calculated using cosine similarity. The preset matching condition is that the semantic similarity is not lower than the matching threshold, and the matching threshold is 0.65. When there is no modified latent knowledge dimension that meets the preset matching conditions, the new question is marked as an unmatched question, and the predicted mastery of the target student on the new question is not output for the time being. Read the de-identified student identifiers of the target students and retrieve the students' mastery of the target students in the dimension of the new topic from the mastery snapshot; when the students' mastery of the target students in the dimension of the new topic is null, read the average of the mastery of all non-null students in the same analysis batch of the target students as the supplementary predicted value. The retrieved student mastery level is used as the target student's predicted mastery level of the new question. The output includes the dimension to which the new question belongs, the corresponding dimension explanation information, and the predicted mastery level. The value range of the predicted mastery level is consistent with the student mastery level. The output content includes the new question identifier, the corrected dimension identifier to which the new question belongs, dimension keywords, representative question identifier, pseudo-label, the de-identified student identifier of the target student, and the predicted mastery level.

[0031] Example 1: To verify the feasibility of this invention in practice, it was applied to an online teaching question-answering analysis scenario. In this scenario, learners complete exercises and tests online, and the platform stores data such as question text, reference answers, question explanations, student answers, student scores, and answer times. Due to the large number of question sources, some questions lack pre-annotated knowledge points, some questions, while similar in wording, test different abilities, and some questions with significant word differences are frequently answered correctly or incorrectly by the same group of students simultaneously. If manual knowledge point annotation is still used, it not only requires a large amount of manual intervention, but also leads to inconsistencies in the understanding of knowledge point boundaries among different personnel, resulting in unstable analysis results of student knowledge mastery.

[0032] In this scenario, educational quiz data is first acquired. Fields such as question identifier, student identifier, student answer, and student score are cleaned, and student identifiers are anonymized. Student scores are then normalized to the maximum possible score for each question. The question text, reference answer, and question explanation are concatenated into a structured educational text, while student answers, normalized scores, and answer times are organized into student response records. This process unifies the previously scattered question and response data into a single analysis workflow, providing a foundation for subsequent semantic encoding and mastery calculations.

[0033] The model extracts judgment-type statements from structured educational texts, automatically generating paraphrased and negated statements. These original sentences, paraphrased statements, and negated statements are then input into RoBERTa for comparative learning and fine-tuning. This processing enables the semantic encoding model to make synonymous expressions closer together, differentiating expressions with negative connotations such as "not," "not," "does not belong to," and "is not true" from the original sentences, reducing misjudgments based on similar wording but opposite meanings. For student answers and reference answers, the model outputs semantic matching results. Simultaneously, it extracts numerical expressions from both, standardizing percentages, fractions, decimals, and values ​​with units, and performing a rigorous consistency comparison. When the reference answer involves numerical values, numerical errors directly affect the student's performance, preventing the model from judging a student's answer as close to correct solely based on word similarity.

[0034] In the question analysis phase, structured educational text is input into a semantic encoding model to obtain semantic representations of the questions. Initial latent knowledge dimensions are generated using density clustering, which does not require pre-setting the number of clusters. Then, based on student answer records, a group answer vector is constructed for each question, comparing whether different questions are frequently answered correctly or incorrectly by the same group of students simultaneously. If two questions are semantically similar and their group answer performance is similar, they are retained in the same latent knowledge dimension; if they are semantically similar but their group answer performance differs significantly, they are split into different dimensions; if the textual differences are significant but the group answer performance is highly similar, they enter the candidate merging range and participate in secondary merging. This process yields a revised set of latent knowledge dimensions, generating keywords, representative questions, and pseudo-labels for each dimension, making the analysis results not only computationally achievable but also easily understandable.

[0035] In the mastery calculation phase, based on each student's performance on each question within the modified latent knowledge dimension, the student's mastery level in the corresponding dimension is calculated, and a mastery snapshot is generated according to the analysis batch. The mastery snapshot stores the modified dimension identifier, dimension explanation information, question identifier set, dimension center vector, and student mastery level. When a new question appears, the new question text is input into the semantic encoding model to obtain the semantic representation of the new question. Semantic similarity is calculated between this representation and the dimension center vectors in the mastery snapshot to determine the dimension to which the new question belongs. The target student's mastery level in that dimension is then read and used as the student's predicted mastery level for the new question.

[0036] Table 1: Comparison of Educational Question-Answer Analysis Results

[0037] As shown in Table 1, this invention outperforms both manual knowledge point annotation methods and ordinary semantic clustering methods in multiple metrics. While manual knowledge point annotation methods can directly utilize a pre-defined knowledge system, they require continuous manual maintenance when dealing with a large number of questions from diverse sources, resulting in a high workload. Furthermore, different annotators may assign different knowledge points to the same question, thus affecting the accuracy of knowledge dimension classification and the stability of mastery calculation. Ordinary semantic clustering methods reduce human intervention but primarily rely on the semantic similarity between question texts, easily grouping questions with similar wording but significantly different answer performances into the same category. Therefore, its improvement in the accuracy of new question dimension matching and the stability of mastery calculation is limited.

[0038] This invention obtains the semantic representation of questions through RoBERTa semantic encoding. It then performs secondary merging and splitting of the initial clustering results using group response vectors, ensuring that the latent knowledge dimensions consider both the textual meaning of the questions and the students' actual response performance. Therefore, the accuracy rate of knowledge dimension segmentation reaches 91.7%. This invention constructs training samples for synonymous and negative expressions and performs standardization and rigorous consistency comparison on numerical expressions, reducing the answer matching misjudgment rate to 6.4% and increasing the accuracy rate of numerical answer recognition to 95.3%. The mastery snapshot can save dimension explanation information and student mastery levels. New questions only need to be matched with the corresponding latent knowledge dimensions to obtain the predicted mastery level of the target students, thereby improving the usability and stability of new question analysis.

[0039] Overall, this invention can improve the accuracy of student knowledge mastery analysis and new question mastery prediction while reducing the workload of manual annotation.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for analyzing educational data based on semantic processing, characterized in that, Includes the following steps: Acquire educational quiz data, perform preprocessing on the educational quiz data, and generate structured educational text and student answer records; Judgment-type statements are extracted from structured educational texts, and negative variation statements are generated based on negative word substitution. Judgment-type statements, negative variation statements, and synonym rewritten statements are input into RoBERTa to perform contrastive learning fine-tuning and generate a semantic encoding model. The structured educational text is input into the semantic encoding model to obtain the semantic representation of the questions. The student's answer and the reference answer are input into the semantic encoding model to obtain the semantic matching result of the answer. Numerical expressions are extracted from student answers and reference answers, the numerical expressions are standardized, the standardized numerical expressions are strictly compared, and the question answer performance value is generated based on the numerical comparison results and the semantic matching results of the answers. Based on the semantic representation of the question, perform initial clustering to generate an initial set of potential knowledge dimensions; Based on students' answer records, construct a group answer vector for questions. Perform secondary merging and splitting on the initial set of potential knowledge dimensions according to the group answer vector for questions to generate a revised set of potential knowledge dimensions. Generate dimension explanation information for the revised set of potential knowledge dimensions. The student mastery level is calculated based on the performance value of answering questions and the modified set of potential knowledge dimensions. Mastery snapshots containing dimension explanation information and student mastery levels are generated according to the analysis batch.

2. The method for analyzing educational data based on semantic processing according to claim 1, characterized in that, The preprocessing of educational answer data specifically includes: Perform field integrity checks and abnormal record cleaning on the educational quiz data, and delete records that are missing question identifiers, student identifiers, student answers, or student scores; Perform desensitization mapping on student identifiers to generate desensitized student identifiers; Based on the full score of the question, the student's score is normalized to generate a normalized score; The question text, reference answer, and question analysis are concatenated according to the field identifiers to generate structured educational text; The student's desensitized identifier, question identifier, student's answer, normalized score, and answer time are linked to generate a record of the student's answers.

3. The method for analyzing educational data based on semantic processing according to claim 2, characterized in that, The comparative learning fine-tuning specifically includes: Sentence segmentation is performed on structured educational texts, and sentences containing judgment predicates are extracted as judgment statements; Replace the judgment predicates in the judgment statements with preset synonym predicates, or replace the words adjacent to the judgment predicates with preset synonyms, automatically generate synonym rewritten statements, and form positive sample pairs with the judgment statements and synonym rewritten statements; Replace the judgment predicate in the judgment statement with a preset negative predicate, or insert a preset negative word before the judgment predicate, automatically generate a negative change statement, and form a negative sample pair with the judgment statement and the negative change statement; Input positive and negative sample pairs into RoBERTa to obtain the sentence vector representation of each sentence; A contrastive learning loss is constructed based on the increasing direction of sentence vector similarity for positive sample pairs and the decreasing direction of sentence vector similarity for negative sample pairs. Based on the contrastive learning loss, the parameters of RoBERTa are fine-tuned to obtain the semantic encoding model.

4. The method for analyzing educational data based on semantic processing according to claim 3, characterized in that, The specific steps of inputting student answers and reference answers into the semantic coding model include: Based on the question identifiers in the students' answer records, the corresponding reference answers are extracted from the structured educational text, and the students' answers and the corresponding reference answers are constructed into answer matching text pairs; By inputting the student's answer, the corresponding reference answer, and the answer-matching text pair into the semantic encoding model, we obtain the independent semantic representation of the student's answer, the independent semantic representation of the reference answer, and the joint semantic representation of the answer pair. The overall semantic proximity result is calculated based on the independent semantic representations of the student's answer and the independent semantic representation of the reference answer. The semantic deviation representation of the student's answer relative to the reference answer is extracted based on the joint semantic representation of the answer pair. The overall semantic proximity result and semantic deviation representation are fused to generate the semantic matching result of the answer.

5. The method for analyzing educational data based on semantic processing according to claim 4, characterized in that, The process of generating question answer performance values ​​based on numerical comparison results and semantic matching results specifically includes: Perform numerical recognition on student answers and reference answers respectively, and extract the numerical sets of student answers and reference answers; Perform format conversion, unit conversion, and precision unification on the numerical expressions in the student answer numerical set and the reference answer numerical set to obtain a standardized student answer numerical set and a standardized reference answer numerical set. Establish a correspondence between the standardized student answer value set and the standardized reference answer value set according to the order of numerical appearance, perform a strict consistency comparison on the corresponding values, and generate numerical comparison results; When no numerical expression is extracted from the reference answer, the semantic matching result of the answer will be used as the performance value of the question answer; When numerical expressions are extracted from the reference answer, the question answer performance value is generated based on the numerical comparison results, the answer semantic matching results, and the normalized score in the student's answer record; the question answer performance value is jointly determined by the normalized score, the answer semantic matching results, and the numerical comparison results.

6. The method for analyzing educational data based on semantic processing according to claim 5, characterized in that, The initial clustering based on the semantic representation of the question specifically includes: For each question, the semantic representation is vectorized and the semantic distance between any two questions is calculated. Construct a question adjacency graph based on semantic distance, treat questions as nodes, and connect question nodes whose semantic distance meets the preset adjacency conditions as edges; Density-based clustering is performed on the adjacency graph of the questions, and the question nodes whose connectivity density meets the preset density condition are divided into the same initial potential knowledge dimension. Mark question nodes that do not meet the preset density conditions as question nodes to be merged; For each initial potential knowledge dimension, an initial dimension identifier is generated, and the correspondence between the initial dimension identifier and the question identifier is recorded to obtain the initial potential knowledge dimension set.

7. The method for analyzing educational data based on semantic processing according to claim 6, characterized in that, The process of performing a secondary merging and splitting of the initial potential knowledge dimension set based on the collective answer vectors of the questions specifically includes: According to the unified sorting of the desensitized student identifiers, the normalized score corresponding to each question is filled into the vector position in sequence to generate the question group answer vector, and the unanswered positions are filled with missing markers. For any two questions, select the vector positions corresponding to students who gave the same answer and calculate the similarity of their answer patterns. Within the same initial potential knowledge dimension, questions whose semantic distance satisfies the preset adjacency condition and whose answer pattern similarity satisfies the preset retention condition are retained in the same dimension, while questions whose semantic distance satisfies the preset adjacency condition but whose answer pattern similarity does not satisfy the preset retention condition are split into questions to be reorganized. Among different initial potential knowledge dimensions, questions whose semantic distance does not meet the preset adjacency condition but whose answer pattern similarity meets the preset merging condition are added to the candidate merging list; The candidate merge list and the questions to be reorganized are re-clustered based on the similarity of the answering patterns to generate a set of corrected potential knowledge dimensions.

8. The method for analyzing educational data based on semantic processing according to claim 7, characterized in that, The specific steps for generating dimensional explanation information to correct the potential knowledge dimension set include: For each potential knowledge dimension to be corrected, the corresponding question identifier set is read, and the text content of the corresponding question is extracted from the structured educational text; The text content is segmented and stop word filtered. The frequency and distinguishability of words in the current corrected potential knowledge dimension are calculated, and words that meet the preset sorting conditions are selected as dimension keywords. Calculate the semantic similarity between the semantic representation of each question and the center vector of the current modified potential knowledge dimension, and select the question with the highest semantic similarity as the representative question; Pseudo-tags are generated based on dimension keywords and representative titles, and the dimension keywords, representative titles, and pseudo-tags are associated with dimension explanation information.

9. The method for analyzing educational data based on semantic processing according to claim 8, characterized in that, The calculation of student mastery based on question-answering performance and a modified set of potential knowledge dimensions specifically includes: Read the correction dimension identifiers and corresponding question identifier sets from the correction potential knowledge dimension set; Based on the set of question identifiers, extract the question performance values ​​of the same student on the corresponding question from the student's answer records; The average performance of the same student in answering questions within the same modified latent knowledge dimension is calculated to obtain the student's mastery level in the corresponding modified latent knowledge dimension. When a student does not have a performance value for answering a question within the corresponding modified potential knowledge dimension, the student's mastery level in the corresponding modified potential knowledge dimension is marked as null.

10. The method for analyzing educational data based on semantic processing according to claim 9, characterized in that, The process of generating a mastery snapshot containing dimensional explanation information and student mastery levels according to the analysis batch specifically includes: According to the analysis batch, the corrected dimension identifier, dimension explanation information, question identifier set, corrected potential knowledge dimension center vector and student mastery are stored together to generate a mastery snapshot; Obtain the new question text, perform field concatenation on the new question text and input it into the semantic encoding model to obtain the semantic representation of the new question; Calculate the semantic similarity between the semantic representation of the new question and the center vector of each modified latent knowledge dimension in the mastery snapshot, and determine the modified latent knowledge dimension with the highest semantic similarity and that meets the preset matching conditions as the dimension to which the new question belongs; Read the de-identified student identifiers of the target students and retrieve the students' mastery of the target students in the dimension to which the new topic belongs from the mastery snapshot; The retrieved student mastery level is used as the target student's predicted mastery level of the new question. The output includes the dimension to which the new question belongs, the corresponding dimension explanation information, and the predicted mastery level.

Images