A novel intelligent review system based on multi-modal recognition and human-computer collaboration

Abstract

This invention discloses a novel intelligent grading system based on multimodal recognition and human-computer collaboration. The system comprises a response data acquisition module, a multimodal recognition module, a human-computer collaborative grading module, a diagnosis and recommendation module, a learning analysis module, and a data security and sharing module. It solves the problems of low efficiency, poor compatibility, and insufficient scoring consistency in traditional grading systems. This invention can achieve full-scenario response data coverage, accurately extract key information from responses, generate structured response data, automatically summarize incorrect answer data and perform attribution analysis, and simultaneously achieve cross-platform information sharing. Compared with traditional grading systems, it can significantly improve grading efficiency, ensure scoring consistency, and provide data support for adjusting teaching strategies and implementing differentiated instruction.

Description

Technical Field

[0001] This invention relates to the technical field of examination evaluation, and in particular to a novel intelligent evaluation system based on multimodal recognition and human-computer collaboration. Background Technology

[0002] There are many pain points in the current field of exam marking: traditional manual marking is inefficient and struggles to handle large-scale answer data, and subjective question scoring is easily influenced by teachers' subjective preferences, resulting in insufficient consistency; single-format data collection solutions have poor compatibility and cannot adapt to various answer formats such as paper scans, electronic documents, images, and online answer data, and problems such as blurriness, tilt, and shadows can also lead to information extraction distortion.

[0003] Existing intelligent grading systems often focus on single recognition tasks, lacking multimodal information fusion capabilities. Their accuracy in recognizing complex content such as formulas and charts is insufficient, and their semantic understanding and error correction mechanisms are inadequate. Furthermore, human-machine collaboration models are superficial; AI scoring models cannot dynamically optimize based on teachers' grading preferences and lack precise error attribution, student learning analysis, and personalized remediation functions. In addition, security risks during data transmission and storage, as well as barriers to cross-platform data sharing, also hinder the large-scale application of intelligent grading systems. Summary of the Invention

[0004] The primary objective of this invention is to overcome the shortcomings of existing technologies and propose a novel intelligent review system based on multimodal recognition and human-machine collaboration, aiming to solve the problems of low efficiency, poor compatibility, and insufficient scoring consistency in traditional review systems.

[0005] The second objective of this invention is to provide a novel intelligent review method based on multimodal recognition and human-machine collaboration.

[0006] A third objective of this invention is to provide a non-transitory computer-readable medium.

[0007] A fourth objective of this invention is to provide a computing device.

[0008] The first objective of this invention is achieved through the following technical solution: a novel intelligent review system based on multimodal recognition and human-machine collaboration, comprising:

[0009] The response data acquisition module is used to collect response data in multiple formats and perform optimized preprocessing on the response data;

[0010] The multimodal recognition module is used to extract objective question answer identifiers, subjective question text content, and information on special types of subjective questions from the answer data, and generate structured answer data.

[0011] The human-computer collaborative review module is used to perform human-computer collaborative review of structured answer data and obtain review result data.

[0012] The diagnosis and recommendation module summarizes students' wrong questions based on the review results, generates personalized wrong question notebooks and suggestions for filling gaps through attribution analysis of wrong questions, and recommends variation training based on knowledge point association rules.

[0013] The learning analysis module constructs visualized, multi-dimensional learning profiles for individual students, classes, and grades based on the evaluation results data, and analyzes and calculates core indicators such as the mastery of knowledge points, differences in learning performance among classes, and trends in learning progress.

[0014] The data security and sharing module uses the AES-256 encryption algorithm to store and encrypt answer data, evaluation results data, and multi-dimensional learning records. It also uses SSL / TLS protocol to encrypt communication and has a pre-set standardized data interface for cross-platform information sharing.

[0015] Furthermore, the response data acquisition module includes:

[0016] The system collects answer data in multiple formats, including scanned copies of paper exam papers, electronic documents, image formats, and data from online answering systems. It also locates the answer sheet filling areas for objective questions, segments the handwritten areas for subjective questions, and independently extracts formulas and charts. A multimodal data fusion acquisition architecture is used to optimize the preprocessing of the answer data, including:

[0017] 1.1) For scanned copies of paper test papers and answer data in image format, the fuzziness problem is optimized using an adaptive Gaussian filtering algorithm, as shown in formula (1):

[0018] (1);

[0019] By combining the Hough transform line detection algorithm, the tilt angle is corrected. As shown in formula (2):

[0020] (2);

[0021] Based on the Lab color space, the brightness channel is separated, and the shadow removal algorithm is used to remove the interference of environmental shadows, as shown in formula (3):

[0022] (3);

[0023] Where L represents the brightness channel value of the original image in the Lab color space, with a value range of 0-255, where 0 represents the darkest and 255 represents the brightest; This represents the corrected value of the luminance channel after shadow removal processing, with a value range of 0-255; k is the adaptive adjustment coefficient, with a value range of 0.1-0.3.

[0024] 1.2) For data from electronic documents and online question-answering systems, structured text and formula objects are extracted using XML tag parsing algorithms, and embedded image data is processed using Base64 encoding and decoding algorithms to ensure the integrity of cross-format data transmission;

[0025] 1.3) Based on the multi-format compatibility verification algorithm, verify the file header identifier and data, automatically identify and repair damaged data, as shown in formula (4):

[0026] CheckSum = ∑ i =0 n-1 data[i] mod 256 (4).

[0027] Furthermore, the multimodal recognition module includes:

[0028] 2.1) For printed and standard handwritten text in the answer data, an improved CRNN model is used for character recognition. Text image features are extracted by CNN, sequence dependencies are captured by RNN, and the CTC algorithm is used in the output layer to solve the character alignment problem, as shown in formula (5):

[0029] (5);

[0030] Where u is the input text image feature sequence, i.e., the response data; v is the predicted character sequence; T is the time step; and K is the number of character categories. Predict the probability of the k-th character at step t;

[0031] 2.2) Integrating a LaTeX formula recognition engine, for mathematical and physical formulas, a formula structure decomposition algorithm is used to decompose complex formulas into atomic symbols and combination relationships, as shown in formula (6):

[0032] (6);

[0033] in, The formula to be identified. Let n be the standard formula feature set, and n be the formula feature set to be identified. The number of features included, where m is the standard formula feature set. The number of features contained therein; For feature weights, This is the feature matching coefficient, which is 1 when there is a match and 0 when there is no match.

[0034] 2.3) For the semantic understanding of subjective questions, a BERT pre-trained model is used to construct a semantic similarity calculation model. Combined with the domain knowledge base, the recognition of professional terms is optimized, as shown in formula (7):

[0035] (7);

[0036] in, , Provide students with their own answer text and a sample answer text. , This is the vector representation of the corresponding text. The cosine similarity function;

[0037] 2.4) Integrate the recognition result error correction engine, and calculate the contextual semantic rationality through the N-gram language model, as shown in formula (8):

[0038] (8);

[0039] Where 'c' represents the candidate error correction character. For context character combinations, For the frequency of character combinations in the corpus, When the confidence level is below 0.3, a manual review prompt is triggered.

[0040] 2.5) The special types of subjective questions include answer information such as formula derivation, chart analysis, and code programming. For the special types of subjective questions, the target detection algorithm is used to locate the answer area, the answer elements are extracted by image segmentation technology, and the data dimensions and numerical relationships are analyzed by combining the type recognition model to extract the special types of subjective questions.

[0041] 2.6) Finally, the objective question answer identifiers, subjective question text content, and special type subjective question information extracted from the answer data in steps 2.1) to 2.5) are transformed into structured answer data.

[0042] Furthermore, the human-machine collaborative review module includes:

[0043] 3.1) Objective questions are reviewed in structured answer data. The answers to objective questions are quickly matched with the standard answers using an answer hash mapping algorithm, as shown in formula (9):

[0044] (9);

[0045] in, This is the standard answer to question i. Provide answers for students. Use the SHA-256 hash function; The matching function takes a value of 1 for a match and 0 for a non-match; n is the total number of objective questions; finally, the evaluation data for the objective questions is obtained.

[0046] 3.2) Preliminary evaluation of subjective questions: Based on the pre-set subject characteristics and scoring criteria, a multi-dimensional scoring model is constructed, as shown in formula (10):

[0047] (10);

[0048] Where m is the number of scoring indicators. Let k be the weight of the indicator. , The score for the k-th indicator ranges from 0 to 1.

[0049] Simultaneous annotation is performed based on AI algorithms:

[0050] a. Scoring criteria are based on the annotations. The key points, correct logic, or standard expressions in the answers that conform to the scoring criteria are analyzed to form a correspondence between scoring points and scoring criteria.

[0051] b. Marking points of contention: Mark ambiguous, boundary-point answers, and content that deviates from the scoring criteria but has reasonable basis, and provide analysis and explanation;

[0052] c. Marking errors to pinpoint conceptual mistakes, logical fallacies, and calculation errors in the answer;

[0053] Ultimately, the evaluation data for the subjective questions was obtained;

[0054] 3.3) For special types of subjective questions in structured answer data, the AI ​​algorithm directly extracts the special subjective question information and analyzes and scores it to obtain the evaluation data of special subjective questions;

[0055] 3.4) Perform manual review. The types of manual review include full review, sampling review and targeted review. Obtain the manual correction results. Adjust the review data obtained in steps 3.1) and 3.3) based on the manual correction results. The review data includes score correction values, correction opinions and indicator weight adjustment preferences. Update the AI ​​scoring model weights through the gradient descent algorithm, as shown in formula (11):

[0056] (11);

[0057] Where t is the number of iterations. The learning rate, with a value ranging from 0.01 to 0.05; The mean squared error between AI scoring and human scoring is shown in formula (12):

[0058] (12);

[0059] in, Give the AI ​​an initial score. Final evaluation for teachers;

[0060] After manual review, the evaluation results data for objective questions, subjective questions, and special subjective questions were obtained.

[0061] Furthermore, the diagnosis and recommendation module includes:

[0062] 4.1) Based on the review results, summarize the students' wrong answer data, conduct attribution analysis on the students' wrong answer data, construct a three-layer attribution model of basic dimension - deep dimension - scenario dimension, and determine the core error type through weighted scoring and feature fusion algorithm, as shown in formula (13):

[0063] (13);

[0064] in, This represents a knowledge gap. Representative method defects, This represents carelessness and mistakes; , , As attribution weights, , For knowledge point matching degree, To ensure the suitability of the problem-solving method, To indicate the standardization of answers, values ​​range from 0 to 1.

[0065] 4.2) Personalized reinforcement recommendation: Based on multi-dimensional data such as the mastery status, difficulty level, and learning progress of the knowledge points associated with the wrong questions, a reinforcement priority calculation model is constructed, as shown in formula (14):

[0066] (14);

[0067] Where k is the knowledge point number, To assess the level of mastery of this knowledge point. This represents the difficulty level of the knowledge point, with a value ranging from 0.1 to 1.0. This represents the time interval between the last review of this knowledge point;

[0068] Based on the error type and the priority of filling in the gaps, corresponding resources are recommended. For errors with knowledge gaps, videos explaining the knowledge points, excerpts from the textbook, and documents clarifying concepts are recommended. For errors with methodological defects, tutorials on problem-solving techniques, detailed explanations of typical examples, and step-by-step guides are recommended. For errors with carelessness, training on question reading skills, checklists for detailed verification, and timed and standardized practice questions are recommended.

[0069] 4.3) Variation training generation: Through feature extraction of incorrect questions, mapping of knowledge point association rules, and intelligent generation algorithms, variation questions that are corely related to the incorrect questions and have ability transfer value are generated, as shown in formula (15):

[0070] (15);

[0071] in, , , As weight, , For knowledge point similarity, For question type structure similarity, For difficulty similarity, the values ​​for knowledge point similarity, question type structure similarity, and difficulty similarity range from 0 to 1.

[0072] Furthermore, the learning analysis module includes:

[0073] 5.1) Calculation of knowledge point mastery: Based on multi-source data such as the distribution of incorrect questions, the accuracy rate of answering questions, and the effect of review, a refined knowledge point mastery assessment model is constructed, as shown in formula (16):

[0074] (16);

[0075] Where k is the knowledge point number, This represents the total number of questions corresponding to this knowledge point. This indicates that the answer to question i is correct. This refers to the number of times this knowledge point has been reviewed. For review purposes, the weighting coefficients should be set between 0.2 and 0.5.

[0076] A dynamic update mechanism will be established to immediately recalculate mastery levels and update learning progress records after students complete new quizzes or review tasks; a time decay coefficient will be introduced for knowledge points that have not been reviewed for a long time. t is the number of days since the last effective review, and the corrected mastery level is shown in formula (17):

[0077] (17);

[0078] 5.2) Class learning difference analysis: The degree of difference in the mastery of knowledge points within the class is assessed by combining the coefficient of variation with stratified statistics, as shown in formula (18):

[0079] (18);

[0080] in, Standard deviation, This is the average value. Let represent the degree of mastery of knowledge point k by the i-th student. The larger the value, the more significant the differences in students' mastery of that knowledge point.

[0081] Based on association rule algorithms and cluster analysis, common learning problems of the class as a whole are identified. The Apriori algorithm is used to calculate the support and confidence between knowledge points, identify high-frequency common error combinations, and determine the weak links between knowledge points. K-means clustering is performed on the error types of the class's wrong questions. If the cluster center of a certain type of error accounts for more than 40%, it is determined to be a common error of the class. Combined with the ability dimensions tested in the questions, the average score and coefficient of variation of each ability dimension of the class are calculated to identify the class's ability weaknesses.

[0082] 5.3) Modeling the learning progress trend: Using exponential smoothing combined with a sliding window, the trajectory of student learning improvement is accurately depicted, as shown in formula (19):

[0083] (19);

[0084] Where t is the time node. This represents the overall score for the t-th test. This is a smoothing coefficient, with a value range of 0.3-0.7; Let be the progress trend value at node t;

[0085] By integrating the exponential smoothing method, the linear regression model, and the LSTM neural network model, a combined prediction model is constructed, as shown in formula (20):

[0086] (20);

[0087] in, These are values ​​predicted using exponential smoothing. These are the predicted values ​​from the linear regression model. The predicted value is from the LSTM neural network. , As weight, ;

[0088] The influence of each factor on the learning progress trend is quantified by using a multiple linear regression model, as shown in formula (21):

[0089] (twenty one);

[0090] in, Let t be the learning duration of stage t. To improve the accuracy of answers, To determine the frequency of review, For constant terms, , , For regression coefficients, This is random error.

[0091] Furthermore, the data security and sharing module includes:

[0092] 6.1) Establish a data encryption storage mechanism. Use the AES-256 symmetric encryption algorithm to encrypt and store the answer data, evaluation result data, and structured data in the multi-dimensional learning archives. The encryption process is combined with salt value to enhance the anti-cracking ability, as shown in formula (22):

[0093] (twenty two);

[0094] in, This is the system master key. A randomly generated 16-byte salt value. The original data, For encrypted data;

[0095] For the unstructured data in the answer data, evaluation results data, and multi-dimensional learning portfolios, a segmented encryption storage strategy is adopted. The segment size is determined by a dynamic adaptive algorithm, as shown in formula (23):

[0096] (twenty three);

[0097] in, The original file size is n; n is the number of partitions, ranging from 5 to 20; this number is dynamically adjusted based on the file size.

[0098] 6.2) Secure transmission guarantee: The transmission channel is built based on the SSL / TLS 1.3 protocol, the session key is negotiated using the ECDHE key exchange algorithm, and data integrity is verified using the HMAC-SHA256 algorithm, as shown in formula (24):

[0099] (twenty four);

[0100] in, To transmit data packets, For session key, Fill the outer layer with the key. Fill the inner layer with keys;

[0101] 6.3) Fine-grained access control: Based on the RBAC model, an access control matrix is ​​constructed, and sensitive information is hierarchically anonymized using a data anonymization algorithm, as shown in formula (25):

[0102] (25);

[0103] in, For visitor role level, Data sensitivity level, For access scenario coefficients; , , ;

[0104] 6.4) Cross-platform data sharing and adaptation: Build standardized RESTful API interfaces and adopt data format conversion algorithms to achieve data compatibility, as shown in formula (26):

[0105] (26);

[0106] in, For source system data fragments, For the target system data format, This is a format conversion function. This is standard format reference data. For data consistency matching functions, is the data length, and m is the total number of data segments.

[0107] The second objective of this invention is achieved through the following technical solution: a novel intelligent evaluation method based on multimodal recognition and human-computer collaboration, which calls the answer data acquisition module, multimodal recognition module, human-computer collaborative evaluation module, diagnosis and recommendation module, learning analysis module, and data security and sharing module in the aforementioned novel intelligent evaluation system based on multimodal recognition and human-computer collaboration.

[0108] The third objective of this invention is achieved through the following technical solution: a non-transitory computer-readable medium storing instructions, which, when executed by a processor, execute the novel intelligent review method based on multimodal recognition and human-machine collaboration described above.

[0109] The fourth objective of this invention is achieved through the following technical solution: a computing device, including a processor and a memory for storing processor-executable programs, wherein when the processor executes the program stored in the memory, it implements the above-mentioned novel intelligent review method based on multimodal recognition and human-machine collaboration.

[0110] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0111] 1. This invention supports the collection of answer data in multiple formats, covering scanned copies of paper test papers, electronic documents, images, and real-time data transmitted from online answering systems. Preprocessing techniques are used to optimize data with issues such as blurriness, tilt, and shadows, ensuring the integrity of the collected information. It is compatible with diverse answer formats including objective questions, subjective questions, formula derivations, chart analysis, and code programming, achieving full-scenario coverage of answer data.

[0112] 2. This invention can accurately extract key information from answers and generate structured answer data, which can significantly improve the efficiency of grading compared with traditional grading systems.

[0113] 3. This invention forms a human-machine collaborative mode of "algorithm initial evaluation + manual review", which can dynamically optimize preferences and further improve the consistency of scoring.

[0114] 4. This invention automatically summarizes students' incorrect answer data, associates the questions with knowledge points, difficulty levels, and assessed abilities, and through error attribution analysis, identifies problems such as knowledge gaps, methodological deficiencies, or careless mistakes, generating personalized error notebooks and remedial suggestions. Combined with knowledge point association rules, it recommends variation exercises to help students target their weaknesses and improve learning outcomes.

[0115] 5. This invention uses data visualization to present weak points in learning and progress trends, deeply explores the learning patterns of individuals and groups, and provides data support for teachers to adjust teaching strategies and implement differentiated instruction.

[0116] 6. This invention establishes a standardized data interface, supports integration with third-party systems, and enables cross-platform sharing of answer data and learning analysis results. Attached Figure Description

[0117] Figure 1 This is a flowchart of a novel intelligent review system based on multimodal recognition and human-machine collaboration. Detailed Implementation

[0118] The present invention will be further described below with reference to specific embodiments.

[0119] Example 1

[0120] See Figure 1 As shown, the novel intelligent review system based on multimodal recognition and human-machine collaboration provided in this embodiment includes:

[0121] 1) Response data acquisition module, used to collect response data in multiple formats and perform optimized preprocessing on the response data.

[0122] The system collects answer data in multiple formats, including scanned copies of paper exam papers, electronic documents, image formats, and data from online answering systems. It also locates the answer sheet filling areas for objective questions, segments the handwritten areas for subjective questions, and independently extracts formulas and charts. A multimodal data fusion acquisition architecture is used to optimize the preprocessing of the answer data, including:

[0123] 1.1) For scanned copies of paper test papers and answer data in image format, the fuzziness problem is optimized using an adaptive Gaussian filtering algorithm, as shown in formula (1):

[0124] (1).

[0125] By combining the Hough transform line detection algorithm, the tilt angle is corrected. As shown in formula (2):

[0126] (2).

[0127] Based on the Lab color space, the brightness channel is separated, and the shadow removal algorithm is used to remove the interference of environmental shadows, as shown in formula (3):

[0128] (3);

[0129] Where L represents the brightness channel value of the original image in the Lab color space, with a value range of 0-255, where 0 represents the darkest and 255 represents the brightest; This represents the corrected value of the luminance channel after shadow removal processing, with a value range of 0-255; k is the adaptive adjustment coefficient, with a value range of 0.1-0.3.

[0130] 1.2) For data from electronic documents and online question-answering systems, structured text and formula objects are extracted using an XML tag parsing algorithm, and embedded image data is processed using a Base64 encoding and decoding algorithm to ensure the integrity of cross-format data transmission.

[0131] 1.3) Based on the multi-format compatibility verification algorithm, verify the file header identifier and data, automatically identify and repair damaged data, as shown in formula (4):

[0132] CheckSum = ∑ i =0 n-1 data[i] mod 256 (4).

[0133] 2) Multimodal recognition module, which integrates multiple engines such as optical character recognition (OCR), handwriting recognition, formula recognition and semantic understanding. Through the multi-engine fusion architecture, it extracts objective question answer identifiers, subjective question text content and special types of subjective question information from the answer data to generate structured answer data.

[0134] 2.1) For printed and standard handwritten text in the answer data, based on the optical character recognition (OCR) engine, an improved CRNN model is used for character recognition. Text image features are extracted by CNN, sequence dependencies are captured by RNN, and the CTC algorithm is used in the output layer to solve the character alignment problem, as shown in formula (5):

[0135] (5);

[0136] Where u is the input text image feature sequence, i.e., the response data; v is the predicted character sequence; T is the time step; and K is the number of character categories. Let t be the probability of predicting the k-th character.

[0137] 2.2) Integrating a LaTeX formula recognition engine, for mathematical and physical formulas, a formula structure decomposition algorithm is used to decompose complex formulas into atomic symbols and combination relationships, as shown in formula (6):

[0138] (6);

[0139] in, The formula to be identified. Let n be the standard formula feature set, and n be the formula feature set to be identified. The number of features included, where m is the standard formula feature set. The number of features contained therein; For feature weights, This is the feature matching coefficient, which is 1 when there is a match and 0 when there is no match.

[0140] 2.3) For the semantic understanding of subjective questions, based on the semantic understanding engine, a BERT pre-trained model is used to construct a semantic similarity calculation model. Combined with the domain knowledge base, the recognition of professional terms is optimized, as shown in formula (7):

[0141] (7);

[0142] in, , Provide students with their own answer text and a sample answer text. , This is the vector representation of the corresponding text. This is the cosine similarity function.

[0143] 2.4) Integrate the recognition result error correction engine, and calculate the contextual semantic rationality through the N-gram language model, as shown in formula (8):

[0144] (8);

[0145] Where 'c' represents the candidate error correction character. For context character combinations, For the frequency of character combinations in the corpus, When the confidence level is below 0.3, a manual review prompt is triggered.

[0146] 2.5) The special types of subjective questions include answer information such as formula derivation, chart analysis, and code programming. For the special types of subjective questions, the target detection algorithm is used to locate the answer area, the answer elements are extracted by image segmentation technology, and the data dimensions and numerical relationships are analyzed by combining the type recognition model to extract the special types of subjective questions.

[0147] 2.6) Finally, the objective question answer identifiers, subjective question text content, and special type subjective question information extracted from the answer data in steps 2.1) to 2.5) are transformed into structured answer data.

[0148] 3) The human-machine collaborative review module adopts a dynamic scoring fusion mechanism to achieve efficient collaboration between AI grading and human review. It is used to conduct human-machine collaborative review of structured answer data to obtain review result data.

[0149] 3.1) Objective questions are reviewed in structured answer data. The answers to objective questions are quickly matched with the standard answers using an answer hash mapping algorithm, as shown in formula (9):

[0150] (9);

[0151] in, This is the standard answer to question i. Provide answers for students. Use the SHA-256 hash function; The matching function takes a value of 1 for a match and 0 for a non-match; n is the total number of objective questions; finally, the evaluation data for the objective questions is obtained.

[0152] 3.2) Preliminary evaluation of subjective questions: Based on the pre-set subject characteristics and scoring criteria, a multi-dimensional scoring model is constructed, as shown in formula (10):

[0153] (10);

[0154] Where m is the number of scoring indicators. Let k be the weight of the indicator. , The score for the k-th indicator ranges from 0 to 1.

[0155] Simultaneous annotation is performed based on AI algorithms:

[0156] a. Scoring criteria are marked, and the key points, correct logic, or standard expressions in the answer are analyzed to form a correspondence between scoring points and scoring criteria.

[0157] b. Marking points of contention: Mark ambiguous, boundary-point answers, and content that deviates from the scoring criteria but has reasonable basis, and provide analysis and explanation.

[0158] c. Marking errors to pinpoint conceptual mistakes, logical fallacies, and calculation errors in the answer;

[0159] Meanwhile, it supports a flexible scoring mode that uses AI algorithms to score subjective questions in segments and with keyword weighting, adapting to the scoring needs of different subjects.

[0160] Ultimately, the evaluation data for the subjective questions was obtained;

[0161] 3.3) Special types of subjective questions include answer information such as formula derivation, chart analysis, and code programming. For special types of subjective questions in structured answer data, the AI ​​algorithm directly extracts the special subjective question information and analyzes and scores it to obtain the evaluation data of special subjective questions.

[0162] 3.4) Manual review is conducted, including full review, sampling review, and targeted review, to obtain manual grading results. Full review involves teachers reviewing the initial assessment results and annotations for each question, adjusting scores independently, and providing feedback. Sampling review involves randomly selecting 5%-20% of the answers based on statistical principles for review, balancing efficiency and quality. Targeted review involves teachers reviewing only marked controversial questions, high-scoring questions, low-scoring questions, and other specific types of questions, focusing on key content.

[0163] The review data obtained in steps 3.1) and 3.3) are adjusted based on the results of manual grading. The review data includes score correction values, grading comments, and indicator weight adjustment preferences. The weights of the AI ​​scoring model are updated using the gradient descent algorithm, as shown in formula (11):

[0164] (11);

[0165] Where t is the number of iterations. The learning rate, with a value ranging from 0.01 to 0.05; The mean squared error between AI scoring and human scoring is shown in formula (12):

[0166] (12);

[0167] in, Give the AI ​​an initial score. The teacher will receive a final evaluation.

[0168] After manual review, the evaluation results data for objective questions, subjective questions, and special subjective questions were obtained.

[0169] To enhance the relevance of model iterations, the system constructs a profile of teachers' grading preferences, encompassing dimensions such as grading rigor, emphasis on specific knowledge points, and expression standards. These preferences are integrated into the training process of the grading model. Simultaneously, an iteration feedback mechanism is implemented, showing teachers performance improvement data after each iteration. Teachers can adjust their iteration strategies based on actual results, forming a closed-loop iteration system of AI initial assessment – ​​human review – data feedback – model optimization. This allows the AI ​​grading model to gradually adapt to different teachers' grading styles, improving long-term collaborative grading efficiency.

[0170] 4) The diagnosis and recommendation module adopts a multi-dimensional attribution mechanism for incorrect questions, personalized recommendation, and dynamic optimization. Combining deep learning algorithms with the needs of teaching scenarios, it constructs a closed-loop diagnosis system of incorrect question collection, attribution analysis, gap-filling recommendation, and effect feedback. This achieves precise transformation from incorrect question data to learning improvement. Based on the review results, it summarizes students' incorrect question data, generates personalized incorrect question notebooks and gap-filling suggestions through attribution analysis of incorrect questions, and recommends variation training based on knowledge point association rules.

[0171] 4.1) Based on the review results, summarize the students' wrong answer data, conduct attribution analysis on the students' wrong answer data, construct a three-layer attribution model of basic dimension - deep dimension - scenario dimension, and determine the core error type through weighted scoring and feature fusion algorithm, as shown in formula (13):

[0172] (13);

[0173] in, This represents a knowledge gap. Representative method defects, This represents carelessness and mistakes; , , As attribution weights, , For knowledge point matching degree, To ensure the suitability of the problem-solving method, To indicate the standardization of the answers, values ​​are all set between 0 and 1.

[0174] To improve the accuracy of attribution, two additional auxiliary judgment mechanisms are implemented: The first layer is a historical correlation analysis of incorrect answers. If the same knowledge point is repeatedly missed in different tests, the weight of the knowledge gap attribution is automatically increased. The second layer is a tracking of the answering process, applicable to online question-and-answer systems. By recording data such as answering time and modification history, it determines whether careless mistakes were caused by time constraints. The attribution results will generate a detailed error analysis report, clearly identifying the error type, core cause, related knowledge points, and improvement directions for students and teachers to refer to.

[0175] 4.2) Personalized reinforcement recommendation: Based on multi-dimensional data such as the mastery status, difficulty level, and learning progress of the knowledge points associated with the wrong questions, a reinforcement priority calculation model is constructed, as shown in formula (14):

[0176] (14);

[0177] Where k is the knowledge point number, To assess the level of mastery of this knowledge point. This represents the difficulty level of the knowledge point, with a value ranging from 0.1 to 1.0. This represents the time interval between the last time this knowledge point was reinforced.

[0178] Based on the error type and priority of remediation, corresponding resources are recommended. For errors due to knowledge gaps, videos explaining the knowledge points, excerpts from textbooks, and document clarifying concepts are recommended. For errors due to methodological deficiencies, tutorials on problem-solving techniques, detailed explanations of typical examples, and step-by-step breakdown guides are recommended. For errors due to carelessness, training on question reading skills, detailed checklists, and timed, standardized practice questions are recommended. The system supports various resource formats, including text, video, audio, and interactive exercises. Students can choose according to their learning habits, and the system records resource usage preferences and optimizes subsequent recommendations.

[0179] 4.3) Variation training generation: Through feature extraction of incorrect questions, mapping of knowledge point association rules, and intelligent generation algorithms, variation questions that are corely related to the incorrect questions and have ability transfer value are generated, as shown in formula (15):

[0180] (15);

[0181] in, , , As weight, , For knowledge point similarity, For question type structure similarity, For difficulty similarity, the values ​​for knowledge point similarity, question type structure similarity, and difficulty similarity range from 0 to 1.

[0182] The variation training generation process includes three quality checks: First, logical consistency check, which uses subject expert rule base and AI semantic analysis to ensure that the variation questions are consistent, logically sound, and have unique and correct answers; second, repetition check, which compares the generated variation questions with the system's question bank to ensure that the repetition rate between the generated variation questions and existing questions is less than 15%, thus avoiding ineffective training; and third, suitability check, which combines the students' ability level to ensure that the difficulty of the variation questions does not exceed the students' current acceptable range.

[0183] 4.4) Effect Feedback and Dynamic Optimization: Establish a closed-loop feedback mechanism for the gap-filling effect. Evaluate the effectiveness of gap-filling recommendations and variant training, and dynamically optimize the diagnostic model and recommendation strategy. Set a period for regularly analyzing the effect evaluation data, and adjust the attribution model weights using the gradient descent algorithm. , , Similarity weights in variant training , , We optimized the parameters of the priority calculation model for filling gaps in learning, making the diagnosis and recommendations more aligned with students' actual learning needs.

[0184] 5) The learning analysis module constructs visualized, multi-dimensional learning profiles for individual students, classes, and grades based on assessment results data. It analyzes and calculates core indicators such as knowledge point mastery, class-level learning differences, and learning progress trends. Employing multi-dimensional data modeling, visualization analysis, and dynamic iteration mechanisms, it integrates statistical algorithms, machine learning models, and teaching scenario requirements to build a comprehensive learning analysis system encompassing "data collection - model calculation - visualization presentation - decision support - feedback optimization." This enables accurate characterization of individual and group learning status, trend prediction, and teaching adaptation.

[0185] 5.1) Calculation of knowledge point mastery: Based on multi-source data such as the distribution of incorrect questions, the accuracy rate of answering questions, and the effect of review, a refined knowledge point mastery assessment model is constructed, as shown in formula (16):

[0186] (16);

[0187] Where k is the knowledge point number, This represents the total number of questions corresponding to this knowledge point. This indicates that the answer to question i is correct. This refers to the number of times this knowledge point has been reviewed. For review purposes, the weighting coefficients should be set between 0.2 and 0.5.

[0188] To ensure the timeliness and accuracy of mastery assessment, a dynamic update mechanism is established. Mastery is recalculated immediately after students complete new quizzes or review tasks, and their learning progress is updated accordingly. For knowledge points that have not been reviewed for a long time, a time decay coefficient is introduced. t is the number of days since the last effective review, and the corrected mastery level is shown in formula (17):

[0189] (17).

[0190] 5.2) Class learning difference analysis: The degree of difference in the mastery of knowledge points within the class is assessed by combining the coefficient of variation with stratified statistics, as shown in formula (18):

[0191] (18);

[0192] in, Standard deviation, This is the average value. Let represent the degree of mastery of knowledge point k by the i-th student. The larger the value, the more significant the differences in students' mastery of that knowledge point.

[0193] Based on association rule algorithms and cluster analysis, common learning problems of the class as a whole are identified. The Apriori algorithm is used to calculate the support and confidence between knowledge points, identify high-frequency common error combinations, and determine the weak links in the linkage between knowledge points. K-means clustering is performed on the error types of the class's wrong questions. If the cluster center of a certain type of error accounts for more than 40%, it is determined to be a common error of the class. Combined with the ability dimensions tested in the questions, the average score and coefficient of variation of each ability dimension of the class are calculated to identify the class's ability weaknesses.

[0194] 5.3) Modeling the learning progress trend: Using exponential smoothing combined with a sliding window, the trajectory of student learning improvement is accurately depicted, as shown in formula (19):

[0195] (19);

[0196] Where t is the time node. This represents the overall score for the t-th test. This is a smoothing coefficient, with a value range of 0.3-0.7; Let be the progress trend value of node t.

[0197] By integrating the exponential smoothing method, the linear regression model, and the LSTM neural network model, a combined prediction model is constructed, as shown in formula (20):

[0198] (20);

[0199] in, These are values ​​predicted using exponential smoothing. These are the predicted values ​​from the linear regression model. The predicted value is from the LSTM neural network. , As weight, .

[0200] The influence of each factor on the learning progress trend is quantified by using a multiple linear regression model, as shown in formula (21):

[0201] (twenty one);

[0202] in, Let t be the learning duration of stage t. To improve the accuracy of answers, To determine the frequency of review, For constant terms, , , For regression coefficients, This is random error.

[0203] 5.4) Precise Adaptation and Personalized Recommendations. Based on multi-dimensional learning profiles, targeted teaching suggestions are provided to teachers. Customized learning guides are generated by combining individual student learning data. A closed-loop feedback mechanism is established to evaluate the effectiveness of the analysis model through multi-dimensional data and continuously optimize it.

[0204] 6) The data security and sharing module uses the AES-256 encryption algorithm to encrypt and store answer data, evaluation results, and multi-dimensional learning profiles. It also uses SSL / TLS encryption for communication and has a pre-defined standardized data interface to support integration with third-party systems such as campus academic management systems and teaching resource platforms, enabling cross-platform information sharing. A full-link security architecture of encrypted storage, secure transmission, access control, and cross-platform compatibility is adopted to ensure the confidentiality, integrity, and availability of core data.

[0205] 6.1) Establish a data encryption storage mechanism. Use the AES-256 symmetric encryption algorithm to encrypt and store the answer data, evaluation result data, and structured data in the multi-dimensional learning archives. The encryption process is combined with salt value to enhance the anti-cracking ability, as shown in formula (22):

[0206] (twenty two);

[0207] in, This is the system master key. A randomly generated 16-byte salt value. The original data, This is the encrypted data.

[0208] For the unstructured data in the answer data, evaluation results data, and multi-dimensional learning portfolios, a segmented encryption storage strategy is adopted. The segment size is determined by a dynamic adaptive algorithm, as shown in formula (23):

[0209] (twenty three);

[0210] in, The original file size is n; the number of segments is n, which ranges from 5 to 20 and is dynamically adjusted based on the file size.

[0211] 6.2) Secure transmission guarantee: The transmission channel is built based on the SSL / TLS 1.3 protocol, the session key is negotiated using the ECDHE key exchange algorithm, and data integrity is verified using the HMAC-SHA256 algorithm, as shown in formula (24):

[0212] (twenty four);

[0213] in, To transmit data packets, For session key, Fill the outer layer with the key. Fill the inner layer with the key.

[0214] 6.3) Fine-grained access control: Based on the RBAC model, an access control matrix is ​​constructed, and sensitive information is hierarchically anonymized using a data anonymization algorithm, as shown in formula (25):

[0215] (25);

[0216] in, For visitor role level, Data sensitivity level, For access scenario coefficients; , , .

[0217] 6.4) Cross-platform data sharing and adaptation: Build standardized RESTful API interfaces and adopt data format conversion algorithms to achieve data compatibility, as shown in formula (26):

[0218] (26);

[0219] in, For source system data fragments, For the target system data format, This is a format conversion function. This is standard format reference data. For data consistency matching functions, is the data length, and m is the total number of data segments.

[0220] Example 2

[0221] The novel intelligent grading method based on multimodal recognition and human-computer collaboration provided in this embodiment is implemented by calling the answer data acquisition module, multimodal recognition module, human-computer collaborative grading module, diagnosis and recommendation module, learning analysis module, and data security and sharing module in the novel intelligent grading system based on multimodal recognition and human-computer collaboration described in Embodiment 1.

[0222] Example 3

[0223] This embodiment discloses a non-transitory computer-readable medium storing instructions that, when executed by a processor, perform the steps of the novel intelligent review method based on multimodal recognition and human-machine collaboration as described in Embodiment 2.

[0224] In this embodiment, the non-transitory computer-readable medium can be a disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), USB flash drive, portable hard drive, etc.

[0225] Example 4

[0226] This embodiment discloses a computing device, including a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the novel intelligent review method based on multimodal recognition and human-machine collaboration described in Embodiment 2.

[0227] The computing device described in this embodiment may be a desktop computer, laptop computer, smartphone, PDA handheld terminal, tablet computer, programmable logic controller (PLC), or other terminal device with processor function.

[0228] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A novel intelligent review system based on multimodal recognition and human-machine collaboration, characterized in that, include: The response data acquisition module is used to collect response data in multiple formats and perform optimized preprocessing on the response data; The multimodal recognition module is used to extract objective question answer identifiers, subjective question text content, and information on special types of subjective questions from the answer data, and generate structured answer data. The human-computer collaborative review module is used to perform human-computer collaborative review of structured answer data and obtain review result data. The diagnosis and recommendation module summarizes students' wrong questions based on the review results, generates personalized wrong question notebooks and suggestions for filling gaps through attribution analysis of wrong questions, and recommends variation training based on knowledge point association rules. The learning analysis module constructs visualized, multi-dimensional learning profiles for individual students, classes, and grades based on the evaluation results data, and analyzes and calculates core indicators such as the mastery of knowledge points, differences in learning performance among classes, and trends in learning progress. The data security and sharing module stores and encrypts answer data, evaluation results data, and multi-dimensional learning records according to a preset encryption algorithm, encrypts communication through a preset protocol, and sets up a standardized data interface for cross-platform information sharing.

2. The novel intelligent review system based on multimodal recognition and human-machine collaboration according to claim 1, characterized in that, The response data acquisition module includes: The system collects answer data in multiple formats, including scanned copies of paper exam papers, electronic documents, image formats, and data from online answering systems. It also locates the answer sheet filling areas for objective questions, segments the handwritten areas for subjective questions, and independently extracts formulas and charts. A multimodal data fusion acquisition architecture is used to optimize the preprocessing of the answer data, including: 1.1) For scanned copies of paper test papers and answer data in image format, the fuzziness problem is optimized using an adaptive Gaussian filtering algorithm, as shown in formula (1): (1)； By combining the Hough transform line detection algorithm, the tilt angle is corrected. As shown in formula (2): (2)； Based on the Lab color space, the brightness channel is separated, and the shadow removal algorithm is used to remove the interference of environmental shadows, as shown in formula (3): (3)； Where L represents the brightness channel value of the original image in the Lab color space, with a value range of 0-255, where 0 represents the darkest and 255 represents the brightest; This represents the corrected value of the luminance channel after shadow removal processing, with a value range of 0-255; k is the adaptive adjustment coefficient, with a value range of 0.1-0.

3. 1.2) For data from electronic documents and online question-answering systems, structured text and formula objects are extracted using XML tag parsing algorithms, and embedded image data is processed using Base64 encoding and decoding algorithms to ensure the integrity of cross-format data transmission; 1.3) Based on the multi-format compatibility verification algorithm, verify the file header identifier and data, automatically identify and repair damaged data, as shown in formula (4): (4)。 3. The novel intelligent review system based on multimodal recognition and human-machine collaboration according to claim 1, characterized in that, The multimodal recognition module includes: 2.1) For printed and standard handwritten text in the answer data, an improved CRNN model is used for character recognition. Text image features are extracted by CNN, sequence dependencies are captured by RNN, and the CTC algorithm is used in the output layer to solve the character alignment problem, as shown in formula (5): (5)； Where u is the input text image feature sequence, i.e., the response data; v is the predicted character sequence; T is the time step; and K is the number of character categories. Predict the probability of the k-th character at step t; 2.2) Integrating a LaTeX formula recognition engine, for mathematical and physical formulas, a formula structure decomposition algorithm is used to decompose complex formulas into atomic symbols and combination relationships, as shown in formula (6): (6)； in, The formula to be identified. Let n be the standard formula feature set, and n be the formula feature set to be identified. The number of features included, where m is the standard formula feature set. The number of features contained therein; For feature weights, This is the feature matching coefficient, which is 1 when there is a match and 0 when there is no match. 2.3) For the semantic understanding of subjective questions, a BERT pre-trained model is used to construct a semantic similarity calculation model. Combined with the domain knowledge base, the recognition of professional terms is optimized, as shown in formula (7): (7)； in, , Provide students with their own answer text and a sample answer text. , This is the vector representation of the corresponding text. The cosine similarity function; 2.4) Integrate the recognition result error correction engine, and calculate the contextual semantic rationality through the N-gram language model, as shown in formula (8): (8)； Where 'c' represents the candidate error correction character. For context character combinations, For the frequency of character combinations in the corpus, When the confidence level is below 0.3, a manual review prompt is triggered. 2.5) The special types of subjective questions include answer information such as formula derivation, chart analysis, and code programming. For the special types of subjective questions, the target detection algorithm is used to locate the answer area, the answer elements are extracted by image segmentation technology, and the data dimensions and numerical relationships are analyzed by combining the type recognition model to extract the special types of subjective questions. 2.6) Finally, the objective question answer identifiers, subjective question text content, and special type subjective question information extracted from the answer data in steps 2.1) to 2.5) are transformed into structured answer data.

4. The novel intelligent review system based on multimodal recognition and human-machine collaboration according to claim 1, characterized in that, The human-machine collaborative review module includes: 3.1) Objective questions are reviewed in structured answer data. The answers to objective questions are quickly matched with the standard answers using an answer hash mapping algorithm, as shown in formula (9): (9)； in, This is the standard answer to question i. Provide answers for students. Use the SHA-256 hash function; The matching function takes a value of 1 for a match and 0 for a non-match; n is the total number of objective questions; finally, the evaluation data for the objective questions is obtained. 3.2) Preliminary evaluation of subjective questions: Based on the pre-set subject characteristics and scoring criteria, a multi-dimensional scoring model is constructed, as shown in formula (10): (10)； Where m is the number of scoring indicators. Let k be the weight of the indicator. , The score for the k-th indicator ranges from 0 to 1. Simultaneous annotation is performed based on AI algorithms: a. Scoring criteria are based on the annotations. The key points, correct logic, or standard expressions in the answers that conform to the scoring criteria are analyzed to form a correspondence between scoring points and scoring criteria. b. Marking points of contention: Mark ambiguous, boundary-point answers, and content that deviates from the scoring criteria but has reasonable basis, and provide analysis and explanation; c. Marking errors to pinpoint conceptual mistakes, logical fallacies, and calculation errors in the answer; Ultimately, the evaluation data for the subjective questions was obtained; 3.3) For special types of subjective questions in structured answer data, the AI ​​algorithm directly extracts the special subjective question information and analyzes and scores it to obtain the evaluation data of special subjective questions; 3.4) Perform manual review. The types of manual review include full review, sampling review and targeted review. Obtain the manual correction results. Adjust the review data obtained in steps 3.1) and 3.3) based on the manual correction results. The review data includes score correction values, correction opinions and indicator weight adjustment preferences. Update the AI ​​scoring model weights through the gradient descent algorithm, as shown in formula (11): (11)； Where t is the number of iterations. The learning rate, with a value ranging from 0.01 to 0.05; The mean squared error between AI scoring and human scoring is shown in formula (12): (12)； in, Give the AI ​​an initial score. Final evaluation for teachers; After manual review, the evaluation results data for objective questions, subjective questions, and special subjective questions were obtained.

5. A novel intelligent review system based on multimodal recognition and human-machine collaboration according to claim 1, characterized in that, The diagnosis and recommendation module includes: 4.1) Based on the review results, summarize the students' wrong answer data, conduct attribution analysis on the students' wrong answer data, construct a three-layer attribution model of basic dimension - deep dimension - scenario dimension, and determine the core error type through weighted scoring and feature fusion algorithm, as shown in formula (13): (13)； in, This represents a knowledge gap. Representative method defects, This represents carelessness and mistakes; , , As attribution weights, , For knowledge point matching degree, To ensure the suitability of the problem-solving method, To indicate the standardization of answers, values ​​range from 0 to 1. 4.2) Personalized reinforcement recommendation: Based on multi-dimensional data such as the mastery status, difficulty level, and learning progress of the knowledge points associated with the wrong questions, a reinforcement priority calculation model is constructed, as shown in formula (14): (14)； Where k is the knowledge point number, To assess the level of mastery of this knowledge point. This represents the difficulty level of the knowledge point, with a value ranging from 0.1 to 1.

0. This represents the time interval between the last review of this knowledge point; Based on the error type and the priority of filling in the gaps, corresponding resources are recommended. For errors with knowledge gaps, videos explaining the knowledge points, excerpts from the textbook, and documents clarifying concepts are recommended. For errors with methodological defects, tutorials on problem-solving techniques, detailed explanations of typical examples, and step-by-step guides are recommended. For errors with carelessness, training on question reading skills, checklists for detailed verification, and timed and standardized practice questions are recommended. 4.3) Variation training generation: Through feature extraction of incorrect questions, mapping of knowledge point association rules, and intelligent generation algorithms, variation questions that are corely related to the incorrect questions and have ability transfer value are generated, as shown in formula (15): (15)； in, , , As weight, , For knowledge point similarity, For question type structure similarity, For difficulty similarity, the values ​​for knowledge point similarity, question type structure similarity, and difficulty similarity range from 0 to 1.

6. A novel intelligent review system based on multimodal recognition and human-machine collaboration according to claim 1, characterized in that, The learning analysis module includes: 5.1) Calculation of knowledge point mastery: Based on multi-source data such as the distribution of incorrect questions, the accuracy rate of answering questions, and the effect of review, a refined knowledge point mastery assessment model is constructed, as shown in formula (16): (16)； Where k is the knowledge point number, This represents the total number of questions corresponding to this knowledge point. This indicates that the answer to question i is correct. This refers to the number of times this knowledge point has been reviewed. For review purposes, the weighting coefficients should be set between 0.2 and 0.

5. A dynamic update mechanism will be established to immediately recalculate mastery levels and update learning progress records after students complete new quizzes or review tasks; a time decay coefficient will be introduced for knowledge points that have not been reviewed for a long time. t is the number of days since the last effective review, and the corrected mastery level is shown in formula (17): (17)； 5.2) Class learning difference analysis: The degree of difference in the mastery of knowledge points within the class is assessed by combining the coefficient of variation with stratified statistics, as shown in formula (18): (18)； in, Standard deviation, This is the average value. Let represent the degree of mastery of knowledge point k by the i-th student. The larger the value, the more significant the differences in students' mastery of that knowledge point. Based on association rule algorithms and cluster analysis, common learning problems of the class as a whole are identified. The Apriori algorithm is used to calculate the support and confidence between knowledge points, identify high-frequency common error combinations, and determine the weak links between knowledge points. K-means clustering is performed on the error types of the class's wrong questions. If the cluster center of a certain type of error accounts for more than 40%, it is determined to be a common error of the class. Combined with the ability dimensions tested in the questions, the average score and coefficient of variation of each ability dimension of the class are calculated to identify the class's ability weaknesses. 5.3) Modeling the learning progress trend: Using exponential smoothing combined with a sliding window, the trajectory of student learning improvement is accurately depicted, as shown in formula (19): (19)； Where t is the time node. This represents the overall score for the t-th test. This is a smoothing coefficient, with a value range of 0.3-0.7; Let be the progress trend value at node t; By integrating the exponential smoothing method, the linear regression model, and the LSTM neural network model, a combined prediction model is constructed, as shown in formula (20): (20)； in, These are values ​​predicted using exponential smoothing. These are the predicted values ​​from the linear regression model. The predicted value is from the LSTM neural network. , As weight, ; The influence of each factor on the learning progress trend is quantified by using a multiple linear regression model, as shown in formula (21): (21)； in, Let t be the learning duration of stage t. To improve the accuracy of answers, To determine the frequency of review, For constant terms, , , For regression coefficients, This is random error.

7. A novel intelligent review system based on multimodal recognition and human-machine collaboration according to claim 1, characterized in that, The data security and sharing module includes: 6.1) Establish a data encryption storage mechanism. Use the AES-256 symmetric encryption algorithm to encrypt and store the answer data, evaluation result data, and structured data in the multi-dimensional learning archives. The encryption process is combined with salt value to enhance the anti-cracking ability, as shown in formula (22): (22)； in, This is the system master key. A randomly generated 16-byte salt value. The original data, For encrypted data; For the unstructured data in the answer data, evaluation results data, and multi-dimensional learning portfolios, a segmented encryption storage strategy is adopted. The segment size is determined by a dynamic adaptive algorithm, as shown in formula (23): (23)； in, The original file size is n; n is the number of partitions, ranging from 5 to 20; this number is dynamically adjusted based on the file size. 6.2) Secure transmission guarantee: The transmission channel is built based on the SSL / TLS 1.3 protocol, the session key is negotiated using the ECDHE key exchange algorithm, and data integrity is verified using the HMAC-SHA256 algorithm, as shown in formula (24): (24)； in, To transmit data packets, For session key, Fill the outer layer with the key. Fill the inner layer with keys; 6.3) Fine-grained access control: Based on the RBAC model, an access control matrix is ​​constructed, and sensitive information is hierarchically anonymized using a data anonymization algorithm, as shown in formula (25): (25)； in, For visitor role level, Data sensitivity level, For access scenario coefficients; , , ; 6.4) Cross-platform data sharing and adaptation: Build standardized RESTful API interfaces and adopt data format conversion algorithms to achieve data compatibility, as shown in formula (26): (26)； in, For source system data fragments, For the target system data format, This is a format conversion function. This is standard format reference data. For data consistency matching functions, is the data length, and m is the total number of data segments.

8. A novel intelligent review method based on multimodal recognition and human-machine collaboration, characterized in that, This method is implemented by invoking the answer data acquisition module, multimodal recognition module, human-computer collaborative evaluation module, diagnosis and recommendation module, learning analysis module, and data security and sharing module of the novel intelligent evaluation system based on multimodal recognition and human-computer collaboration as described in any one of claims 1-7.

9. A non-transitory computer-readable medium storing instructions, characterized in that, When the instruction is executed by the processor, the novel intelligent review method based on multimodal recognition and human-machine collaboration as described in claim 8 is executed.

10. A computing device, characterized in that, It includes a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the novel intelligent review method based on multimodal recognition and human-machine collaboration as described in claim 8.

Images