An augmented reality (AR) glasses-based assisted learning method and server combining analysis and a transformer

Abstract

An auxiliary learning method and server based on AR glasses analysis combined with a transformer are applied to the education field, and the method comprises the following steps: obtaining a question text and an answer text in a test paper through AR glasses; vectorizing the question text to obtain a question vector; comparing the question vector with a knowledge vector in a preset knowledge base to obtain a target knowledge vector with the highest similarity to the question vector; determining target knowledge corresponding to the target knowledge vector; inputting the target knowledge and the answer text into a transformer model respectively to obtain a target knowledge semantic analysis result and an answer text semantic analysis result; and comparing the target knowledge semantic analysis result and the answer text semantic analysis result to determine whether the answer text is correct. The application has the effect of improving learning evaluation accuracy.

Description

Technical Field

[0001] This application relates to the field of education, and in particular to an assisted learning method and server based on AR glasses analysis combined with Transformer. Background Technology

[0002] With the continuous development of technology, learning support technologies and products are also constantly being improved and innovated. Among related technologies, learning support products, such as online course platforms, utilize information technology to provide a brand-new learning model and rich learning resources. By digitizing educational resources through the Internet, learners can study anytime, anywhere, breaking the time and space limitations of traditional education.

[0003] However, online course platforms also have certain limitations. For example, in assessing learners' learning outcomes, they often can only check multiple-choice and true / false questions, making it difficult to check open-ended questions. This is because the automated scoring systems of online course platforms are usually based on a pre-set answer database, which often only contains standard answers or a limited number of options. For open-ended questions, there may be multiple correct answers or ways to solve them, making it difficult to find a perfectly matching answer in the pre-set answer database, thus affecting the accuracy of the assessment. Summary of the Invention

[0004] This application provides an assisted learning method and server based on AR glasses analysis combined with Transformer, which is used to improve the accuracy of learning evaluation.

[0005] Firstly, this application provides an assisted learning method based on AR glasses analysis combined with Transformer, the method including: acquiring the question text and answer text from the test paper through AR glasses; vectorizing the question text to obtain question vectors; comparing the similarity between the question vectors and knowledge vectors in a preset knowledge base to obtain the target knowledge vector with the highest similarity to the question vectors; determining the target knowledge corresponding to the target knowledge vector; inputting the target knowledge and answer text into the Transformer model respectively to obtain the semantic analysis results of the target knowledge and the semantic analysis results of the answer text; comparing the semantic analysis results of the target knowledge and the semantic analysis results of the answer text to determine whether the answer text is correct.

[0006] In the above embodiment, by extracting the question text and student's answer text from the test paper, the question text is vectorized, converting the text content into a numerical representation to obtain a question vector. The question vector is then compared with the knowledge vectors in a preset knowledge base to find the target knowledge vector with the highest similarity, thus determining the target knowledge most relevant to the question. The target knowledge and answer text are then input into a transformer model for semantic analysis. Through model processing, semantic analysis results for both the target knowledge and the answer text are obtained. The semantic analysis results for the target knowledge and the answer text are compared to determine the correctness of the answer text. If the two match closely, the answer text is considered correct; otherwise, an error may exist. This facilitates semantic analysis and correctness judgment of the answer text, improving the accuracy of learning assessment. Furthermore, through AR glasses, students can accurately see the location of the questions and answers, improving learners' understanding of the questions and the accuracy of their answers, and providing students with a more interactive and engaging learning method.

[0007] In conjunction with some embodiments of the first aspect, in some embodiments, before the step of obtaining the question text and answer text in the test paper through AR glasses, the method further includes: determining a paper area and a question area and an answer area in the paper area; generating an AR test paper according to the user's selection, wherein the AR test paper is a virtual test paper generated using augmented display technology, including questions extracted from a preset question bank; and displaying the AR test paper in the paper area and the question area and answer area in the paper area through the display screen of the AR glasses.

[0008] In the above embodiments, by defining the paper area and the question and answer areas within it, students no longer need to carry numerous paper exam papers. They can simply wear AR glasses to answer questions anytime, anywhere, seeing virtual questions and answer areas on virtual paper. This enhances the realism and immersion of the exam, providing a more concrete visual experience. The generation and display of the exam paper can be adjusted and modified according to needs, offering more flexible usage. The AR exam paper can connect to the system to achieve real-time feedback and automatic evaluation. Students can instantly receive their answers and evaluations, receiving personalized learning guidance and suggestions.

[0009] In conjunction with some embodiments of the first aspect, in some embodiments, before the step of obtaining the question text and answer text in the test paper through AR glasses, the method further includes: inputting the knowledge text in the preset knowledge base into the preset word embedding model to obtain the knowledge word vector; calculating the average value of each knowledge word vector to obtain the knowledge vector in the preset knowledge base.

[0010] In the above embodiments, by inputting the knowledge text into a preset word embedding model, each knowledge word can be converted into a vector representation. The preset word embedding model can capture the semantic relationships and contextual information between words, ensuring that each word has a corresponding vector representation. By calculating the average value of the knowledge word vectors, multiple knowledge word vectors in the preset knowledge base can be combined into a single knowledge vector. This knowledge vector can be considered as the overall representation of the knowledge base, representing its main features and semantic information. When comparing the similarity between the question vector and the knowledge vectors in the preset knowledge base, using the knowledge vectors in the preset knowledge base as a reference, the target knowledge most relevant to the question can be found. This helps improve the accuracy of matching and ensures that the selected target knowledge is highly relevant to the question.

[0011] In conjunction with some embodiments of the first aspect, in some embodiments, the question text is vectorized to obtain a question vector, specifically including: inputting the question text into a preset word embedding model to obtain question word vectors; calculating the average value of each question word vector to obtain the question vector.

[0012] In the above embodiments, during the question vectorization process, the preset word embedding model can capture the semantic features of each word in the question text, enabling the question vectors to express the semantic similarities and differences between the questions. Calculating the average of the question word vectors can combine multiple word vectors in the question text into a single vector. This dimensionality reduction and synthesis operation can reduce the dimensionality of the vectors and simultaneously integrate multiple semantic features of the question into a holistic feature, better representing the semantic meaning of the question. Through question vectorization, the question vectors can be compared with other vectors to find the target knowledge most relevant to the question, improving the accuracy of matching.

[0013] In conjunction with some embodiments of the first aspect, in some embodiments, after comparing the semantic analysis results of the target knowledge with the semantic analysis results of the answer text to determine whether the answer text is correct, the method further includes: if the answer text is correct, comparing the semantic analysis results of the answer text of the next question with the semantic analysis results of the target knowledge; if the answer text is incorrect, marking the corresponding question as an incorrect question.

[0014] In the above embodiments, after determining whether the answer to the current question is correct, the system continues to evaluate whether the answer to the next question is consistent with or related to the target knowledge. This allows for real-time monitoring of students' answers and provides more comprehensive feedback. If an answer is deemed incorrect, the corresponding question is marked as wrong, which helps record student errors and facilitates targeted follow-up teaching and tutoring. By combining error marking with continuous assessment, more personalized and precise learning guidance and feedback can be provided, helping students understand their mistakes and weaknesses and enabling targeted knowledge supplementation and reinforcement exercises.

[0015] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of comparing the semantic analysis results of the answer text of the next question with the semantic analysis results of the target knowledge, the method further includes: after determining that the semantic analysis results of all questions in the test paper have been compared, obtaining modification suggestions for the answer text based on the semantic analysis results of the target knowledge of the questions; evaluating all questions in the test paper and the students' answers to obtain the difficulty evaluation of the test paper and the level evaluation of the students; and displaying the modification suggestions, difficulty evaluation, and level evaluation through the display screen of the AR glasses.

[0016] In the above embodiments, by providing suggestions for modifying the answer text, targeted assistance is provided to help students correct errors and strengthen their knowledge application, thus offering personalized learning guidance and helping students better understand and master the target knowledge. Through the evaluation of the test paper and the students, an assessment of the test paper's difficulty and the students' level can be obtained, helping to understand the overall learning status of students and the quality of the test paper, providing a reference for subsequent teaching and improvement. Through the display screen of the AR glasses, the modification suggestions, difficulty assessment, and level assessment can be displayed to the students in real time. This allows students to receive feedback and guidance directly during the problem-solving process, providing a more flexible and convenient learning experience.

[0017] In conjunction with some embodiments of the first aspect, in some embodiments, after marking the corresponding question as an incorrect question when the answer text is incorrect, the method further includes: stemming or restoring the word form of the incorrect question text to obtain a standardized incorrect question text; determining an incorrect question vocabulary list based on the standardized incorrect question text; and inputting the incorrect question vocabulary list into the transformer model to obtain questions of the same type as the incorrect question.

[0018] In the above embodiments, the incorrect question text is standardized through operations such as stemming or word form restoration to reduce the impact of vocabulary changes and extract the core meaning of the incorrect questions. An embedded vector of the incorrect questions is generated using a preset word embedding model, transforming the incorrect questions into semantic representations for easier subsequent semantic analysis and processing. Utilizing the semantic understanding and generation capabilities of the Transformer model, questions of the same type as the incorrect questions are generated to help students review and consolidate their knowledge in a targeted manner.

[0019] Secondly, embodiments of this application provide a server, which includes: an acquisition module for acquiring question text and answer text from an exam paper via AR glasses; a vectorization module for vectorizing the question text to obtain question vectors; a comparison module for comparing the question vectors with knowledge vectors in a preset knowledge base to obtain a target knowledge vector with the highest similarity to the question vectors; a first determination module for determining the target knowledge corresponding to the target knowledge vector; a second determination module for inputting the target knowledge and answer text into a transformer model respectively to obtain semantic analysis results of the target knowledge and semantic analysis results of the answer text; and a third determination module for comparing the semantic analysis results of the target knowledge and the semantic analysis results of the answer text to determine whether the answer text is correct.

[0020] Thirdly, embodiments of this application provide a server comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the server to perform the method described in the first aspect and any possible implementation thereof.

[0021] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a server, cause the server to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fifthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a server, cause the server to perform the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the servers provided in the second and third aspects, the computer program product provided in the fourth aspect, and the computer storage medium provided in the fifth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] 1. Thanks to AR glasses analytics, learners can directly view virtual test questions on the exam paper using augmented reality, achieving a virtual learning experience within a real-world environment. In traditional education, students typically need to read questions and write answers on paper exam papers, which can lead to misunderstandings and operational difficulties. However, with AR glasses analytics, students can accurately see the location of the questions and answers, improving their comprehension and accuracy. This technological innovation not only enhances learning outcomes but also provides students with a more interactive and engaging learning method.

[0026] 2. By applying the Transformer model to the field of learning assistance, this invention can conduct in-depth semantic analysis of target knowledge and response text, accurately assess students' mastery of knowledge and development of thinking ability, and effectively solve the problem that traditional learning assessment methods often only focus on the correctness of answers while neglecting the cultivation of students' understanding and application of knowledge. Thus, this personalized assessment method provides students with targeted learning suggestions to help them better understand and master knowledge.

[0027] 2. By employing a method that combines a pre-set knowledge base and similarity comparison, the question vector is compared with the knowledge vector in the pre-set knowledge base to accurately locate the target knowledge. This effectively solves the problem in traditional learning where students may face a large amount of knowledge points and information, making it difficult to determine the focus and core content of their learning. In this way, it helps students focus on learning key knowledge points, thereby improving learning efficiency and learning outcomes. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating an assisted learning method based on AR glasses analysis combined with Transformer in an embodiment of this application.

[0029] Figure 2 This is another flowchart illustrating the assisted learning method based on AR glasses analysis combined with Transformer in the embodiments of this application;

[0030] Figure 3 This is a schematic diagram of a functional module structure of the server in an embodiment of this application;

[0031] Figure 4 This is a schematic diagram of the physical device structure of a server in an embodiment of this application. Detailed Implementation

[0032] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0033] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0034] To facilitate understanding, the application scenarios of this application's embodiments are described below. In the education field, an increasing number of students are choosing online learning platforms. However, existing online learning platforms have some problems, such as students often lacking personalized learning guidance and feedback, leading to poor learning outcomes. Furthermore, students may have varying levels of understanding of knowledge points during the learning process, and the platform cannot provide accurate assessment and assistance.

[0035] To address this issue, we propose an assisted learning method based on AR glasses analysis combined with Transformer. This method utilizes AR glasses technology to present virtual exam papers to students in an augmented display manner, and combines this with semantic analysis using a Transformer model, aiming to provide more accurate and personalized learning assistance.

[0036] AR glasses (Augmented Reality Glasses) are smart glasses devices that combine augmented reality technology with the functionality of glasses. Augmented reality is a technology that blends virtual information with the real world, providing users with a rich interactive and information display experience by overlaying virtual images, text, videos, or other digital content onto their field of vision.

[0037] The Transformer model is a neural network architecture based on a self-attention mechanism, used to process sequential data.

[0038] In related technologies, traditional online learning platforms rely on computers or mobile devices for learning, primarily depending on the course content and online quizzes provided by the platform. However, this approach has several drawbacks. First, platforms often fail to accurately assess students' understanding of knowledge points due to a lack of automated semantic analysis and personalized learning support. Second, students may lack timely feedback and guidance, hindering timely correction of errors and improvement of learning outcomes.

[0039] The AR glasses-based learning method combined with Transformer analysis, as described in this embodiment, involves students wearing AR glasses to view virtual exam questions through augmented reality. The AR glasses acquire exam images via a built-in camera and transmit them to a server for processing. The server uses a Transformer model to perform semantic analysis on the question text and student responses, comparing them with knowledge vectors in a pre-defined knowledge base to determine the target knowledge and assess the correctness of the responses.

[0040] This solution allows students to receive real-time semantic analysis results of their answers and compare them with the target knowledge. This enables students to promptly identify their errors and shortcomings and receive corresponding suggestions and guidance for improvement. Simultaneously, the server can provide personalized learning recommendations and assistance based on students' learning progress and semantic analysis results. This personalized learning support will help students better understand and master knowledge points, improving learning outcomes.

[0041] To facilitate understanding, the method provided in this implementation will be described in detail below, using the above scenario as an example. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating an assisted learning method based on AR glasses analysis combined with Transformer in an embodiment of this application.

[0042] S101. Obtain the question text and answer text from the exam paper using AR glasses;

[0043] First, the AR glasses' camera captures images of the virtual exam paper. Then, computer vision and optical character recognition (OCR) technologies are used to extract text information from the images. The OCR algorithm can recognize the question text and answer text on the exam paper and convert them into processable text data.

[0044] After obtaining the question text and answer text from optical character recognition, the text is processed and analyzed, including cleaning, standardizing and segmenting the text to extract the specific content of each question and the corresponding answer text. Natural language processing technology is used to process and parse the text data for subsequent operations and analysis.

[0045] S102. Vectorize the question text to obtain the question vector;

[0046] Each word in the question text is segmented or cut to obtain a word sequence. For each word, a selected word embedding model is used to convert it into a corresponding word vector. The word vectors of all words are averaged or weighted to obtain the vector representation of the question text, i.e., the question vector.

[0047] Optionally, in general, the question text can be vectorized to obtain the question vector in the following way:

[0048] Input the question text into the preset word embedding model to obtain the question word vector; calculate the average value of each question word vector to obtain the question vector.

[0049] First, determine a suitable word embedding model, such as Word2Vec, GloVe, or BERT. Word2Vec, GloVe, and BERT are all commonly used word vector representation methods or models in the field of Natural Language Processing (NLP).

[0050] Word2Vec (Word to Vector) is a neural network-based word vector representation model that learns distributed representations of words by training a shallow neural network.

[0051] GloVe (Global Vectors for Word Representation) is a word vector representation model based on global lexical statistics. GloVe constructs a global lexical co-occurrence matrix by analyzing the statistical information of word co-occurrence in context. Then, by performing matrix factorization on this matrix, it obtains the word vector representation for each word. The GloVe model can capture the semantic relationships and co-occurrence information between words.

[0052] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on the Transformer model. It learns rich semantic and syntactic representations through self-supervised learning on large-scale unlabeled text data. Unlike traditional left-to-right or right-to-left language models, BERT employs a bidirectional encoder and leverages bidirectional contextual information. During pre-training, BERT learns context-sensitive word vector representations through tasks such as padding, masking, and next-sentence prediction. In many NLP tasks, BERT's pre-trained model can achieve excellent performance through transfer learning with fine-tuning.

[0053] These models can map words to fixed-length vector representations; then, each word in the question text is segmented or cut to obtain a word sequence; for each word, a selected word embedding model is used to convert it into a corresponding word vector; the word vectors of all words are averaged or weighted to obtain the vector representation of the question text, i.e., the question vector.

[0054] S103. Compare the similarity between the question vector and the knowledge vectors in the preset knowledge base to obtain the target knowledge vector with the highest similarity to the question vector.

[0055] Using appropriate similarity metrics, such as cosine similarity, Euclidean distance, Manhattan distance, etc., the question vector is compared with each knowledge vector in the knowledge base, and the similarity score between the question vector and each knowledge vector is calculated.

[0056] The target knowledge vector with the highest similarity to the question vector is obtained by sorting the similarity scores.

[0057] S104. Determine the target knowledge corresponding to the target knowledge vector;

[0058] Select the target knowledge vector with the highest similarity score from the sorting results as the knowledge point most similar to the question vector.

[0059] S105. Determine the target knowledge corresponding to the target knowledge vector;

[0060] Select the target knowledge vector with the highest similarity score from the sorting results as the knowledge point most similar to the question vector.

[0061] S106. Compare the semantic analysis results of the target knowledge with the semantic analysis results of the answer text to determine whether the answer text is correct.

[0062] Compare the main idea or core content of the semantic analysis results of the target knowledge and the response text to determine whether they are consistent conceptually and logically. If they are consistent, the response text is determined to be correct.

[0063] In the above embodiment, by introducing an auxiliary learning method that combines AR glasses analysis and Transformer, the acquisition, vectorization, similarity comparison, and semantic analysis of question text and answer text are realized to determine the student's answer results, thereby improving the evaluation accuracy of auxiliary learning. In practical applications, this method can be further optimized.

[0064] The following provides supplementary information regarding the scenario in this embodiment.

[0065] Student Xiaoming is using an AR glasses-based assisted learning system to practice math. He puts on the AR glasses, and the problems are presented in an augmented display. Xiaoming is solving a complex math problem, but he makes a mistake.

[0066] On the AR glasses' display, Xiaoming's answer was marked as incorrect, and the explanation of the correct answer was shown. By observing the explanation, Xiaoming discovered his mistake in the problem-solving process. He noticed that he had missed a crucial intermediate step in his calculations, leading to the incorrect result.

[0067] The server displayed hints about the problem on the AR glasses' screen, pointing out the step Xiaoming might have made in solving the problem and providing relevant tips and methods. Xiaoming reconsidered his approach based on the hints and tried again on the AR glasses.

[0068] This time, Xiaoming carefully followed the instructions to solve the problem. He completed the calculation using AR glasses and arrived at the correct answer. The display on the AR glasses again marked his answer as correct.

[0069] With the help of AR glasses, Xiaoming was able to promptly identify and correct his mistakes while solving problems. The AR glasses provided precise error markers and analytical hints, helping Xiaoming understand the reasons for his errors and guiding him to rethink his problem-solving process. This instant feedback and personalized assistance enabled Xiaoming to better understand and master mathematical knowledge, improving his problem-solving abilities.

[0070] In addition to correcting this error, the AR glasses also recorded Xiaoming's problem-solving process and related data. This data can be transmitted to the server for analysis to generate learning reports and personalized learning recommendations. By analyzing Xiaoming's problem-solving data, the server can understand his strengths and weaknesses in math learning and provide him with targeted learning resources and suggestions to further improve his learning outcomes.

[0071] In light of the above scenarios, the method provided in this implementation will now be described in more detail. Please refer to [link / reference]. Figure 2 This is another flowchart illustrating the assisted learning method based on AR glasses analysis combined with Transformer in the embodiments of this application.

[0072] S201. Determine the paper area and the question area and answer area within that paper area;

[0073] AR glasses use built-in cameras or external sensors to acquire real-time images of the scene. By applying visual tracking algorithms, such as Simultaneous Localization and Mapping (SLAM) or feature-based tracking, the camera's position and pose relative to the environment are tracked.

[0074] In real-time images, computer vision methods are used to detect paper. This can be achieved by using techniques such as edge detection, color filtering, and shape matching to find the location and boundaries of the paper in the image.

[0075] Once the paper is detected, a pose estimation algorithm can be used to determine the paper's rotation and tilt angles. Based on the paper's position and pose information, computer vision methods are used to locate the question and answer regions, finding their positions and boundaries in the image.

[0076] S202. Generate AR test papers based on user selections;

[0077] This AR test paper is a virtual test paper generated using augmented reality technology, which includes questions extracted from a preset question bank.

[0078] Based on the user's selection, the test questions can be either real or virtual, and the types of questions include multiple choice, fill-in-the-blank, true / false, etc.

[0079] Based on the selected question type and number, the content of the AR test paper is generated using the corresponding algorithm or template.

[0080] S203. Display the AR test paper on the paper area and the question area and answer area in the paper area through the display screen of the AR glasses;

[0081] AR development tools and frameworks are used to display the generated AR test paper content on the AR glasses' screen, and the AR test paper is positioned according to the location and boundaries of the question area and answer area to ensure that it is aligned with the paper area and displayed correctly.

[0082] S204. Input the knowledge text from the preset knowledge base into the preset word embedding model to obtain knowledge word vectors;

[0083] First, the knowledge text in the pre-defined knowledge base is preprocessed, including text cleaning, word segmentation, and stop word removal. Then, a vocabulary is constructed based on the preprocessed text data. The vocabulary is a collection of unique words from all the text data, ensuring that each word is mapped to a unique index. Next, a pre-defined word embedding model (such as Word2Vec, GloVe, etc.) is used to train the model, generating word vectors for each word. During training, the model learns the semantic and contextual relationships between words.

[0084] Using a trained word embedding model, the corresponding word vector for each word in a pre-defined knowledge base is obtained by querying the vocabulary.

[0085] S205. Calculate the average value of each knowledge word vector to obtain the knowledge vector in the preset knowledge base;

[0086] Vector operations can be used, such as array operations using the NumPy library, to sum all the knowledge word vectors and divide the result by the total number of knowledge words to obtain the average value of the knowledge word vectors. The average value of the knowledge word vectors is the knowledge vector in the preset knowledge base.

[0087] NumPy (Numerical Python) is the foundational library for scientific computing in the Python language. It provides high-performance multidimensional array objects (ndarrays) and functions for manipulating arrays, as well as tools for performing mathematical, logical, statistical, and linear algebraic operations on arrays.

[0088] S206. If the answer text is correct, compare the semantic analysis results of the answer text of the next question with the semantic analysis results of the target knowledge.

[0089] Using natural language processing techniques and related tools and libraries, such as word embedding models, text classification algorithms, and semantic similarity calculations, semantic analysis is performed on the answer text and target knowledge for the next question.

[0090] Semantic analysis yields the semantic representations of the answer text and target knowledge for the next question. These semantic representations can be word vectors, sentence vectors, or other forms of representation.

[0091] Similarity calculation methods are used to compare the semantic representation of the answer text for the next question with the semantic representation of the target knowledge. Commonly used similarity calculation methods include cosine similarity, Euclidean distance, and Manhattan distance.

[0092] By comparing the similarity results, we determine whether the answer to the next question is correct. If the similarity is high, it means that the answer is semantically consistent with the target knowledge, and the answer can be considered correct. Then, we compare the semantic analysis results of the answer text to the next question with the semantic analysis results of the target knowledge.

[0093] S207. If the answer is incorrect, mark the corresponding question as wrong.

[0094] By comparing the similarity results, we can determine whether the answer to the next question is correct. If the similarity is low, it may mean that the answer differs significantly from the target knowledge semantically, and the answer can be considered incorrect, so the corresponding question is marked as wrong.

[0095] S208. After comparing the semantic analysis results of all questions in the test paper, suggestions for revising the answer text are obtained based on the semantic analysis results of the target knowledge of the questions.

[0096] By calculating similarity, distance, or other semantic matching metrics, the semantic representation of the response text is compared with the semantic representation of the target knowledge to identify potential errors or inconsistencies in the response text.

[0097] Detected errors or inconsistencies are categorized to provide appropriate modification suggestions for different types of errors. For example, errors can include conceptual errors, logical errors, factual errors, etc. By categorizing errors, more precise suggestions can be provided for each error type.

[0098] Based on the error classification results, corresponding modification suggestions are provided for each error type. These suggestions include correcting the erroneous concept, providing correct arguments or evidence, and revising the erroneous logical reasoning. The modification suggestions can be customized according to specific error types and knowledge domains.

[0099] S209. Evaluate all questions in the test paper and the students' answers to obtain an assessment of the test paper's difficulty and the students' level.

[0100] Calculate the average score and score rate (the percentage of students who scored high on each question) for each question. A lower score rate indicates that the question was relatively difficult, while a higher score rate indicates that the question was relatively easy. Analyze the question types (multiple choice, fill-in-the-blank, problem-solving, etc.) and the difficulty level of the knowledge points involved to take into account the differences in difficulty between different types and knowledge points, and comprehensively evaluate the overall difficulty of the test paper.

[0101] Calculate the overall student average score and score rate. Higher average scores and score rates indicate a higher overall student level, while lower scores indicate a lower overall student level. Analyzing student scores across different question types and knowledge points helps determine students' mastery levels in different areas, thus providing a more accurate assessment of their abilities.

[0102] S210. Display the modification suggestion, the difficulty rating, and the skill level rating on the AR glasses' display screen;

[0103] Establish a connection with the AR glasses and display the modification suggestion, difficulty rating, and level rating on the AR glasses' display screen via remote control.

[0104] S211. Stem extraction or word form restoration of the incorrect question text to obtain a standardized incorrect question text;

[0105] Stemming is a process of converting words into their basic stem or root form, usually achieved by removing suffixes from words to obtain a shared stem.

[0106] For example, stemming can convert the verb "running" to "run"; and stemming can convert the noun "cats" to "cat".

[0107] Lemmatization is the process of restoring words to their original lexical form or root. Unlike stemming, lemmatization results in a true word form, taking into account the word's context and part of speech to ensure semantic accuracy.

[0108] For example, the word form reduction of the verb "running" can be reduced to "run"; the word form reduction of the noun "cats" can be reduced to "cat".

[0109] Standardized error texts are obtained by stemming and word form restoration of the error texts.

[0110] S212. Based on this standardized error text, determine the error vocabulary list;

[0111] First, the standardized incorrect question text is segmented into individual words; then, a vocabulary list of incorrect questions can be constructed based on the segmentation results.

[0112] S213. Input the list of incorrect questions into the transformer model to obtain questions of the same type as the incorrect questions.

[0113] The list of incorrectly answered questions is organized into a text sequence according to certain rules or order, using spaces or other delimiters to separate the words. This text sequence is then fed as input to the Transformer model, which runs the generation process. Specific generation methods and parameter settings will vary depending on the Transformer model used. A common approach is to use autoregressive methods, generating the next word from the given input text, progressively building a new sequence of questions.

[0114] In this embodiment, since the AR glasses record Xiaoming's problem-solving process and related data, this data can be transmitted to the server for analysis to generate learning reports and personalized learning recommendations, effectively solving the problem of targeted learning tutoring and further improving learning outcomes.

[0115] The server in this application embodiment is described below from a module perspective. Please refer to [link / reference]. Figure 3 This is a schematic diagram of a functional module structure of the server in an embodiment of this application.

[0116] The server includes: an acquisition module (301) for acquiring the question text and answer text in the test paper through AR glasses; a vectorization module (302) for vectorizing the question text to obtain a question vector; a comparison module (303) for comparing the question vector with the knowledge vector in the preset knowledge base to obtain the target knowledge vector with the highest similarity to the question vector; a first determination module (304) for determining the target knowledge corresponding to the target knowledge vector; a second determination module (305) for inputting the target knowledge and answer text into the transformer model respectively to obtain the semantic analysis results of the target knowledge and the semantic analysis results of the answer text; and a third determination module (306) for comparing the semantic analysis results of the target knowledge and the semantic analysis results of the answer text to determine whether the answer text is correct.

[0117] In some embodiments, the server further includes a test paper determination module, specifically used for:

[0118] Determine the paper area, as well as the question area and answer area within the paper area;

[0119] An AR test paper is generated based on the user's selection. The AR test paper is a virtual test paper generated using augmented reality technology, which includes questions extracted from a preset question bank.

[0120] The AR exam paper is displayed on the screen of the AR glasses, along with the question area and answer area within the paper area.

[0121] In some embodiments, the server further includes a knowledge vectorization module, specifically used for:

[0122] The knowledge text in the preset knowledge base is input into the preset word embedding model to obtain knowledge word vectors;

[0123] Calculate the average value of each knowledge word vector to obtain the knowledge vector in the preset knowledge base.

[0124] In some embodiments, the vectorization module is specifically used for:

[0125] Input the question text into a preset word embedding model to obtain the question word vectors;

[0126] Calculate the average of the word vectors for each question to obtain the question vector.

[0127] In some embodiments, the server further includes a processing module, specifically configured to:

[0128] If the answer text is correct, compare the semantic analysis results of the answer text of the next question with the semantic analysis results of the target knowledge.

[0129] If the answer is incorrect, the corresponding question will be marked as wrong.

[0130] In some embodiments, the processing module is specifically used for:

[0131] After comparing the semantic analysis results of all questions in the test paper, suggestions for revising the answer text are obtained based on the semantic analysis results of the target knowledge of the questions.

[0132] The entire test paper and the students' answers are evaluated to obtain an assessment of the test paper's difficulty and the students' level.

[0133] The AR glasses' display screen will show modification suggestions, difficulty rating, and skill level rating.

[0134] In some embodiments, the processing module is further configured to:

[0135] The incorrect question text is stemmed or word form restored to obtain a standardized incorrect question text;

[0136] Based on standardized error texts, a vocabulary list for incorrect answers was determined;

[0137] Input the vocabulary list of incorrect questions into the transformer model to obtain questions of the same type as the incorrect questions.

[0138] The server in the embodiments of this application has been described above from the perspective of modular functional entities. The server in the embodiments of this application will now be described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 4 This is a schematic diagram of the physical device structure of a server in an embodiment of this application.

[0139] It should be noted that, Figure 4 The server structure shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0140] like Figure 4 As shown, the server includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 402 or a program loaded from storage portion 408 into Random Access Memory (RAM) 403, such as performing the methods described in the above embodiments. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0141] The following components are connected to I / O interface 405: input section 406 including a camera, etc.; output section 407 including a liquid crystal display (LCD) and a monitor, etc.; storage section 408 including a hard disk, etc.; and communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 409 performs communication processing via a network such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 410 as needed so that computer programs read from them can be installed into storage section 408 as needed.

[0142] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the various functions defined in the present invention.

[0143] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. The transmitted data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] Specifically, the server in this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the assisted learning method based on AR glasses analysis combined with Transformer provided in the above embodiment.

[0146] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the server described in the above embodiments; or it may exist independently and not assembled into the server. The storage medium carries one or more computer programs that, when executed by a processor of the server, cause the server to implement the methods provided in the above embodiments.

[0147] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0148] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0149] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An assisted learning method based on AR glasses analyzing and combining with a Transformer, characterized in that, The method comprises: acquiring a question text and an answer text in a test paper through an AR glasses; vectorizing the question text to obtain a question vector; comparing the question vector with a knowledge vector in a preset knowledge base to obtain a target knowledge vector with the highest similarity to the question vector; determining a target knowledge corresponding to the target knowledge vector; inputting the target knowledge and the answer text into a transformer model respectively to obtain a target knowledge semantic analysis result and an answer text semantic analysis result; comparing the target knowledge semantic analysis result and the answer text semantic analysis result to determine whether the answer text is correct; Before the step of acquiring the question text and the answer text in the test paper through the AR glasses, the method further comprises: determining a paper region, a question region and an answer region in the paper region; generating an AR test paper according to a user's selection, the AR test paper being a virtual test paper generated by using an augmented display technology and comprising questions extracted from a preset question bank; and displaying the AR test paper on the paper region, the question region and the answer region in the paper region through a display screen of the AR glasses; Before the step of acquiring the question text and the answer text in the test paper through the AR glasses, the method further comprises: inputting knowledge texts in a preset knowledge base into a preset word embedding model to obtain knowledge word vectors; and calculating an average value of each knowledge word vector to obtain a knowledge vector in the preset knowledge base; The vectorizing the question text to obtain a question vector specifically comprises: inputting the question text into a preset word embedding model to obtain question word vectors; and calculating an average value of each question word vector to obtain a question vector; After the comparison of the semantic analysis results of all questions in the test paper is completed, a modification suggestion for the answer text is obtained according to the target knowledge semantic analysis result of the question; the questions in the test paper and the answer of the student are evaluated to obtain a difficulty evaluation of the test paper and a level evaluation of the student; and the modification suggestion, the difficulty evaluation and the level evaluation are displayed through the display screen of the AR glasses.

2. The method of claim 1, wherein, After the step of comparing the target knowledge semantic analysis result and the answer text semantic analysis result to determine whether the answer text is correct, the method further comprises: in the case that the answer text is correct, comparing an answer text semantic analysis result of a next question with a target knowledge semantic analysis result; and in the case that the answer text is incorrect, marking the corresponding question as a wrong question.

3. The method of claim 2, wherein, After the step of marking the corresponding question as a wrong question in the case that the answer text is incorrect, the method further comprises: performing stem extraction or lemmatization on the wrong question text to obtain a standardized wrong question text; determining a wrong question vocabulary table according to the standardized wrong question text; and inputting the wrong question vocabulary table into a transformer model to obtain wrong questions of the same type.

4. A server, characterized by Comprise: one or more processors and a memory; the memory coupled with the one or more processors, the memory configured to store computer program code including computer instructions that, when executed by the one or more processors, cause the server to perform the method of any one of claims 1-3.

5. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the server, cause the server to perform the method of any one of claims 1-3.

Images