An AR-based intelligent lesson preparation method and system, and electronic devices.

Abstract

This invention relates to the field of intelligent teaching technology, specifically to an AR-based intelligent lesson preparation method, system, and electronic device. It involves acquiring the current frame image of target material captured by an AR camera on a user-worn AR device; performing OCR recognition on the current frame image to extract the words with the highest frequency of occurrence in the current frame image (ranked first by a preset value); and obtaining the words with the highest frequency of occurrence in the target courseware found by the user (ranked second by a preset value), recording them as the target vocabulary set. This is then used to train an intelligent indexing model for the courseware. In subsequent lesson preparation, this intelligent indexing model is used to intelligently index the courseware, and the courseware output by the intelligent indexing model is displayed on the user's AR device. This achieves the goal of optimizing query results based on the user's historical query behavior or the correlation between data when the user attempts to extract or query specific courseware data, significantly improving the efficiency and intelligence of the lesson preparation process.

Description

Technical Field

[0001] This invention relates to the field of intelligent teaching technology, specifically to an AR-based intelligent lesson preparation method and system, and electronic equipment. Background Technology

[0002] With the rapid growth of digital information, data integration and management has become a major challenge in the field of modern information technology. Traditional data storage and retrieval systems face many limitations when processing large-scale and diverse courseware data.

[0003] In the field of intelligent teaching, most existing systems lack effective tools to analyze and mine the potential correlations between target keywords and courseware data. They typically only offer simple keyword-based searches, failing to retrieve information based on deeper connections and patterns between data, thus limiting users' ability to quickly discover target courseware from a large amount of data. When users attempt to extract or query specific courseware data, existing systems often cannot optimize search results based on users' historical query behavior or the correlations between data. This results in users being unable to quickly obtain relevant and potentially more valuable courseware, leading to a heavy workload and low efficiency in lesson preparation.

[0004] Therefore, the existing technology still needs further development. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide an AR-based intelligent lesson preparation method, system, and electronic device to solve the problems existing in the prior art.

[0006] To achieve the above-mentioned technical objectives, according to a first aspect of the present invention, the present invention provides an AR-based intelligent lesson preparation method, comprising:

[0007] S100. Receive courseware data from data sources of different types and formats to construct a courseware database; during the current lesson preparation process, acquire the current frame image of the target material captured by the AR camera on the user's head-mounted AR device; perform OCR recognition on the current frame image, extract the words with the highest frequency of occurrence in the current frame image as target keywords, sort the target keywords in the current frame image according to their frequency of occurrence, and assign preset weights to each target keyword according to a preset weight list; calculate the frequency of occurrence of target keywords for each courseware in the courseware database, and determine the target keywords based on the frequency of occurrence of each target keyword and the target keywords in the courseware database. The system assigns preset weights to keywords and scores each courseware in the courseware database. The courseware with the highest score is displayed on the user's head-mounted AR device for human-computer interaction. The user confirms whether the target courseware has been found. If so, the system retrieves the words that appear most frequently in the target courseware, ranked from largest to smallest, with the second-to-last preset ranking as the target word set. The system also records the target keywords and target word set extracted from the current frame image during the lesson preparation process as the courseware indexing process data set. A preset number of courseware indexing process data sets are collected and divided into training datasets and validation datasets according to a first preset ratio.

[0008] S200. Train the courseware intelligent index model using the collected training dataset, optimize the courseware intelligent index model using the validation dataset, optimize the model parameters, and complete the training of the courseware intelligent index model.

[0009] S300. After the model training is completed, during the next lesson preparation process, the current frame image of the target material captured by the AR camera on the user's head-mounted AR device is obtained, OCR recognition is performed on the current frame image, the target keywords in the current frame image are extracted, the extracted target keywords are input into the courseware intelligent index model, and the courseware output by the courseware intelligent index model is displayed on the user's head-mounted AR device.

[0010] Specifically, the target data includes at least one of the following:

[0011] The content of the teaching materials to be used in this lesson preparation, the content of the teaching guidance documents to be used in this lesson preparation, and the content of the reference books to be used in this lesson preparation.

[0012] Specifically, the human-computer interaction with the user to confirm whether the target courseware has been found includes:

[0013] The system outputs a prompt signal to the user indicating whether the target courseware has been found, and obtains feedback signals input by the user. The user-input feedback signals include feedback signals confirming that the target courseware has been found and feedback signals indicating that the target courseware has not been found. The system confirms whether the target courseware has been found based on the user-input feedback signals.

[0014] Specifically, the human-computer interaction with the user to confirm whether the target courseware has been found also includes:

[0015] If the target courseware is not found, the current frame image of the target data captured by the AR camera on the user's head-mounted AR device is retrieved again, and process S100 is executed again.

[0016] Specifically, the process of training a courseware intelligent indexing model using the collected training dataset, optimizing the model using the validation dataset, and completing the training of the courseware intelligent indexing model includes:

[0017] The training dataset is input into a preset network layer in batches for training. The preset network layer includes a Transformer network layer, which is used to predict the data set of the next courseware indexing process based on the data set of the current courseware indexing process through forward propagation, thereby obtaining the predicted loss value. The loss value of the preset network layer is calculated and input into the optimizer for optimization, determining the direction of the fastest gradient descent of the courseware intelligent indexing model's parameters. The courseware intelligent indexing model then performs backpropagation based on the loss value and the model's parameter gradient to optimize the parameters of the courseware intelligent indexing model.

[0018] Specifically, the process of training a courseware intelligent indexing model using the collected training dataset, optimizing the model using the validation dataset, and completing the training of the courseware intelligent indexing model includes:

[0019] After each training iteration, the validation dataset is input into the preset network layer of the previous training in batches to validate the model parameters. The training is repeated in cycles, and the total number of training rounds is set to the first preset round. The loss value of the preset network layer is recorded, and it is determined whether the loss value meets the first preset condition. Based on the determination result, it is determined whether to end the training and output the model parameters.

[0020] Specifically, determining whether the loss value meets the first preset condition, and determining whether to end training and output model parameters based on the determination result, includes:

[0021] If the loss value meets the first preset condition, the training ends and the parameters of the current courseware intelligent indexing model are output; if the loss value does not meet the first preset condition, the training continues.

[0022] The first preset conditions include:

[0023] After the second preset training round, the loss values ​​obtained in the next training round are all greater than or equal to the loss values ​​that occurred during the second preset training round.

[0024] Specifically, the scoring of each courseware in the courseware database is based on the frequency of occurrence of each target keyword and the preset weight of each target keyword, including:

[0025] Select a courseware and choose a target keyword from it. Calculate the product of the preset weight of the target keyword and the number of times it appears in the courseware. Record this as the first score of the target keyword. Calculate the sum of the first scores of all target keywords in the courseware and record this as the score of the courseware.

[0026] According to a second aspect of the present invention, an AR-based intelligent lesson preparation system is provided, comprising:

[0027] The acquisition module is used to receive courseware data from data sources of different types and formats and build a courseware database; it is also used to acquire the current frame image of the target material captured by the AR camera on the user's head-mounted AR device during the current lesson preparation process.

[0028] The control module performs OCR recognition on the current frame image, extracts the words with the highest frequency of occurrence in the current frame image, records them as target keywords, sorts the target keywords in the current frame image according to their frequency of occurrence, and assigns preset weights to each target keyword according to a preset weight list. It then calculates the frequency of occurrence of target keywords for each courseware in the courseware database. Based on the frequency of occurrence and preset weights of each target keyword, it scores each courseware in the database and displays the highest-scoring courseware on the user's head-mounted AR device for human-computer interaction. The user confirms whether the target courseware has been found. If so, it retrieves the words in the target courseware that are ranked from highest to lowest frequency and have the highest and second-highest preset ranking, recording them as the target word set. This set is then used to extract the target keywords from the current frame image during the lesson preparation process. The target keywords and target vocabulary set are denoted as the courseware indexing process data set. A preset number of courseware indexing process data sets are collected, and the collected courseware indexing process data sets are divided into training datasets and validation datasets according to a first preset ratio. The training dataset is used to train the courseware intelligent indexing model, and the validation dataset is used to optimize the courseware intelligent indexing model, optimize the model parameters, and complete the training of the courseware intelligent indexing model. After the model training is completed, in the next lesson preparation process, the current frame image of the target material taken by the AR camera on the user's head-mounted AR device is obtained, the current frame image is subjected to OCR recognition, the target keywords in the current frame image are extracted, the extracted target keywords are input into the courseware intelligent indexing model, and the courseware output by the courseware intelligent indexing model is displayed on the user's head-mounted AR device.

[0029] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory; and a processor, wherein the memory stores computer-readable instructions, which, when executed by the processor, implement the above-described AR-based intelligent lesson preparation method.

[0030] Beneficial effects:

[0031] This invention, during lesson preparation, acquires the current frame image of the target material captured by the AR camera on the user's head-mounted AR device. It then performs OCR recognition on the current frame image, extracting the words with the highest frequency of occurrence in the current frame image (ranked first by a preset value). It also acquires the words from the target courseware found by the user, ranked from highest to lowest frequency (ranked second by a preset value), and records them as the target word set. The target keywords and target word set extracted during this lesson preparation process are recorded as the courseware indexing process data set. This data is then used to train a courseware intelligent indexing model. In subsequent lesson preparation processes, the intelligent indexing model is used to intelligently index the courseware, and the courseware output by the intelligent indexing model is displayed on the user's head-mounted AR device. This allows for the optimization of search results based on the user's historical search behavior or the correlation between data when a user attempts to extract or query specific courseware data. This enables the rapid retrieval of related and potentially more valuable courseware, significantly improving the efficiency, intelligence, and accuracy of the lesson preparation process, greatly reducing the workload of lesson preparation, and significantly enhancing the user experience. Attached Figure Description

[0032] Figure 1 This is a flowchart of an AR-based intelligent lesson preparation method provided in a specific embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the system composition of the AR-based intelligent lesson preparation system provided in a specific embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the structure of the intelligent indexing model for courseware provided in a specific embodiment of the present invention. Detailed Implementation

[0035] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments in this application, other similar embodiments obtained by those skilled in the art without creative effort should all fall within the scope of protection of this application. Furthermore, directional terms mentioned in the following embodiments, such as "up," "down," "left," and "right," are only for reference to the directions in the accompanying drawings; therefore, the directional terms used are for illustrative purposes and not for limiting the invention.

[0036] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0037] Please see Figure 1 This invention provides an AR-based intelligent lesson preparation method, comprising:

[0038] S100. Receive courseware data from data sources of different types and formats to construct a courseware database; during the current lesson preparation process, acquire the current frame image of the target material captured by the AR camera on the user's head-mounted AR device; perform OCR recognition on the current frame image, extract the words with the highest frequency of occurrence in the current frame image as target keywords, sort the target keywords in the current frame image according to their frequency of occurrence, and assign preset weights to each target keyword according to a preset weight list; calculate the frequency of occurrence of target keywords for each courseware in the courseware database, and determine the target keywords based on the frequency of occurrence of each target keyword and the target keywords in the courseware database. The system assigns preset weights to keywords and scores each courseware in the courseware database. The courseware with the highest score is displayed on the user's head-mounted AR device for human-computer interaction. The user confirms whether the target courseware has been found. If so, the system retrieves the words that appear most frequently in the target courseware, ranked from highest to lowest, with the second-highest preset ranking being recorded as the target word set. The target keywords and target word set extracted from the current frame image during the lesson preparation process are recorded as the courseware indexing process data set. A preset number of courseware indexing process data sets are collected and divided into training datasets and validation datasets according to a first preset ratio.

[0039] Specifically, the target data includes at least one of the following:

[0040] The content of the teaching materials to be used in this lesson preparation, the content of the teaching guidance documents to be used in this lesson preparation, and the content of the reference books to be used in this lesson preparation.

[0041] Specifically, the human-computer interaction with the user to confirm whether the target courseware has been found includes:

[0042] The system outputs a prompt signal to the user indicating whether the target courseware has been found, and obtains feedback signals input by the user. The user-input feedback signals include feedback signals confirming that the target courseware has been found and feedback signals indicating that the target courseware has not been found. The system confirms whether the target courseware has been found based on the user-input feedback signals.

[0043] Specifically, the human-computer interaction with the user to confirm whether the target courseware has been found also includes:

[0044] If the target courseware is not found, the current frame image of the target data captured by the AR camera on the user's head-mounted AR device is retrieved again, and process S100 is executed again.

[0045] It is understood that the above process can continuously update the dataset of the courseware intelligent indexing model of the present invention and ensure the reliability of the dataset. After the courseware intelligent indexing model of the present invention is trained and put into use, it can still be trained again using the continuously updated dataset, so as to continuously optimize the courseware intelligent indexing model of the present invention and continuously improve the efficiency and accuracy of the courseware intelligent indexing of the present invention.

[0046] It should be noted here that step S100 includes the following:

[0047] The control module pre-sets the preset quantity, first preset ratio, first preset round, second preset round, preset weight list, top first preset ranking, and top second preset ranking.

[0048] It is understood that the preset quantity, first preset ratio, first preset round, second preset round, preset weight list, top first preset ranking, and top second preset ranking can be specifically set according to the actual needs of the user according to the present invention. The present invention does not limit the specific values ​​of the above parameters, and there is no rule for setting the specific values ​​of the above parameters, as long as they are applicable to the courseware management intelligent indexing method proposed by the present invention.

[0049] Preferably, the present invention sets the preset quantity to 500, the first preset ratio to 8:2, the first preset rounds to 10,000, the second preset rounds to 200, the preset weight list to {100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 0}, the first preset ranking to the top 5, and the second preset ranking to the top 20. These settings further improve the model's performance and training efficiency, significantly enhancing the accuracy and reliability of the courseware intelligent indexing model's predictions.

[0050] It is understandable that when a user is preparing lessons, if the user finds the target courseware by the words that appear in the current frame image with the highest number of occurrences, there is a certain coupling relationship between the words that appear in the current frame image with the highest number of occurrences and the words that appear in the target courseware in descending order of frequency and the words that appear in the target courseware with the highest number of occurrences and the words that appear in the target courseware with the highest number of occurrences. That is, there is a certain coupling relationship between the current frame image of the target material captured by the AR camera on the user's head-mounted AR device and the content of the target courseware. Based on this, the present invention trains the courseware intelligent indexing model.

[0051] Specifically, the step of receiving courseware data from data sources of different types and formats and constructing a courseware database includes:

[0052] It can receive courseware data from data sources of different types and formats, support multiple data access protocols, and handle a large number of concurrent data streams.

[0053] Specifically, it supports multiple data access protocols, including HTTP / HTTPS, FTP, SFTP, WebDAV, and various database connection protocols such as JDBC and ODBC.

[0054] Specifically, by employing a concurrent processing mechanism to handle large volumes of concurrent data streams, the engine can optimize data transmission, reduce latency, and ensure the stability and efficiency of data access even under high load conditions.

[0055] Specifically, the process of receiving courseware data from data sources of different types and formats and constructing a courseware database includes:

[0056] Using OCR (Optical Character Recognition) technology, paper courseware is scanned to obtain electronic courseware data.

[0057] Specifically, the different types and formats of data sources include:

[0058] File data sources: including Excel data sources and CSV data sources, etc.

[0059] Database data sources: such as MySQL, MariaDB, Oracle, SQL Server, PostgreSQL, Greenplum, etc.

[0060] Big data engine data sources: such as Kylin, Cloudera Impala, Hive, MemSQL, Presto, MongoDB, etc.

[0061] Third-party API data sources: such as OData, Rest, and DataInsight.

[0062] S200. Train the courseware intelligent index model using the collected training dataset, optimize the courseware intelligent index model using the validation dataset, optimize the model parameters, and complete the training of the courseware intelligent index model.

[0063] Specifically, the process of training a courseware intelligent indexing model using the collected training dataset, optimizing the model using the validation dataset, and completing the training of the courseware intelligent indexing model includes:

[0064] The training dataset is input into a preset network layer in batches for training. The preset network layer includes a Transformer network layer, which is used to predict the data set of the next courseware indexing process based on the data set of the current courseware indexing process through forward propagation, thereby obtaining the predicted loss value. The loss value of the preset network layer is calculated and input into the optimizer for optimization, determining the direction of the fastest gradient descent of the courseware intelligent indexing model's parameters. The courseware intelligent indexing model then performs backpropagation based on the loss value and the model's parameter gradient to optimize the parameters of the courseware intelligent indexing model.

[0065] It is understandable that Transformer is a deep learning model based on a self-attention mechanism, consisting of a multi-head attention mechanism and a feedforward neural network. It can process sequential data and capture long-range dependencies. Transformer models do not contain traditional convolutional layers, but instead process each element in the input sequence through attention layers. A typical Transformer structure includes an encoder and a decoder, where the encoder is composed of multiple identical layers stacked together, and the decoder adds a self-attention layer on top of the encoder to handle sequence generation tasks. This invention utilizes Transformer to supplement global contextual information, enhances the model's ability in feature extraction and global understanding, greatly improves the model's performance, and significantly reduces computational costs.

[0066] Specifically, the preset network layer also includes a ResNet network layer, which is used to predict the output result based on the collected courseware indexing process data set, and the prediction result is a courseware image.

[0067] It's important to note that ResNet is a deep convolutional neural network that addresses the vanishing gradient problem during deep network training by introducing residual connections. The core component of ResNet is the residual block, which typically contains two or three convolutional layers and a skip connection. The skip connection allows gradients to bypass these convolutional layers directly. ResNet's network structure can be very deep. These networks extract features of target keywords and target vocabulary sets from the current frame image by stacking multiple residual blocks, and predict the output results through global average pooling and fully connected layers. The predicted results are the courseware images.

[0068] Please see Figure 3This invention combines ResNet and Transformer network layers. ResNet excels at extracting local features of target keywords and target vocabulary sets in the current frame image during the lesson preparation process, while Transformer can supplement global contextual information. This combination can enhance the model's ability in feature extraction and global understanding, greatly improving the model's performance and significantly reducing computational costs.

[0069] It is understood that the ResNet network layer is used to predict the output results using the courseware indexing process data set collected in the training dataset and validation dataset. The specific steps are as follows:

[0070] (1) First, the training dataset is put into the Backbone network layer of the model. Backbone refers to a series of convolutional layers that make up the backbone of the neural network. The main function of these layers is to extract the features of the input data. Backbone networks are usually composed of multiple convolutional layers, pooling layers and activation functions, which can extract meaningful feature representations from the original data. Backbone consists of multiple Backbones, and the core network layer in Backbone is the ResNet network layer. After the dataset is input into the model, the courseware indexing process data set will be input into Backbone for training.

[0071] (2) When constructing Backbone, a ResNet network layer was added. After the training dataset is input, ResNet starts training layer by layer to extract features from the courseware indexing process data set, which assists Transformer in predicting courseware.

[0072] Specifically, the process of training a courseware intelligent indexing model using the collected training dataset, optimizing the model using the validation dataset, and completing the training of the courseware intelligent indexing model includes:

[0073] After each training iteration, the validation dataset is input into the preset network layer of the previous training in batches to validate the model parameters. The training is repeated in cycles, and the total number of training rounds is set to the first preset round. The loss value of the preset network layer is recorded, and it is determined whether the loss value meets the first preset condition. Based on the determination result, it is determined whether to end the training and output the model parameters.

[0074] Specifically, after each training iteration, the validation dataset is input in batches into the preset network layers of the previous training iteration for model parameter validation, and the training is repeated cyclically, including the following methods:

[0075] When the number of training iterations is insufficient and model optimization is desired, the original approach was to start training from scratch. Therefore, this invention develops a method that allows for continued training based on a pre-trained model parameter file.

[0076] (1) Provides custom parameters, which can be set according to user needs to continue training based on any training parameters;

[0077] (2) Load the parameters of the custom model file;

[0078] (3) Put the validation dataset into the custom parameter network layer for validation, obtain the loss value, and record the model parameters of the model corresponding to the smallest loss value during training.

[0079] (4) Based on this set of model parameters, perform forward propagation training, back propagation, and optimizer parameter optimization to obtain a new set of data;

[0080] (5) The validation data is put into the new data network layer for inference to obtain the loss value;

[0081] (6) Perform iterative training to obtain the best model and complete the model optimization.

[0082] It is understood that the training method used in this invention is batch training, which means that each time the model parameters are updated, only a portion of the samples in the validation dataset is used, called a batch. The advantages of batch training are that it can reduce memory consumption, speed up training, increase randomness, and is beneficial to the generalization of the model.

[0083] It should be noted that in this invention, batch_size = 2, where batch_size is the batch size. For example, when batch_size = 2, it means that two courseware index process data sets are selected from the validation dataset each time and put into the model for validation.

[0084] Specifically, determining whether the loss value meets the first preset condition, and determining whether to end training and output model parameters based on the determination result, includes:

[0085] If the loss value meets the first preset condition, the training ends and the parameters of the current courseware intelligent indexing model are output; if the loss value does not meet the first preset condition, the training continues.

[0086] The first preset conditions include:

[0087] After the second preset training round, the loss values ​​obtained in the next training round are all greater than or equal to the loss values ​​that occurred during the second preset training round.

[0088] Specifically, the scoring of each courseware in the courseware database is based on the frequency of occurrence of each target keyword and the preset weight of each target keyword, including:

[0089] Select a courseware and choose a target keyword from it. Calculate the product of the preset weight of the target keyword and the number of times it appears in the courseware. Record this as the first score of the target keyword. Calculate the sum of the first scores of all target keywords in the courseware and record this as the score of the courseware.

[0090] It is understood that the maximum number of training iterations in this invention is preferably 10,000, which can effectively ensure the accuracy of the model's predictions. The loss value of the model training is recorded. The second preset number of iterations is set to 200. That is, when the training loss value no longer decreases within 200 training iterations, the iteration with the smallest loss value in those 200 training iterations is saved as the parameters of the best validation iteration, generating the optimal model file. The above settings allow the number of training iterations to be automatically trained using a deep learning model instead of relying on manual judgment, improving the accuracy of the model's predictions. Furthermore, training automatically ends when the required accuracy is achieved, greatly saving training time and effectively preventing overfitting. This significantly improves the intelligence level and model training efficiency of this invention.

[0091] Furthermore, the specific process of training the intelligent indexing model for courseware according to this invention is as follows:

[0092] (1) Obtain the latent vector mean mu and logvariance logvar during the forward propagation process of the intelligent indexing model of courseware;

[0093] (2) Calculate the reasonable loss value representing the training effect:

[0094] Calculate the divergence: k1 = -0.5*(1 + logvar - mu^2 - (e^logvar));

[0095] Calculate the loss value between the courseware index process data set in the validation dataset and the predicted values:

[0096] loss = L1 + k1 * k1_weight;

[0097] Where loss is the loss value, k1 is the divergence, k1_weight is the proportion of k1, with a value of 10, and L1 is the absolute difference between the predicted value and the true value. The formula for calculating L1 is as follows:

[0098]

[0099] Here, A is the L1 loss in PyTorch, which calculates the average or sum of the absolute differences between two tensors. A has a shape of N*C*H*W, indicating that it is a four-dimensional tensor. N represents the batch size, indicating that there are N data points. C represents the number of channels. H and W represent the height and width, respectively, which are usually the two-dimensional spatial dimensions of the input features. P represents the padding value, which is a Boolean value. P has a shape of c*h*w*1. The dimensions of P are the same as those of A in terms of channels, height, and width, but without the number of samples N. This means that P is a mask or padding information for a certain feature map for each sample. The Boolean value of P may be used to indicate whether the elements in A are padded (True) or valid (False).

[0100] (3) The loss value when using the validation dataset to validate the current model network;

[0101] (4) If the validation loss in this round is less than that in the previous round, the optimal model parameters are replaced with the training model in this round.

[0102] (5) Put the round number and loss value into a dictionary one by one;

[0103] (6) Compare the loss value of the current epoch with the loss values ​​of the previous 200 epochs (a parameter is provided, which can be set according to actual needs, with the default value being 200). That is, iterate through the loss values ​​from epoch-200 to epoch-1 (epoch is the training epoch, and comparison is made when epoch is greater than 200). If the loss value of the current epoch is greater than the loss values ​​from (epoch-200) to (epoch-1), then stop training.

[0104] (7) Save the optimal model parameters in the cyclic validation as the ckpt model file.

[0105] S300. After the model training is completed, during the next lesson preparation process, the current frame image of the target material captured by the AR camera on the user's head-mounted AR device is obtained, OCR recognition is performed on the current frame image, the target keywords in the current frame image are extracted, the extracted target keywords are input into the courseware intelligent index model, and the courseware output by the courseware intelligent index model is displayed on the user's head-mounted AR device.

[0106] Understandably, during lesson preparation, this invention acquires the current frame image of the target material captured by the AR camera on the user's head-mounted AR device; performs OCR recognition on the current frame image to extract the words with the highest frequency of occurrence in the current frame image, and obtains the words with the highest frequency of occurrence in the target courseware found by the user, ranked from largest to smallest, and ranked in the second-highest preset order, recording them as the target word set. The target keywords and target word set extracted from the current frame image during this lesson preparation process are recorded as the courseware indexing process data set, thereby training a courseware intelligent indexing model. In subsequent lesson preparation processes, this intelligent indexing model is used to intelligently index the courseware, and the courseware output by the intelligent indexing model is displayed on the user's head-mounted AR device. This enables the optimization of query results based on the user's historical query behavior or the correlation between data when the user attempts to extract or query specific courseware data, achieving rapid retrieval of related and potentially more valuable courseware. This significantly improves the efficiency, intelligence, and accuracy of the lesson preparation process, greatly reduces the workload of lesson preparation, and significantly enhances the user experience.

[0107] Please see Figure 2 The present invention provides another embodiment, which provides an AR-based intelligent lesson preparation system, the AR-based intelligent lesson preparation system comprising:

[0108] The acquisition module 100 is used to receive courseware data from data sources of different types and formats and build a courseware database; it is also used to acquire the current frame image of the target material captured by the AR camera on the user's head-mounted AR device during the current lesson preparation process.

[0109] The control module 200 is used to perform OCR recognition on the current frame image, extract the words with the highest frequency of occurrence in the current frame image, record them as target keywords, sort the target keywords in the current frame image according to their frequency of occurrence, and assign preset weights to each target keyword according to a preset weight list. It then calculates the frequency of occurrence of target keywords for each courseware in the courseware database, scores each courseware in the database based on the frequency of occurrence and the preset weights of each target keyword, and displays the highest-scoring courseware on the user's head-mounted AR device for human-computer interaction. The user confirms whether the target courseware has been found. If so, it obtains the words in the target courseware that are ranked from highest to lowest frequency and have the highest and second-highest preset ranking, recording them as the target word set. This is then used in conjunction with the current frame image extracted during the lesson preparation process. The target keywords and target vocabulary set are denoted as the courseware indexing process data set. A preset number of courseware indexing process data sets are collected, and the collected courseware indexing process data sets are divided into training datasets and validation datasets according to a first preset ratio. The training dataset is used to train the courseware intelligent indexing model, and the validation dataset is used to optimize the courseware intelligent indexing model, optimize the model parameters, and complete the training of the courseware intelligent indexing model. After the model training is completed, in the next lesson preparation process, the current frame image of the target material taken by the AR camera on the user's head-mounted AR device is obtained, the current frame image is subjected to OCR recognition, the target keywords in the current frame image are extracted, the extracted target keywords are input into the courseware intelligent indexing model, and the courseware output by the courseware intelligent indexing model is displayed on the user's head-mounted AR device.

[0110] It should be noted that, during lesson preparation, this invention acquires the current frame image of the target material captured by the AR camera on the user's head-mounted AR device; performs OCR recognition on the current frame image to extract the words with the highest frequency of occurrence in the current frame image, and obtains the words with the highest frequency of occurrence in the target courseware found by the user, ranked from highest to lowest, and ranked in the second-highest preset order, recording them as the target word set. The target keywords and target word set extracted from the current frame image during this lesson preparation process are recorded as the courseware indexing process data set, thereby training a courseware intelligent indexing model. In subsequent lesson preparation processes, this intelligent indexing model is used to intelligently index the courseware, and the courseware output by the intelligent indexing model is displayed on the user's head-mounted AR device. This enables the optimization of search results based on the user's historical search behavior or the correlation between data when the user attempts to extract or query specific courseware data, achieving rapid retrieval of related and potentially more valuable courseware. This significantly improves the efficiency, intelligence, and accuracy of the lesson preparation process, greatly reduces the workload of lesson preparation, and significantly enhances the user experience.

[0111] In a preferred embodiment, this application also provides an electronic device, the electronic device comprising:

[0112] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the AR-based intelligent lesson preparation method. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the method of the present invention.

[0113] This invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0114] Those skilled in the art will understand that the method steps of this invention can be performed by a computer program instructing related hardware, such as a computer device or processor, to perform the steps of this invention when executed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0115] Understandably, during lesson preparation, this invention acquires the current frame image of the target material captured by the AR camera on the user's head-mounted AR device; performs OCR recognition on the current frame image to extract the words with the highest frequency of occurrence in the current frame image, and obtains the words with the highest frequency of occurrence in the target courseware found by the user, ranked from largest to smallest, and ranked in the second-highest preset order, recording them as the target word set. The target keywords and target word set extracted from the current frame image during this lesson preparation process are recorded as the courseware indexing process data set, thereby training a courseware intelligent indexing model. In subsequent lesson preparation processes, this intelligent indexing model is used to intelligently index the courseware, and the courseware output by the intelligent indexing model is displayed on the user's head-mounted AR device. This enables the optimization of query results based on the user's historical query behavior or the correlation between data when the user attempts to extract or query specific courseware data, achieving rapid retrieval of related and potentially more valuable courseware. This significantly improves the efficiency, intelligence, and accuracy of the lesson preparation process, greatly reduces the workload of lesson preparation, and significantly enhances the user experience.

[0116] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0117] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An AR-based intelligent lesson preparation method, characterized in that, The method includes: S100. Receive courseware data from data sources of different types and formats to construct a courseware database; during the current lesson preparation process, acquire the current frame image of the target material captured by the AR camera on the user's head-mounted AR device; perform OCR recognition on the current frame image, extract the words with the highest frequency of occurrence in the current frame image as target keywords, sort the target keywords in the current frame image according to their frequency of occurrence, and assign preset weights to each target keyword according to a preset weight list; calculate the frequency of occurrence of target keywords for each courseware in the courseware database, and determine the target keywords based on the frequency of occurrence of each target keyword and the target keywords in the courseware database. The system assigns preset weights to keywords and scores each courseware in the courseware database. The courseware with the highest score is displayed on the user's head-mounted AR device for human-computer interaction. The user confirms whether the target courseware has been found. If so, the system retrieves the words that appear most frequently in the target courseware, ranked from largest to smallest, with the second-to-last preset ranking as the target word set. The system also records the target keywords and target word set extracted from the current frame image during the lesson preparation process as the courseware indexing process data set. A preset number of courseware indexing process data sets are collected and divided into training datasets and validation datasets according to a first preset ratio. S200. Train the courseware intelligent index model using the collected training dataset, optimize the courseware intelligent index model using the validation dataset, optimize the model parameters, and complete the training of the courseware intelligent index model. S300. After the model training is completed, during the next lesson preparation process, the current frame image of the target material captured by the AR camera on the user's head-mounted AR device is obtained, OCR recognition is performed on the current frame image, the target keywords in the current frame image are extracted, the extracted target keywords are input into the courseware intelligent index model, and the courseware output by the courseware intelligent index model is displayed on the user's head-mounted AR device.

2. The AR-based intelligent lesson preparation method according to claim 1, characterized in that, The target data includes at least one of the following: The content of the teaching materials to be used in this lesson preparation, the content of the teaching guidance documents to be used in this lesson preparation, and the content of the reference books to be used in this lesson preparation.

3. The AR-based intelligent lesson preparation method according to claim 1, characterized in that, The process of interacting with the user to confirm whether the target courseware has been found includes: The system outputs a prompt signal to the user indicating whether the target courseware has been found, and obtains feedback signals input by the user. The user-input feedback signals include feedback signals confirming that the target courseware has been found and feedback signals indicating that the target courseware has not been found. The system confirms whether the target courseware has been found based on the user-input feedback signals.

4. The AR-based intelligent lesson preparation method according to claim 1, characterized in that, The process of interacting with the user to confirm whether the target courseware has been found also includes: If the target courseware is not found, the current frame image of the target data captured by the AR camera on the user's head-mounted AR device is retrieved again, and process S100 is executed again.

5. The AR-based intelligent lesson preparation method according to claim 1, characterized in that, The process of training a courseware intelligent indexing model using the collected training dataset, optimizing the model using the validation dataset, and completing the training of the courseware intelligent indexing model includes: The training dataset is input into a preset network layer in batches for training. The preset network layer includes a Transformer network layer, which is used to predict the data set of the next courseware indexing process based on the data set of the current courseware indexing process through forward propagation, thereby obtaining the predicted loss value. The loss value of the preset network layer is calculated and input into the optimizer for optimization, determining the direction of the fastest gradient descent of the courseware intelligent indexing model's parameters. The courseware intelligent indexing model then performs backpropagation based on the loss value and the model's parameter gradient to optimize the parameters of the courseware intelligent indexing model.

6. The AR-based intelligent lesson preparation method according to claim 1, characterized in that, The process of training a courseware intelligent indexing model using the collected training dataset, optimizing the model using the validation dataset, and completing the training of the courseware intelligent indexing model includes: After each training iteration, the validation dataset is input into the preset network layer of the previous training in batches to validate the model parameters. The training is repeated in cycles, and the total number of training rounds is set to the first preset round. The loss value of the preset network layer is recorded, and it is determined whether the loss value meets the first preset condition. Based on the determination result, it is determined whether to end the training and output the model parameters.

7. The AR-based intelligent lesson preparation method according to claim 6, characterized in that, The step of determining whether the loss value meets the first preset condition, and determining whether to end training and output model parameters based on the determination result, includes: If the loss value meets the first preset condition, the training ends and the parameters of the current courseware intelligent indexing model are output; if the loss value does not meet the first preset condition, the training continues. The first preset conditions include: After the second preset training round, the loss values ​​obtained in the next training round are all greater than or equal to the loss values ​​that occurred during the second preset training round.

8. The AR-based intelligent lesson preparation method according to claim 1, characterized in that, The scoring of each courseware in the courseware database is based on the frequency of occurrence of each target keyword and the preset weight of each target keyword, including: Select a courseware and choose a target keyword from it. Calculate the product of the preset weight of the target keyword and the number of times it appears in the courseware. Record this as the first score of the target keyword. Calculate the sum of the first scores of all target keywords in the courseware and record this as the score of the courseware.

9. An AR-based intelligent lesson preparation system, characterized in that, include: The acquisition module is used to receive courseware data from data sources of different types and formats and build a courseware database; Used to acquire the current frame image of the target data captured by the AR camera on the user's head-mounted AR device during the current lesson preparation process; The control module performs OCR recognition on the current frame image, extracts the words with the highest frequency of occurrence in the current frame image, records them as target keywords, sorts the target keywords in the current frame image according to their frequency of occurrence, and assigns preset weights to each target keyword according to a preset weight list. It then calculates the frequency of occurrence of target keywords for each courseware in the courseware database. Based on the frequency of occurrence and preset weights of each target keyword, it scores each courseware in the database and displays the highest-scoring courseware on the user's head-mounted AR device for human-computer interaction. The user confirms whether the target courseware has been found. If so, it retrieves the words in the target courseware that are ranked from highest to lowest frequency and have the highest and second-highest preset ranking, recording them as the target word set. This set is then used to extract the target keywords from the current frame image during the lesson preparation process. The target keywords and target vocabulary set are denoted as the courseware indexing process data set. A preset number of courseware indexing process data sets are collected, and the collected courseware indexing process data sets are divided into training datasets and validation datasets according to a first preset ratio. The training dataset is used to train the courseware intelligent indexing model, and the validation dataset is used to optimize the courseware intelligent indexing model, optimize the model parameters, and complete the training of the courseware intelligent indexing model. After the model training is completed, in the next lesson preparation process, the current frame image of the target material taken by the AR camera on the user's head-mounted AR device is obtained, the current frame image is subjected to OCR recognition, the target keywords in the current frame image are extracted, the extracted target keywords are input into the courseware intelligent indexing model, and the courseware output by the courseware intelligent indexing model is displayed on the user's head-mounted AR device.

10. An electronic device, characterized in that, include: Memory; The memory stores computer-readable instructions that, when executed by the processor, implement the AR-based intelligent lesson preparation method according to any one of claims 1 to 8.

Images