Teaching plan animation generation method based on large language model

Through the lesson plan animation generation method based on the large language model, the problem of teachers' lack of skills in making lesson plan animation is solved, and the automatic generation of high-quality animations is achieved, and the teaching effect is improved.

CN119091010APending Publication Date: 2024-12-06BEIJING BAIGEFEICHI TECH LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411051523.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Teachers lack professional skills when making lesson plan animations, resulting in editing errors and poor picture quality, which increases workload and affects teaching effectiveness.

Method used

The lesson plan animation generation method based on the large language model is adopted. By obtaining lesson plan information, analyzing and constructing constraints, using the propt template to generate lesson plan html file until the answer constraints are met, the lesson plan animation is generated.

Benefits of technology

High-quality lesson plan animations can be generated without manual participation, which reduces production costs, improves teaching effectiveness, and facilitates teachers' explanations and students' learning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091010A_ABST
    Figure CN119091010A_ABST
Patent Text Reader

Abstract

The invention provides a teaching plan animation generation method based on a large language model, and belongs to the technical field of animation teaching, and the method comprises the steps: carrying out the analysis according to teaching plan information, constructing a first constraint condition and a second constraint condition corresponding to the teaching plan information, carrying out the answer solving through the object information matching calculation formula in the teaching plan information, and obtaining the answer information; feature information is extracted according to the teaching plan information, and a corresponding prompt template is selected from a prompt template library; based on the prompt template, generating a corresponding teaching plan html file through a large language model according to the target information of the teaching plan information; and constructing an answer constraint condition according to the answer information, and when the teaching plan html file meets the answer constraint condition, running the teaching plan html file to obtain a teaching plan animation. According to the method, the teaching plan animation corresponding to the teaching plan information can be generated without manual participation, the animation effect is effectively improved, the manufacturing cost is reduced, explanation of the teacher and learning of students are facilitated while the teacher is prevented from spending time and energy, and then the teaching effect is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of animation teaching, and in particular to a teaching plan animation generation method based on a large language model. Background Art

[0002] With the popularization of multimedia technology, teachers use projectors to play teaching animations to explain theorems and questions during the teaching process, making it easier for students to understand and learn the corresponding knowledge.

[0003] Teaching animations are usually obtained from the teaching CDs provided by the textbooks. When explaining theorems of exercises outside the textbooks, teachers are required to use UI and other software to make teaching animations themselves. Teachers lack professional animation production skills and experience, which causes problems such as editing errors and blurred image quality in the teaching animations they produce, affecting the teaching effect. In addition, while requiring teachers to make teaching animations, they also need to spend more time and energy to learn and make animations, resulting in a sharp increase in workload.

[0004] Therefore, a teaching plan animation generation method based on a large language model is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a teaching plan animation generation method based on a large language model, which is used to solve the problem that manual production of teaching plan animation in traditional technology is time-consuming and labor-intensive, and the animation quality is poor.

[0006] An embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising:

[0007] Get lesson plan information;

[0008] Parse the lesson plan information, construct a first constraint condition and a second constraint condition corresponding to the lesson plan information, and solve the answer by matching the object information in the lesson plan information with a calculation formula to obtain the answer information;

[0009] According to the teaching plan information, feature information is extracted, and a corresponding prompt template is selected from a prompt template library; based on the prompt template, a corresponding teaching plan html file is generated through a large language model according to target information of the teaching plan information;

[0010] Construct answer constraints according to the answer information, determine whether the lesson plan html file meets the answer constraints, and when the lesson plan html file does not meet the answer constraints, regenerate the corresponding lesson plan html file based on the prompt template and the target information of the lesson plan information through a large language model; when the lesson plan html file meets the answer constraints, run the lesson plan html file to obtain a lesson plan animation.

[0011] Preferably, an embodiment of the present invention provides a method for generating a teaching plan animation based on a large language model, wherein the step of obtaining teaching plan information comprises:

[0012] Get the original teaching plan title picture;

[0013] Performing text recognition based on the original teaching plan title picture, extracting teaching plan text information, performing integrity check, and obtaining teaching plan information; optionally, including:

[0014] Performing layout analysis on the original teaching plan title picture photo, analyzing the arrangement direction of text information in the original teaching plan title picture photo, adjusting the layout of text information, and obtaining a pre-processed teaching plan image;

[0015] Performing a line segmentation operation according to the gaps in the text line in the preprocessed teaching plan image to obtain a text line sub-image; recognizing each character baseline in the text line sub-image by a character recognition engine, segmenting the text characters in the text line sub-image to obtain a plurality of character images;

[0016] According to the character image, extract corresponding shape and texture features to obtain a character feature vector;

[0017] The character feature vector is encoded by a bidirectional temporal memory model to obtain a character encoding vector; optionally, including:

[0018] The character feature vector T is input to the jth coding unit in the forward time memory sub-model, and the jth coding unit generates the character coding vector c1 according to the output of the j-1th coding unit. j-1 and the character feature vector T j , output the character encoding sub-vector c1 through the activation function j ;

[0019] The character feature vector T j Input to the mjth coding unit in the reverse time memory sub-model, the mjth coding unit outputs the character coding vector c2 according to the m-j+1th coding unit m-j+1 and the character feature vector T j , output the character encoding sub-vector c2 through the activation function j ;

[0020] According to the character encoding sub-vector c1 j and the character encoding subvector c2 j , construct a character encoding vector; a bidirectional time memory model, including a forward time memory sub-model and a reverse time memory sub-model; the forward time memory sub-model and the reverse time memory sub-model both contain m encoding units;

[0021] Obtain the state encoding vector of the model decoder in the current state, and calculate the similarity parameter S(c i , d t );

[0022]

[0023] Among them, c i is the character encoding vector corresponding to the i-th character image, is the character encoding vector c i The transpose of t is the state encoding vector of the model decoder at the current state t;

[0024] According to the similarity parameter, obtaining the attention weight corresponding to the character feature vector;

[0025]

[0026] Among them, β it is the character feature vector c i The corresponding attention weight, n is the number of character images;

[0027] Determine a context vector A of the attention mechanism according to the character feature vector and its corresponding attention weight;

[0028]

[0029] According to the character encoding vector and the context association vector, the model decoder is input to perform decoding text prediction processing to obtain the teaching plan text information.

[0030] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the steps of: parsing according to the teaching plan information, constructing a first constraint condition and a second constraint condition corresponding to the teaching plan information, solving the answer by matching the object information in the teaching plan information with a calculation formula, and obtaining the answer information, including:

[0031] Parse the lesson plan information to obtain the subject scope corresponding to the lesson plan information and construct a first constraint condition;

[0032] Extracting object information contained in the teaching plan information, and matching a calculation formula corresponding to the object information based on the first constraint condition;

[0033] Extracting the scene information and auxiliary information contained in the teaching plan information to construct a second constraint condition;

[0034] Taking the problem information in the teaching plan information as the solution direction, based on the second constraint condition, the answer is analyzed by matching the calculation formula to obtain the answer information.

[0035] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the steps of: extracting feature information according to the teaching plan information, selecting a corresponding prompt template in a prompt template library; based on the prompt template, generating a corresponding teaching plan html file through a large language model according to target information of the teaching plan information; comprising:

[0036] Performing image recognition according to the teaching plan information;

[0037] When it is found that the teaching plan information contains original illustration information, the target object, background image and data information in the original illustration information are identified to obtain chart element information;

[0038] According to the teaching plan information, feature information is extracted, and a corresponding prompt template is selected from a prompt template library;

[0039] Based on the prompt template, the image element information and the target information are string encoded, and a lesson plan HTML file is generated through the large language model.

[0040] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, wherein the steps of: when parsing the teaching plan information to include original illustration information, identifying the target object, background image and data information in the original illustration information to obtain chart element information include:

[0041] The original illustration information is obtained, and based on the file type contained in the original illustration information, encoding is performed by a preset multimodal encoder to obtain a multimodal feature vector; optionally, including:

[0042] When the original illustration information includes image information, the image information is divided into a plurality of sub-image information according to a preset sub-image size by an image modality encoder, position encoding is performed on the plurality of sub-image information, and feature extraction is performed on the plurality of sub-image information after position encoding by a Transformer encoder to generate a corresponding image feature vector;

[0043] When the original illustration information includes chart information, data is extracted based on the marking information in the chart information by using a chart modal encoder to obtain a chart data group; the marking information is vector-encoded according to a preset dictionary to obtain a preset label vector identifier, and an initial chart vector is formed with the chart data group; a mask operation is performed on the vector data in the initial chart vector according to a random masker to obtain a masked chart vector, and feature extraction is performed based on a Mamba encoder to obtain a corresponding chart feature vector;

[0044] Projecting the vectors in the multimodal feature vector into the vector space through a cross attention mechanism, performing vector alignment, and obtaining a preprocessed feature vector;

[0045] The multimodal large language model performs semantic understanding to obtain signal tokens according to the preprocessed feature vectors, and obtains corresponding chart element information through the modality generator.

[0046] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, wherein the steps of: extracting feature information according to the teaching plan information, and selecting a corresponding prompt template in a prompt template library include:

[0047] Build prompt template library; optionally, include:

[0048] According to different question types in different subjects, set corresponding prompt templates; the prompt templates are as follows:

[0049] similar Question Q S similar Equation E S similar Answer A S Que Q R Ans A R

[0050] Among them, Que is the problem-solving identifier in the prompt template. Q R is the question information in the feature information, and Ans is the answer identifier in the prompt template. A R is the template answer information corresponding to the question information in the feature information; similarQuestion is the similar question identifier in the prompt template, Q S is the same type of question information as the question information in the feature information, similar Equation is the formula identifier in the prompt template, E S is the solution calculation formula corresponding to the similar question information, and similar Answer is the similar answer identifier in the prompt template. A S The answer corresponding to the information of the same type of questions;

[0051] The prompt template library generates a template identifier corresponding to the prompt template according to the problem identification Que and the similar question identification similar Question in the stored prompt template, and stores the template identifier in the prompt template library;

[0052] Preprocess the teaching plan information, remove auxiliary word information, vectorize the acquired keyword information, and construct feature information; perform information matching retrieval based on the subject classification dictionary according to the feature information, obtain the subject classification and teaching plan question type with the highest matching degree with the feature information, and generate a retrieval identifier;

[0053] According to the search identifier, a match is performed with a template identifier in the prompt template library, and the prompt template corresponding to the template identifier is selected.

[0054] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the steps of: based on the prompt template, character string encoding the image element information and the target information, and generating a teaching plan HTML file through the large language model; comprising:

[0055] Extract the topic information from the lesson plan information and fill it into the prompt template Q R middle;

[0056] Based on the subject and question type of the lesson plan information, select an external data set to match similar questions, and fill the obtained similar questions into the prompt template Q S , further index the question calculation formula and question answer information corresponding to the similar questions and fill them into the prompt template E S , A S middle;

[0057] The large language model is based on the known Q S , E S , A S and Q R , find the template answer information corresponding to the question information A R ;

[0058] The large language model obtains a preset HTML structure template, extracts the template string in the HTML structure template, encodes the obtained image element information, target information and template answer information into strings, generates variable strings, replaces the template string in the HTML structure template, and obtains the lesson plan HTML file.

[0059] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising: constructing a large language model; optionally, comprising:

[0060] Obtain an initial large language model;

[0061] By acquiring HTML code data under different teaching subjects, a training data set is constructed; the HTML code data includes text-code data, code-code data and vector-code data;

[0062] Preprocessing the training data set, filtering out repeated HTML code data and noise HTML code data in the training data set, and obtaining preprocessed training data;

[0063] Training the initial large language model according to the preprocessed training data to obtain a pretrained large language model;

[0064] A test data set is constructed, and the pre-trained large language model is optimized based on time complexity, string replacement accuracy, and inference accuracy to obtain a large language model.

[0065] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the step of: performing image recognition according to the teaching plan information, and then further comprising:

[0066] When it is parsed that the teaching plan information does not contain the original illustration information, extracting the target information in the teaching plan information;

[0067] According to the teaching plan information, feature information is extracted, and a corresponding prompt template is selected from a prompt template library;

[0068] Based on the prompt template, a lesson plan HTML file is generated through a large language model according to the target information of the lesson plan information.

[0069] Preferably, an embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the steps of: constructing an answer constraint condition according to the answer information, judging whether the teaching plan html file satisfies the answer constraint condition, and when the teaching plan html file does not satisfy the answer constraint condition, regenerating a corresponding teaching plan html file based on the prompt template and the target information of the teaching plan information through a large language model; when the teaching plan html file satisfies the answer constraint condition, running the teaching plan html file to obtain a teaching plan animation, including:

[0070] When the lesson plan html file does not meet the answer constraint condition, based on the prompt template and according to the target information of the lesson plan information, the corresponding lesson plan html file is regenerated through a large language model to determine whether the regenerated lesson plan html file meets the answer constraint condition. Until the lesson plan html file meets the answer constraint condition or reaches a preset number of code generation times, the judgment of the lesson plan html file based on the answer constraint condition is stopped, the lesson plan html file is run, and the lesson plan animation is obtained.

[0071] Compared with traditional technologies, the beneficial effects of the present invention are: by parsing the lesson plan information, based on the selected prompt template, a corresponding lesson plan HTML file is generated through a large language model, and when the answer constraint conditions are met, a lesson plan animation is generated; compared with traditional technologies, the above method can generate a lesson plan animation corresponding to the lesson plan information without human participation, while avoiding teachers spending time and energy on making lesson plan animations, effectively improving the animation effect, reducing the cost of making lesson plan animations, facilitating teachers' explanations and students' learning, and thus effectively improving the teaching effect.

[0072] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0073] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A schematic diagram of the flow of a teaching plan animation generation method based on a large language model provided by the present invention. DETAILED DESCRIPTION

[0075] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0076] Embodiment 1:

[0077] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, referring to Figure 1 ,include:

[0078] Get lesson plan information;

[0079] Parse the lesson plan information, construct the first constraint condition and the second constraint condition corresponding to the lesson plan information, and solve the answer by matching the calculation formula with the object information in the lesson plan information to obtain the answer information;

[0080] According to the lesson plan information, feature information is extracted, and the corresponding prompt template is selected from the prompt template library; based on the prompt template, the corresponding lesson plan HTML file is generated through the large language model according to the target information of the lesson plan information;

[0081] Construct answer constraints based on the answer information, and determine whether the lesson plan HTML file meets the answer constraints. When the lesson plan HTML file does not meet the answer constraints, regenerate the corresponding lesson plan HTML file based on the prompt template and the target information of the lesson plan information through the large language model; when the lesson plan HTML file meets the answer constraints, run the lesson plan HTML file to obtain the lesson plan animation.

[0082] In the above embodiments, the first constraint and the second constraint corresponding to the lesson plan information are constructed by parsing the acquired lesson plan information, and the object information in the lesson plan information is matched with the calculation formula to solve the answer and obtain the answer information; according to the lesson plan information, the feature information is extracted, and the corresponding prompt template is selected in the prompt template library; and based on the prompt template, the corresponding lesson plan html file is generated through the large language model according to the target information of the lesson plan information; and the answer constraint is constructed according to the answer information, and it is determined whether the lesson plan html file meets the answer constraint. When the lesson plan html file meets the answer constraint, the lesson plan html file is run to obtain the lesson plan animation.

[0083] In the above embodiment, when the lesson plan HTML file does not meet the answer constraint conditions, based on the prompt template, the corresponding lesson plan HTML file is regenerated through the large language model according to the target information of the lesson plan information until the lesson plan HTML file meets the answer constraint conditions.

[0084] The beneficial effects of the above technology are: by parsing the lesson plan information, based on the selected prompt template, the corresponding lesson plan HTML file is generated through the large language model, and when the answer constraints are met, the lesson plan animation is generated; compared with traditional technologies, the above method does not require human participation to generate lesson plan animations corresponding to the lesson plan information, while avoiding teachers spending time and energy on making lesson plan animations, effectively improving the animation effect, reducing the cost of lesson plan animation production, facilitating teachers' explanations and students' learning, and thus effectively improving teaching results.

[0085] Embodiment 2:

[0086] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the steps of obtaining teaching plan information include:

[0087] Get the original teaching plan title picture;

[0088] Perform text recognition based on the original teaching plan title photo, extract the teaching plan text information, perform integrity check, and obtain the teaching plan information; optional, including:

[0089] Performing layout analysis on the original teaching plan title picture photo, analyzing the arrangement direction of text information in the original teaching plan title picture photo, adjusting the layout of text information, and obtaining a pre-processed teaching plan image;

[0090] Performing a line segmentation operation according to the gaps in the text line in the preprocessed teaching plan image to obtain a text line sub-image; using a character recognition engine to recognize each character baseline in the text line sub-image, segmenting the text characters in the text line sub-image to obtain a number of character images;

[0091] According to the character image, the corresponding shape and texture features are extracted to obtain the character feature vector;

[0092] The character feature vector is encoded by a bidirectional temporal memory model to obtain a character encoding vector; optionally, including:

[0093] The character feature vector T is input to the jth encoding unit in the forward time memory sub-model. The jth encoding unit generates the character encoding vector c1 according to the output of the j-1th encoding unit. j-1 and character feature vector T j , output the character encoding sub-vector c1 through the activation function j ;

[0094] Character feature vector T j Input to the mjth encoding unit in the reverse time memory sub-model, the mjth encoding unit outputs the character encoding vector c2 according to the m-j+1th encoding unit m-j+1 and character feature vector T j , output the character encoding sub-vector c2 through the activation function j ;

[0095] According to the character encoding sub-vector c1 j and character encoding subvector c2 j , construct a character encoding vector; a bidirectional time memory model, including a forward time memory sub-model and a reverse time memory sub-model; the forward time memory sub-model and the reverse time memory sub-model both contain m encoding units;

[0096] Get the state encoding vector of the model decoder in the current state, and calculate the similarity parameter S(ci , d t );

[0097]

[0098] Among them, c i is the character encoding vector corresponding to the i-th character image, is the character encoding vector c i The transpose of t is the state encoding vector of the model decoder at the current state t;

[0099] According to the similarity parameter, obtain the attention weight corresponding to the character feature vector;

[0100]

[0101] Among them, β it is the character feature vector c i The corresponding attention weight, n is the number of character images;

[0102] According to the character feature vector and its corresponding attention weight, determine the context association vector A of the attention mechanism;

[0103]

[0104] According to the character encoding vector and the context association vector, the model decoder is input to perform decoding text prediction processing to obtain the teaching plan text information.

[0105] In the above embodiment, after obtaining the original teaching plan title picture, text recognition is performed to extract the teaching plan text information, and an integrity check is performed to obtain the teaching plan information.

[0106] In the above embodiments, the layout of the original lesson plan title picture is analyzed, the arrangement direction of the text information in the original lesson plan title picture is analyzed, the layout of the text information is adjusted, and a preprocessed lesson plan image is obtained; a line segmentation operation is performed according to the gaps in the text lines in the preprocessed lesson plan image to obtain a text line sub-image; the baseline of each character in the text line sub-image is recognized by a character recognition engine, and the text characters in the text line sub-image are segmented to obtain a plurality of character images.

[0107] In the above embodiments, the corresponding shape and texture features are extracted according to the character image to obtain the character feature vector, and the character feature vector is encoded through a bidirectional temporal memory model to obtain the character encoding vector; the similarity parameter between the character feature vector and the state encoding vector of the model decoder in the current state is calculated, and the attention weight corresponding to the character feature vector is further obtained to determine the context association vector of the attention mechanism; according to the character encoding vector and the context association vector, the model decoder is input for decoding text prediction processing to obtain the teaching plan text information.

[0108] In the above embodiments, the bidirectional time memory model includes a forward time memory sub-model and a reverse time memory sub-model. Each character feature vector is processed by the forward time memory sub-model and the reverse time memory sub-model to construct a character encoding vector.

[0109] In the above embodiments, the forward time memory sub-model and the reverse time memory sub-model can be implemented as an LSM neural network model.

[0110] The beneficial effects of the above technology are: by performing layout analysis on the original lesson plan title picture photo, and performing line segmentation and baseline recognition on the text in the pre-processed lesson plan image, the character image corresponding to each character in the original lesson plan title picture photo is obtained, which facilitates the subsequent steps of recognizing the text in the original lesson plan title picture photo; by extracting the shape and texture features of the character image, obtaining the character feature vector, and performing text prediction through the decoder based on the context association vector, the lesson plan text information corresponding to the character image is obtained, so that the obtained lesson plan text information is associated with the context features of the character image while extracting the character image features, further improving the accuracy of the obtained lesson plan text information.

[0111] Embodiment 3:

[0112] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the steps of: parsing according to teaching plan information, constructing a first constraint condition and a second constraint condition corresponding to the teaching plan information, solving the answer by matching the calculation formula with the object information in the teaching plan information, and obtaining the answer information, including:

[0113] Parse the lesson plan information to obtain the subject scope corresponding to the lesson plan information and construct the first constraint condition;

[0114] Extracting object information contained in the teaching plan information, and matching a calculation formula corresponding to the object information based on the first constraint condition;

[0115] Extracting the scene information and auxiliary information contained in the teaching plan information, and constructing the second constraint condition;

[0116] The problem information in the teaching plan information is used as the solution direction. Based on the second constraint condition, the answer is analyzed through the matched calculation formula to obtain the answer information.

[0117] In the above embodiments, the lesson plan information is parsed, and based on the corresponding subject scope and object information, the first constraint condition and the second constraint condition are constructed, the corresponding calculation formula is matched, the answer is parsed, and the corresponding answer information is obtained.

[0118] The beneficial effect of the above technology is that by analyzing the teaching plan information, the first constraint condition and the second constraint condition are obtained, the corresponding calculation formula is obtained, and the answer is analyzed, thereby realizing the solution of the teaching plan information.

[0119] Embodiment 4:

[0120] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising the steps of: extracting feature information according to teaching plan information, selecting a corresponding prompt template in a prompt template library; based on the prompt template, generating a corresponding teaching plan HTML file through a large language model according to target information of the teaching plan information; comprising:

[0121] Perform image recognition based on lesson plan information;

[0122] When the lesson plan information is parsed to contain original illustration information, the target object, background image and data information in the original illustration information are identified to obtain chart element information;

[0123] According to the teaching plan information, extract the feature information and select the corresponding prompt template in the prompt template library;

[0124] Based on the prompt template, the image element information and target information are encoded as strings, and the lesson plan HTML file is generated through the large language model.

[0125] In the above embodiments, when it is parsed that the lesson plan information contains original illustration information, the target object, background image and data information in the original illustration information are identified to obtain the chart element information; and based on the lesson plan information, the feature information is extracted, and the corresponding prompt template is selected from the prompt template library; based on the prompt template, the image element information and target information are string encoded, and the lesson plan html file is generated through the large language model.

[0126] The beneficial effects of the above technology are: by identifying the target object, background image and data information of the original illustration information in the lesson plan information, the chart element information is obtained; by extracting the feature information in the lesson plan information, the prompt template is selected; further, through the large language model, the image element information and target information are string encoded according to the selected prompt template, so as to generate the lesson plan HTML file. The above technical solution can obtain the lesson plan HTML file through the large language model while realizing the parsing of the lesson plan information.

[0127] Embodiment 5:

[0128] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising the steps of: when the teaching plan information is parsed to contain original illustration information, identifying the target object, background image and data information in the original illustration information, and obtaining chart element information, including:

[0129] The original illustration information is obtained, and based on the file type contained in the original illustration information, encoding is performed through a preset multimodal encoder to obtain a multimodal feature vector; optionally, including:

[0130] When the original illustration information contains image information, the image information is divided into a plurality of sub-image information according to a preset sub-image size by an image modality encoder, and position encoding is performed on the plurality of sub-image information, and feature extraction is performed on the plurality of sub-image information after position encoding by a Transformer encoder to generate a corresponding image feature vector;

[0131] When the original illustration information contains chart information, data is extracted based on the marking information in the chart information through the chart modal encoder to obtain a chart data group; the marking information is vector-encoded according to a preset dictionary to obtain a preset label vector identifier, and an initial chart vector is formed with the chart data group, and a mask operation is performed on the vector data in the initial chart vector according to a random masker to obtain a masked chart vector, and feature extraction is performed based on the Mamba encoder to obtain a corresponding chart feature vector;

[0132] The vectors in the multimodal feature vector are projected into the vector space through the cross attention mechanism, and the vectors are aligned to obtain the preprocessed feature vector;

[0133] The multimodal large language model performs semantic understanding to obtain signal tokens based on the preprocessed feature vectors, and obtains the corresponding chart element information through the modal generator.

[0134] In the above embodiments, when the teaching plan information includes original illustration information, the original illustration information is encoded by a preset multimodal encoder to obtain a multimodal feature vector, and the vectors in the multimodal feature vector are projected to the vector space through a cross-attention mechanism to perform vector alignment to obtain a preprocessed feature vector; the multimodal large language model performs semantic understanding based on the preprocessed feature vector to obtain a signal token, and obtains the corresponding chart element information through a modal generator.

[0135] In the above embodiments, the multi-modal encoder includes an image modality encoder and a chart modality encoder.

[0136] In the above embodiments, the image modality encoder segments the image information in the original illustration information, and position-encodes the segmented sub-image information. The Transformer encoder extracts features of the position-encoded sub-image information to generate corresponding image feature vectors.

[0137] In the above embodiments, the chart modal encoder performs data extraction on the chart modal encoder in the original illustration information to obtain a chart data group; the marking information is vector encoded according to a preset dictionary to obtain a preset label vector identifier, which is composed of an initial chart vector with the chart data group, and the vector data in the initial chart vector is masked by a random masker to obtain a masked chart vector, and feature extraction is performed based on the Mamba encoder to obtain a corresponding chart feature vector.

[0138] The beneficial effect of the above technology is that when the lesson plan information contains original illustration information, the multimodal feature vector in the original illustration information is extracted through the multimodal encoder, and the semantic understanding and reasoning of the multimodal feature vector is realized through the cross-attention mechanism and the multimodal large language model, and the signal token is obtained. The signal token is parsed into the corresponding chart element information through the modal generator. The above technical solution realizes the extraction of feature elements in the original illustration information and generates chart element information, providing support for the large language model to generate lesson plan html files.

[0139] Embodiment 6:

[0140] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, the steps of: extracting feature information according to teaching plan information, and selecting a corresponding prompt template in a prompt template library, including:

[0141] Build prompt template library; optionally, include:

[0142] According to different question types in different subjects, set corresponding prompt templates; the prompt templates are as follows:

[0143] similar Question Q S similar Equation E S similar Answer A S Que Q R Ans A R

[0144] Among them, Que is the problem-solving identifier in the prompt template. Q R is the question information in the feature information, and Ans is the answer identifier in the prompt template. A R is the template answer information corresponding to the question information in the feature information; similarQuestion is the similar question identifier in the prompt template, Q S is the same type of question information as the question information in the feature information, similar Equation is the formula identifier in the prompt template, E S is the solution calculation formula corresponding to the same type of question information, and similar answer is the similar answer identifier in the prompt template. A S Provide answers to similar questions;

[0145] The prompt template library generates a template identifier corresponding to the prompt template according to the problem identification Que and the similar question identification similar Question in the stored prompt template, and stores it in the prompt template library;

[0146] Preprocess the teaching plan information, remove auxiliary word information, vectorize the obtained keyword information, and construct feature information; perform information matching retrieval based on the subject classification dictionary according to the feature information, obtain the subject classification and teaching plan question type with the highest matching degree with the feature information, and generate a retrieval mark;

[0147] According to the search identifier, match it with the template identifier in the prompt template library, and select the prompt template corresponding to the template identifier.

[0148] In the above embodiments, a prompt template library is constructed by acquiring prompt templates corresponding to different question types under different subjects.

[0149] In the above embodiments, the lesson plan information is preprocessed to remove auxiliary word information, and the obtained keyword information is vector-encoded to construct feature information; information matching and retrieval is performed based on the subject classification dictionary according to the feature information to obtain the subject classification and lesson plan question type that best matches the feature information, and generate a retrieval identifier; based on the retrieval identifier, it is matched with the template identifier in the prompt template library to select the prompt template corresponding to the template identifier.

[0150] The beneficial effects of the above technology are: by preprocessing the lesson plan information, the acquired keyword information is vector-encoded to achieve the acquisition of feature information; and based on the feature information, information matching retrieval is performed based on the subject classification dictionary to generate a retrieval identifier, and the retrieval identifier is matched with the template identifier in the prompt template library to achieve the selection of the prompt template.

[0151] Embodiment 7:

[0152] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising the steps of: based on a prompt template, encoding image element information and target information by character string, and generating a teaching plan HTML file through a large language model; comprising:

[0153] Extract the topic information from the lesson plan information and fill it into the prompt template Q R middle;

[0154] Based on the subject and question type of the lesson plan information, select an external data set to match similar questions, and fill the obtained similar questions into the prompt template Q S , further index the calculation formula and answer information of similar questions and fill them into the prompt template E S , A S middle;

[0155] The large language model is based on the known Q S , E S , A S and Q R , find the template answer information corresponding to the question information A R ;

[0156] The large language model obtains a preset HTML structure template, extracts the template string in the HTML structure template, encodes the obtained image element information, target information and template answer information into strings, generates variable strings, replaces the template string in the HTML structure template, and obtains the lesson plan HTML file.

[0157] In the above embodiment, based on the subject and question type to which the teaching plan information belongs, an external data set is selected for similar question matching, and the obtained similar questions are filled into the prompt template. Q S , further index the calculation formula and answer information of similar questions and fill them into the prompt template E S , A S Extract the topic information from the lesson plan information and fill it into the prompt template Q R In the large language model, the knownQ S , E S , A S and Q R , find the template answer information corresponding to the question information A R .

[0158] In the above embodiment, the large language model obtains a preset HTML structure template, encodes the obtained image element information, target information and template answer information into strings, generates a variable string, replaces the template string in the HTML structure template, and obtains the lesson plan HTML file.

[0159] The beneficial effects of the above technology are: through the lesson plan information and the corresponding external data set, the prompt template can be filled in, and the template answer information corresponding to the question information can be solved through the large language model; the large language model obtains the preset HTML structure template and replaces the variable string to obtain the lesson plan html file, and then the above technical solution realizes the generation of the lesson plan html file according to the prompt template by the large language model.

[0160] Embodiment 8:

[0161] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising: constructing a large language model; optionally, comprising:

[0162] Obtain an initial large language model;

[0163] By acquiring HTML code data under different teaching subjects, a training data set is constructed; HTML code data includes text-code data, code-code data, and vector-code data;

[0164] Preprocess the training data set, filter out repeated HTML code data and noise HTML code data in the training data set, and obtain preprocessed training data;

[0165] According to the preprocessed training data, the initial large language model is trained to obtain a pretrained large language model;

[0166] Build a test dataset and optimize the pre-trained large language model based on time complexity, string replacement accuracy, and inference accuracy to obtain a large language model.

[0167] In the above embodiments, the initial large language model includes the open source LLaMA3 model, the Mistral 8x7B model and the Yi-34B model.

[0168] In the above embodiments, the initial large language model is trained by preprocessing the training data to obtain the pre-trained large language model, and the pre-trained large language model is optimized by the test data set to obtain the large language model.

[0169] In the above embodiments, the html code data includes text-code data, code-code data and vector-code data, so as to enable the model to generate codes for various types of input data.

[0170] In the above embodiments, the training data set is constructed by acquiring HTML code data under different teaching subjects, and the repeated HTML code data and noise HTML code data in the training data set are filtered out to acquire the pre-processed training data.

[0171] In the above embodiments, based on time complexity, a pre-trained large language model reads a certain instruction problem in a test data set to generate code. When the generated instruction code does not meet the time requirement, the model control parameters are optimized through the standard instruction code.

[0172] In the above embodiment, based on the string replacement accuracy, the pre-trained large language model reads a certain instruction question in the test data set, obtains the HTML structure template corresponding to the instruction question, encodes according to the instruction elements in the instruction question, replaces the template string in the HTML structure template, performs a string replacement accuracy test through the standard instruction code, and optimizes the model control parameters.

[0173] In the above embodiment, based on the inference accuracy, the pre-trained large language model reads a certain instruction question in the test data set to generate code, extracts the instruction answer information in the generated instruction code and compares it with the standard instruction code to determine whether it is consistent, and optimizes the model control parameters.

[0174] The beneficial effects of the above technology are: by preprocessing the training data, the initial large language model is trained, the global parameters of the initial large language model are fine-tuned, and the pre-trained large language model is obtained; and the pre-trained large language model is optimized through the test data set, so that the obtained large language model can meet the requirements of time complexity, string replacement accuracy and reasoning accuracy, thereby improving the accuracy and timeliness of the large language model to generate the lesson plan HTML file.

[0175] Embodiment 9:

[0176] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising the steps of: performing image recognition according to teaching plan information, and then further comprising:

[0177] When it is found that the lesson plan information does not contain the original illustration information, the target information in the lesson plan information is extracted;

[0178] According to the teaching plan information, extract the feature information and select the corresponding prompt template in the prompt template library;

[0179] Based on the prompt template, the lesson plan HTML file is generated through the large language model according to the target information of the lesson plan.

[0180] In the above embodiments, when it is parsed that the lesson plan information does not contain the original illustration information, the target information in the lesson plan information is extracted; based on the lesson plan information, the feature information is extracted, and the corresponding prompt template is selected from the prompt template library; based on the prompt template, a lesson plan html file is generated through a large language model according to the target information of the lesson plan information.

[0181] In the above embodiments, the target information includes the physical object information, data information and question information in the teaching plan information.

[0182] The beneficial effects of the above technology are: extracting feature information through lesson plan information, selecting the corresponding prompt template in the prompt template library, and based on the prompt template, generating a lesson plan html file through a large language model according to the target information, thereby realizing the acquisition of the lesson plan html file when the original illustration information is not included in the lesson plan information.

[0183] Embodiment 10:

[0184] The embodiment of the present invention provides a teaching plan animation generation method based on a large language model, comprising the steps of: constructing answer constraints according to answer information, judging whether a teaching plan HTML file satisfies the answer constraints, and when the teaching plan HTML file does not satisfy the answer constraints, regenerating a corresponding teaching plan HTML file based on a prompt template and target information of the teaching plan information through a large language model; when the teaching plan HTML file satisfies the answer constraints, running the teaching plan HTML file to obtain a teaching plan animation, including:

[0185] When the lesson plan HTML file does not meet the answer constraint conditions, based on the prompt template, the corresponding lesson plan HTML file is regenerated through the large language model according to the target information of the lesson plan information, and it is judged whether the regenerated lesson plan HTML file meets the answer constraint conditions. Until the lesson plan HTML file meets the answer constraint conditions or reaches the preset number of code generation times, the judgment of the lesson plan HTML file based on the answer constraint conditions is stopped, the lesson plan HTML file is run, and the lesson plan animation is obtained.

[0186] In the above embodiment, the lesson plan HTML file output by the large language model is judged based on the answer constraint condition. When the lesson plan HTML file does not meet the answer constraint condition, based on the prompt template, the large language model is used to regenerate the corresponding lesson plan HTML file according to the target information of the lesson plan information for further judgment to determine whether the HTML answer information in the lesson plan HTML file is consistent with the answer constraint condition, until the lesson plan HTML file meets the answer constraint condition or reaches the preset number of code generation times.

[0187] The beneficial effect of the above technology is that by judging the answer constraints of the lesson plan HTML file output by the large language model, the judgment of the HTML answer information in the lesson plan HTML file is realized, and when the answer constraints are not met, the corresponding lesson plan HTML file is regenerated for re-judgment to ensure that the generated lesson plan HTML file meets the answer constraints, and by presetting the number of code generation times, the large language model is prevented from falling into local optimality.

[0188] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A teaching plan animation generation method based on a large language model, characterized in that: include: Get lesson plan information; Parse the lesson plan information, construct a first constraint condition and a second constraint condition corresponding to the lesson plan information, and solve the answer by matching the object information in the lesson plan information with a calculation formula to obtain the answer information; According to the teaching plan information, feature information is extracted, and a corresponding prompt template is selected from a prompt template library; Based on the prompt template, a corresponding lesson plan HTML file is generated through a large language model according to the target information of the lesson plan information; Construct answer constraints according to the answer information, determine whether the lesson plan html file meets the answer constraints, and when the lesson plan html file does not meet the answer constraints, regenerate the corresponding lesson plan html file based on the prompt template and the target information of the lesson plan information through a large language model; when the lesson plan html file meets the answer constraints, run the lesson plan html file to obtain a lesson plan animation.

2. The teaching plan animation generation method based on a large language model according to claim 1, characterized in that: The step of obtaining teaching plan information includes: Get the original teaching plan title picture; Performing text recognition based on the original teaching plan title picture, extracting teaching plan text information, performing integrity check, and obtaining teaching plan information; optionally, including: Performing layout analysis on the original teaching plan title picture photo, analyzing the arrangement direction of text information in the original teaching plan title picture photo, adjusting the layout of text information, and obtaining a pre-processed teaching plan image; Performing a line segmentation operation according to the gaps in the text line in the preprocessed teaching plan image to obtain a text line sub-image; recognizing each character baseline in the text line sub-image by a character recognition engine, segmenting the text characters in the text line sub-image to obtain a plurality of character images; According to the character image, extract corresponding shape and texture features to obtain a character feature vector; The character feature vector is encoded by a bidirectional temporal memory model to obtain a character encoding vector; optionally, including: The character feature vector T is input to the jth coding unit in the forward time memory sub-model, and the jth coding unit generates the character coding vector c1 according to the output of the j-1th coding unit. j -1 and the character feature vector T j , output the character encoding sub-vector c1 through the activation function j ; The character feature vector T j Input to the mjth coding unit in the reverse time memory sub-model, the mjth coding unit outputs the character coding vector c2 according to the m-j+1th coding unit m-j+1 and the character feature vector T j , output the character encoding sub-vector c2 through the activation function j ; According to the character encoding sub-vector c1 j and the character encoding subvector c2 j , construct a character encoding vector; a bidirectional time memory model, including a forward time memory sub-model and a reverse time memory sub-model; the forward time memory sub-model and the reverse time memory sub-model both contain m encoding units; Obtain the state encoding vector of the model decoder in the current state, and calculate the similarity parameter S(c i , d t ); Among them, c i is the character encoding vector corresponding to the i-th character image, is the transpose of the character encoding vector ci, d t is the state encoding vector of the model decoder at the current state t; According to the similarity parameter, obtaining the attention weight corresponding to the character feature vector; Among them, β it is the character feature vector c i The corresponding attention weight, n is the number of character images; Determine a context vector A of the attention mechanism according to the character feature vector and its corresponding attention weight; According to the character encoding vector and the context association vector, the model decoder is input to perform decoding text prediction processing to obtain the teaching plan text information.

3. The teaching plan animation generation method based on a large language model according to claim 1, characterized in that: The step of parsing the teaching plan information, constructing the first constraint condition and the second constraint condition corresponding to the teaching plan information, and solving the answer by matching the calculation formula with the object information in the teaching plan information to obtain the answer information includes: Parse the lesson plan information to obtain the subject scope corresponding to the lesson plan information and construct a first constraint condition; Extracting object information contained in the teaching plan information, and matching a calculation formula corresponding to the object information based on the first constraint condition; Extracting the scene information and auxiliary information contained in the teaching plan information to construct a second constraint condition; Taking the problem information in the teaching plan information as the solution direction, based on the second constraint condition, the answer is analyzed by matching the calculation formula to obtain the answer information.

4. The teaching plan animation generation method based on a large language model according to claim 1, characterized in that: The step: extracting feature information according to the teaching plan information, and selecting a corresponding prompt template in a prompt template library; Based on the prompt template, a corresponding lesson plan HTML file is generated through a large language model according to the target information of the lesson plan information; including: Performing image recognition according to the teaching plan information; When it is found that the teaching plan information contains original illustration information, the target object, background image and data information in the original illustration information are identified to obtain chart element information; According to the teaching plan information, feature information is extracted, and a corresponding prompt template is selected from a prompt template library; Based on the prompt template, the image element information and the target information are string encoded, and a lesson plan HTML file is generated through the large language model.

5. The teaching plan animation generation method based on a large language model according to claim 4 is characterized in that: The step of: when parsing the teaching plan information to include original illustration information, identifying the target object, background image and data information in the original illustration information to obtain chart element information includes: The original illustration information is obtained, and based on the file type contained in the original illustration information, encoding is performed by a preset multimodal encoder to obtain a multimodal feature vector; optionally, including: When the original illustration information includes image information, the image information is divided into a plurality of sub-image information according to a preset sub-image size by an image modality encoder, position encoding is performed on the plurality of sub-image information, and feature extraction is performed on the plurality of sub-image information after position encoding by a Transformer encoder to generate a corresponding image feature vector; When the original illustration information includes chart information, data is extracted based on the marking information in the chart information by using a chart modal encoder to obtain a chart data group; the marking information is vector-encoded according to a preset dictionary to obtain a preset label vector identifier, and an initial chart vector is formed with the chart data group; a mask operation is performed on the vector data in the initial chart vector according to a random masker to obtain a masked chart vector, and feature extraction is performed based on a Mamba encoder to obtain a corresponding chart feature vector; Projecting the vectors in the multimodal feature vector into the vector space through a cross attention mechanism, performing vector alignment, and obtaining a preprocessed feature vector; The multimodal large language model performs semantic understanding to obtain signal tokens according to the preprocessed feature vectors, and obtains corresponding chart element information through the modality generator.

6. The teaching plan animation generation method based on a large language model according to claim 4, characterized in that: The step of extracting feature information according to the teaching plan information and selecting a corresponding prompt template in a prompt template library includes: Build prompt template library; optionally, include: According to different question types in different subjects, set corresponding prompt templates; the prompt templates are as follows: similar Question Q S similar Equation E S similar Answer A S Que Q R Ans A R Among them, Que is the problem-solving identifier in the prompt template. Q R is the question information in the feature information, and Ans is the answer identifier in the prompt template. A R is the template answer information corresponding to the question information in the feature information; similarQuestion is the similar question identifier in the prompt template, Q S is the same type of question information as the question information in the feature information, similar Equation is the formula identifier in the prompt template, E S is the solution calculation formula corresponding to the similar question information, and similar Answer is the similar answer identifier in the prompt template. A S The answer corresponding to the information of the same type of questions; The prompt template library generates a template identifier corresponding to the prompt template according to the problem identification Que and the similar question identification similar Question in the stored prompt template, and stores the template identifier in the prompt template library; Preprocess the teaching plan information, remove auxiliary word information, vectorize the acquired keyword information, and construct feature information; perform information matching retrieval based on the subject classification dictionary according to the feature information, obtain the subject classification and teaching plan question type with the highest matching degree with the feature information, and generate a retrieval identifier; According to the search identifier, a match is performed with a template identifier in the prompt template library, and the prompt template corresponding to the template identifier is selected.

7. The method for generating teaching plan animation based on a large language model according to claim 6, characterized in that: The step of: encoding the image element information and the target information by character string based on the prompt template, and generating a lesson plan HTML file through the large language model; comprises: Extract the topic information from the lesson plan information and fill it into the prompt template Q R middle; Based on the subject and question type of the lesson plan information, select an external data set to match similar questions, and fill the obtained similar questions into the prompt template Q S , further index the question calculation formula and question answer information corresponding to the similar questions and fill them into the prompt template E S , A S middle; The large language model is based on the known Q S , E S , A S and Q R , find the template answer information corresponding to the question information A R ; The large language model obtains a preset HTML structure template, extracts the template string in the HTML structure template, encodes the obtained image element information, target information and template answer information into strings, generates variable strings, replaces the template string in the HTML structure template, and obtains the lesson plan HTML file.

8. The teaching plan animation generation method based on a large language model according to claim 6, characterized in that: include: Build a large language model; Optionally include: Obtain an initial large language model; By acquiring HTML code data under different teaching subjects, a training data set is constructed; the HTML code data includes text-code data, code-code data and vector-code data; Preprocessing the training data set, filtering out repeated HTML code data and noise HTML code data in the training data set, and obtaining preprocessed training data; Training the initial large language model according to the preprocessed training data to obtain a pretrained large language model; A test data set is constructed, and the pre-trained large language model is optimized based on time complexity, string replacement accuracy, and inference accuracy to obtain a large language model.

9. The method for generating teaching plan animation based on a large language model according to claim 4, characterized in that: The step of performing image recognition according to the teaching plan information further includes: When it is parsed that the teaching plan information does not contain the original illustration information, extracting the target information in the teaching plan information; According to the teaching plan information, feature information is extracted, and a corresponding prompt template is selected from a prompt template library; Based on the prompt template, a lesson plan HTML file is generated through a large language model according to the target information of the lesson plan information.

10. The teaching plan animation generation method based on a large language model according to claim 1, characterized in that: The step: constructing an answer constraint condition according to the answer information, determining whether the lesson plan HTML file satisfies the answer constraint condition, and when the lesson plan HTML file does not satisfy the answer constraint condition, regenerating a corresponding lesson plan HTML file based on the prompt template and the target information of the lesson plan information through a large language model; When the lesson plan HTML file satisfies the answer constraint condition, the lesson plan HTML file is run to obtain a lesson plan animation, including: When the lesson plan html file does not meet the answer constraint condition, based on the prompt template and according to the target information of the lesson plan information, the corresponding lesson plan html file is regenerated through a large language model to determine whether the regenerated lesson plan html file meets the answer constraint condition. Until the lesson plan html file meets the answer constraint condition or reaches a preset number of code generation times, the judgment of the lesson plan html file based on the answer constraint condition is stopped, the lesson plan html file is run, and the lesson plan animation is obtained.