Teaching method for ancient poetry artistic conception based on open source model architecture

Abstract

The invention discloses an ancient poetry artistic conception teaching method based on an open source model architecture, and relates to the technical field of teaching. A word segmentation result after reconstruction is used for training a Word2Vec model, the semantic relation between words is captured, word vectors are obtained through training of the Word2Vec model, a deep learning framework TensorFlow is used for constructing a word vector sequence obtained through processing of a long-short-term memory network LSTM model, the LSTM utilizes a memory unit and a gate mechanism to capture the long-distance dependency relation in poems, and the long-distance dependency relation in the poems is obtained. Converting the poem data after word segmentation into word indexes, and constructing an input sequence and target words of the model; after training is completed, named entity recognition is carried out on unmarked ancient poetry texts, and recognized entities are classified and marked; the analysis result is visualized; a lightweight deep learning model is established to realize ancient poetry imagination and scenario, rationality and accuracy of model classification are improved, an ancient poetry atmosphere scene is created, and unity scene establishment and a user interaction mode are guided.

Description

Technical Field

[0001] The present invention relates to the field of intelligent teaching of ancient poetry, and in particular to a teaching method for the artistic conception of ancient poetry based on an open source model architecture. Background Art

[0002] At present, there are the following problems in the ancient poetry teaching method based on artificial intelligence: 1) The traditional teaching method of ancient poetry relies on the teacher's literal analysis in class, which lacks embodiment and context. It is difficult for students to fully understand and absorb both knowledge information and sensory information at the same time, and their learning motivation and enthusiasm are insufficient.

[0003] 2) Applying the recommendation algorithm based on deep learning methods to the ancient poetry interactive system requires a large amount of data and computing power while obtaining excellent learning capabilities. Ordinary CPUs can no longer meet the requirements of deep learning. The mainstream computing power uses GPUs and TPUs, which have high hardware requirements and are therefore very expensive. Secondly, because deep learning relies on data and is not highly interpretable, gender bias, discrimination, and other issues may arise when the training data is unbalanced.

[0004] 3) The current VR teaching of ancient poetry is limited to watching ancient poetry scenes, the interactive method is relatively simple, and the students' participation is insufficient.

[0005] 4) The existing AI-generated ancient poetry scenes do not construct the artistic conception of poetry and painting, and lack in-depth teaching guidance. Students are unable to fully integrate the picture scenes with the artistic conception of ancient poetry, and then comprehend the deep cognition in the artistic conception space of ancient poetry.

[0006] Therefore, a teaching method for the artistic conception of ancient poetry based on an open source model architecture is provided to solve the above problems. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a teaching method for the artistic conception of ancient poetry based on an open source model architecture, builds a lightweight deep learning model to realize the visualization and scenario of ancient poetry, improves the rationality and accuracy of model classification, creates an atmosphere scene of ancient poetry, and further guides the construction of Unity scenes and user interaction methods.

[0008] To achieve the above purpose, the present invention provides a teaching method for the artistic conception of ancient poetry based on an open source model architecture, which specifically includes the following steps: Step S1: using natural language processing (NLP) to pre-process ancient poems, including text cleaning, word segmentation, part-of-speech tagging, and stop word removal; Step S2: Train a Word2Vec model using the reconstructed word segmentation results. The Word2Vec model captures the semantic relationships between words, and word vectors are obtained through training the Word2Vec model, including two model structures: CBOW and Skip-gram; Step S3: Use the deep learning framework TensorFlow to construct a long short-term memory network (LSTM) model to process the word vector sequence obtained in Step S2. The LSTM uses its memory units and gate mechanisms to capture long-distance dependencies in ancient poems, convert the segmented poem data into word indices, and construct the input sequence and target words of the model; Step S4: Identify and classify concrete content. Concrete content is the knowledge information presented specifically. The classification criteria are based on images, behaviors, and states. After completing corpus annotation and model design, train the NER model; Step S5: After training, perform named entity recognition on unlabeled ancient poem texts and classify and label the recognized entities; Step S6: Identify abstract concepts, which are the knowledge information that cannot be directly presented in Step S4. Use BERT-CCPoem to convert each poem sentence into a vector representation, utilize the similarity of sentence vectors to analyze the relevance of images, and identify metaphors and similes by comparing the vectors of different sentences; Step S7: Visualize the analysis results, use a heatmap to display the similarity matrix, and represent the high and low similarity by the depth of color; Step S8: Create the atmosphere of ancient poem scenes, collect reference images and use Stable Diffusion to generate single-scene images; Step S9: Build an image-guided ancient poem scene and complete the interaction between players and the ancient poem scene.

[0009] Preferably, in Step S2, the training of the Word2Vec model specifically includes the following steps: S21: Use the gensim library in Python for training and set the following key parameters: vector_size: The dimension supported by the word, and the dimension is set to 100 or 200; window: The size of the sliding window, and generally the window size is set to 3 - 5; min_count: The minimum word frequency, and words with occurrences lower than this will be ignored; sg: 0 represents the CBOW model, 1 represents the Skip-gram model, and the Skip-gram model is suitable for small corpora; S22: Train the Word2Vec model: model = Word2Vec(sentences, vector_size=100, window=3, min_count=1, sg=1); View word vectors: print("Word vector of '·':", model.wv["·"]); S23: After training is completed, assign instructions to the words in the sentence to obtain the representation of the entire sentence, and convert the orange or oral form into an expression; S24: Through the word vectors obtained by training with the Word2Vec model, expand the calculation of word similarity, semantic analysis of word meanings, clustering, and subject analysis.

[0010] Preferably, in step S4, the images include people, scenery, and animals; the behaviors include independent behaviors and non-independent behaviors; the states include adjectives and adverbs.

[0011] Preferably, the specific applications in step S5 include using the trained NER model to identify the images in ancient poems, identify the behaviors and states described by the poets, count the images and emotion words appearing in ancient Chinese poems, generate a word cloud chart, analyze the common images and emotion themes of the poets, and analyze the identified images, behaviors, and states to discover the common themes in ancient poems.

[0012] Preferably, in step S8, creating the atmosphere of ancient poem scenes specifically includes optimizing text prompt words, collecting reference images, and generating single ancient poem scene images; optimizing text prompt words includes preliminary mood paving, picture logic arrangement, and precise restraint expression; The preliminary mood paving includes adding a description of the background story of the image to the prompt words to enable the AI to understand the emotional tone behind the picture, describing the emotional state and facial expressions of the characters, and adding multiple sensory elements to the prompt words. The sensory elements include visual elements, auditory elements, and tactile elements; In the visual elements, describe the scene with light, shadow, and color tone; in the auditory elements, add sound elements; in the tactile elements, express the temperature and texture of the scene with words; The picture logic arrangement includes the viewing order, parallel perspectives, guiding the line of sight, and focusing on details; In the viewing order, guide the line of sight by describing the layout of the picture; in the parallel perspectives, clarify the positions and relationships of different elements in the picture; in guiding the line of sight, add elements that guide the line of sight; in focusing on details, set static details at key positions; The precise restraint expression includes turning the abstract into the concrete, turning the static into the dynamic, and detail description; turning the abstract into the concrete makes the abstract emotions concrete; turning the static into the dynamic gives dynamic to static objects; detail description adds detailed descriptions.

[0013] Preferably, in the collection of reference images in step S8, a text-image retrieval model is used to automatically retrieve reference images that meet the description from the image database according to the prompt words; specifically, it includes the following steps: S81: Prepare a large image database, such as the public datasets LAION, Unsplash, or Getty Images; S82: Use the multi-modal model CLIP with text-image matching ability to encode the prompt words, and at the same time encode all the images in the database to enable the computer to understand the semantic association between the text and the images; S83: Calculate the similarity between the prompt words and each image, and select several images with the highest similarity as reference images.

[0014] Preferably, the specific steps for generating a single-scene image using Stable Diffusion are as follows: Step 1: Generate an edge map or a depth map based on the reference image, load the ControlNet plugin in Stable Diffusion, input the edge map or depth map of the reference image, and set the control intensity of ControlNet, where the control intensity is 0.5 - 0.8; Step 2: Generate an image in cooperation with the prompt words, and ControlNet controls the overall structure of the generated image according to the edge or depth map; Step 3: ControlNet maintains a specific image structure during the generation process to ensure that the picture composition meets the expectations; If the generated picture has a specific character pose or scene layout, use the pose detection OpenPose or edge detection Canny of ControlNet; If there is a sense of layering or depth in the ancient poetry atmosphere, use the depth map generated by the depth estimation model MiDaS to control the sense of space of the image.

[0015] Preferably, in step S9, building an image-guided ancient poetry scene includes a direct scene and an indirect scene, and the direct scene includes texture mapping and environment mapping.

[0016] Preferably, in step S9, the interaction between the player and the ancient poetry scene includes building a teaching system, character teleportation, click interaction, grab interaction, ray interaction, visual interaction, and emotional interaction; The process of building the teaching system includes initial contact and perception, emotional experience and resonance, rational thinking and understanding, knowledge consolidation and application, emotional internalization and expression, rational integration and innovation, and the integration of knowledge and emotion.

[0017] Preferably, during the initial contact and perception process, it includes virtual enrollment, an overview of ancient poems, and basic recitation; During the emotional experience and resonance process, it includes emotional immersion, role-playing, and interactive learning; During the rational thinking and understanding process, it includes clue exploration, logical reasoning, and creative practice; During the knowledge consolidation and application process, it includes scenario review and knowledge application; During the emotional internalization and expression process, it includes emotional resonance and emotional expression; During the rational integration and innovation process, it includes clue integration and innovative thinking; During the integration process of knowledge and emotion, it includes knowledge review and testing, and emotional experience sharing.

[0018] Therefore, the present invention adopts the above-mentioned teaching method for the artistic conception of ancient Chinese poems based on an open-source model architecture, constructs a lightweight deep learning model to realize the visualization and scenarization of ancient Chinese poems, improves the rationality and accuracy of model classification, creates an atmosphere scene of ancient Chinese poems, and further guides the construction of unity scenes and user interaction methods.

[0019] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0020] Figure 1 It is a schematic diagram of a teaching method for the artistic conception of ancient Chinese poems based on an open-source model architecture of the present invention. Detailed Embodiments

[0021] The technical solution of the present invention will be further described below with reference to the drawings and embodiments.

[0022] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the field to which the present invention belongs.

[0023] In the present invention, words such as "including" or "comprising" and the like mean that the elements before this word cover the elements listed after this word, and do not exclude the possibility of also covering other elements. The orientation or positional relationship indicated by terms such as "inside", "outside", "above", "below", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, terms such as "attachment" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be directly connected, or indirectly connected through an intermediate medium, and can be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0024] Embodiment A teaching method for the artistic conception of ancient Chinese poems based on an open-source model architecture specifically includes the following steps: Step S1: Use natural language processing NLP to perform preprocessing on the ancient Chinese poem corpus. The preprocessing includes text cleaning, word segmentation, part-of-speech tagging, and stop word removal; Clean the ancient Chinese poem text, remove unnecessary punctuation marks, special characters, and redundant spaces, etc., and execute through the following operations: import re def clean_text(text): # Remove punctuation marks and special characters text = re.sub(r'[^\w\s]', '', text) # Remove redundant spaces and line breaks text = re.sub(r'\s+', ' ', text).strip() return text.

[0025] Use an ancient Chinese word segmentation tool to segment the cleaned text, and execute through the following operations: from hanlp import HanLP def segment_text(text): tokenizer=HanLP.load(hanlp.pretrained.tok.CTB9_POS_RNN_FASTTEXT_ZH) words = tokenizer(text) return [word for word, _ in words].

[0026] Perform part-of-speech tagging on the word segmentation results by the following operations: def pos_tagging(text): postagger= HanLP.load(hanlp.pretrained.pos.CTB9_POS_ALBERT_BASE) words = postagger(text) return [(word, tag) for word, tag in words].

[0027] Create a stop word list for ancient poetry and remove stop words by doing the following: stopwords = set(["之", "乎", "者", "也"]) # Example stop word list def remove_stopwords(words): return [word for word in words if word not in stopwords].

[0028] Step S2: Use the reconstructed word segmentation results to train the Word2Vec model. The Word2Vec model captures the semantic relationship between words. The word vector is obtained through the Word2Vec model training, including two model structures: CBOW and Skip-gram. In step S2, the training Word2Vec model mainly includes two model structures: CBOW and Skip-gram. Taking Red Cliff Fu as an example, the specific steps include: S21: Use Python's gensim library for training and set the following key parameters: vector_size: the dimension supported by the word, the dimension is set to 100 or 200; Window: sliding window size, generally the window size is set to 3~5; min_count: minimum word frequency, words with a lower frequency will be ignored; sg: 0 means CBOW model, 1 means Skip-gram model, Skip-gram is suitable for small corpus; S22: from gensim.models import Word2Vec # Assume the processed corpus format is as follows sentences = ['Renxu', 'zhi', 'qiu'], ['Qiyue', 'ji', 'wang'], ['Suzi', 'yu', 'ke', 'fanzhou', 'you', 'yu', 'Chibi', 'zhi', 'xia'] ; Train the Word2Vec model: model = Word2Vec(sentences, vector_size = 100, window = 3, min_count = 1, sg = 1); View the word vector: print("The word vector of 'Chibi': ", model.wv["Chibi"]); S23: After training is completed, assign instructions to the words in the sentence to obtain the representation of the entire sentence, and convert the sentence or speech into an expression; Define a function: Convert a sentence into a vector def sentence_vector(sentence, model): words = list(jieba.cut(sentence)) word_vecs = [model.wv[word] for word in words if word in model.wv] Take the average value as the vector representation of the sentence return sum(word_vecs) / len(word_vecs) if word_vecs else [] Example: Convert a sentence into a vector sentence_vec = sentence_vector("Suzi yu ke fanzhou you yu Chibi zhi xia", model) print("Sentence vector: ", sentence_vec).

[0029] S24: Through the word vectors obtained by training with the Word2Vec model, carry out similarity calculation between words, semantic analysis of word meanings, clustering, and subject analysis, and carry out the following analysis applications: Calculating the similarity between words, calculating the similarity between the two words "Red Cliff" and "river water" in "Ode to the Red Cliff": similarity = model.wv.similarity("Red Cliff", "river water")print("The similarity between 'Red Cliff' and 'river water':", similarity); Semantic relationship analysis, through the provided words, similar words of a certain word can be found. For example, finding words with similar semantics to "Red Cliff": similar_words = model.wv.most_similar("Red Cliff", topn=5)print("Words similar to 'Red Cliff':", similar_words); Clustering and topic analysis, by thinking about each sentence, first performing Kui analysis to accurately infer similar sentences. For example, aggregating sentences related to "rowing a boat" into one category and sentences related to "moonlight" into another category.

[0030] Step S3: Use the deep learning framework TensorFlow to build a long short-term memory network (LSTM) model to process the word vector sequence obtained in Step S2. The LSTM uses its memory units and gate mechanisms to capture long-distance dependencies in the poem; import numpy as np import tensorflow as tf from tensorflow.keras.models import Sequential from tensorflow.keras.layers import LSTM, Dense, Embedding Obtain the embedding matrix word_index = {word: i for i, word in enumerate(model.wv.index_to_key)} Word-index dictionary embedding_matrix = model.wv.vectors Word vectors in the Word2Vec model LSTM model parameters seq_length = 5 Input sequence length embedding_dim = embedding_matrix.shape[1] Word vector dimension vocab_size = embedding_matrix.shape[0] Vocabulary size Build the model model = Sequential( Embedding(input_dim=vocab_size, output_dim=embedding_dim, weights=[embedding_matrix], input_length=seq_length, trainable=False), LSTM(128, return_sequences=True), The first layer of LSTM LSTM(128), The second layer of LSTM Dense(vocab_size, activation='softmax') Output layer, predicting the probability distribution of the next word]) model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy']) model.summary().

[0031] Convert the tokenized poem data into word indices, and construct the input sequence and target word of the model; def prepare_data(sentences, word_index, seq_length): X = [] y = [] for sentence in sentences: Convert the sentence into word indices words = [word_index[word] for word in sentence if word in word_index] Construct the input sequence and target word for i in range(len(words) - seq_length): X.append(words[i:i + seq_length]) y.append(words[i + seq_length]) return np.array(X), np.array(y) Prepare data X, y = prepare_data(sentences, word_index, seq_length) Train the model model.fit(X, y, epochs=20, batch_size=64).

[0032] Step S4: Identify and classify concrete content. The concrete content is the knowledge information presented specifically. The classification criteria are based on images, actions, and states. After completing the corpus annotation and model design, train the NER model; In step S4, the images include people (such as hermits, wanderers, warriors, etc.), landscapes (such as mountains and waters, clouds and mists, pavilions and towers, etc.), animals (such as cuckoos, swans, apes, etc.); the actions include independent actions (such as pacing, wandering, looking back, etc.) and non-independent actions (such as strolling, lingering, leaning, etc.); the states include adjectives (such as secluded, solitary, thick, green, etc.) and adverbs (such as still, yet, also, gradually, etc.).

[0033] Step S5: After training, perform named entity recognition on the unlabeled ancient poetry texts and classify the recognized entities; The specific applications in step S5 include using the trained NER model to identify the images in the poems, identify the actions and states described by the poets, count the images and emotion words that appear in the ancient poems, generate a word cloud chart, analyze the common images and emotion themes of the poets, and analyze the identified images, actions, and states to discover the common themes in the poems.

[0034] Identify the images in the poems and use the trained NER model to identify the common images (such as mountains, waters, flowers, birds, etc.) in the ancient poems. Example: "The river flows beyond the bounds of heaven and earth, and the mountain color is hazy" → Identify images such as "river flow", "heaven and earth", "mountain color".

[0035] Analysis of actions and states. Use the model to identify the actions (such as "thinking", "looking", "drinking", "being drunk", etc.) and states (such as "sorrow", "solitary", "cold", etc.) described by the poets to help understand the poets' state of mind at that time. Example: "When life is good, one should enjoy to the full; don't let the golden goblet face the moon in vain" → Identify "being good" (state), "joy" (emotion), "facing in vain" (action).

[0036] Generate word clouds of images and emotions. It is possible to count the images and emotion words that appear in the ancient poems and generate a word cloud chart to help analyze the common images and emotion themes of a certain poet.

[0037] Clustering and theme analysis are performed on the identified images, behaviors, states, etc. to discover common themes in poems. For example, image-related verses such as "mountains and waters" and "wind and moon" are classified into the "natural theme", and emotion-related ones such as "sorrow" and "homesickness" are classified into the "theme of parting sorrow".

[0038] Step S6: Identify abstract concepts. For the knowledge information that cannot be directly presented in Step S4, each poem sentence is transformed into a vector representation through BERT-CCPoem. The knowledge information that cannot be directly presented includes the following categories: The abstract meaning behind the image (e.g., "partridge" implies parting, sorrow of parting, love, and yearning); Metaphors and similes (e.g., in Li Qingzhao's "Ru Meng Ling", "remaining wine" is used to metaphorize the unvanished sorrow); Abstract concepts and philosophies (e.g., in Chen Ziang's "Climbing the Youzhou Tower Song", it contains the exploration of the meaning of life); Cultural and historical backgrounds (e.g., in "Throughout history, no hero like Sun Zhongmou can be found on this magnificent land", the author's nostalgic feeling for the hero Sun Quan in the Three Kingdoms period); The personal experiences of the poet (e.g., Li Yu's "Yu Mei Ren"), rhyme and rhythm (e.g., Du Fu's "Spring View").

[0039] Transform each poem sentence into a vector representation through BERT-CCPoem: def get_sentence_vector(sentence): inputs=tokenizer(sentence,return_tensors='pt',padding=True,truncation=True) with torch.no_grad(): outputs = model( ) Obtain the vector of [CLS] as the sentence representation sentence_vector = outputs.last_hidden_state[:, 0, :] return sentence_vector Generate the vector representation of each poem sentence poem_vectors = [get_sentence_vector(line) for line in text].

[0040] Analyze the relevance of images by using the similarity of sentence vectors. By comparing the vectors of different sentences, identify the metaphors and similes therein; identify how the images of "moonlight" and "river water" in the sentences are related to the overall artistic conception of the poem by analyzing the similarity.

[0041] from sklearn.metrics.pairwise import cosine_similarity; Calculate the similarity between sentences: similarities = cosine_similarity(torch.cat(poem_vectors).numpy()); print("Similarity matrix:\n", similarities); Combined with historical background knowledge, identify cultural elements through cluster analysis or topic analysis. Cluster analysis can help us identify the cultural backgrounds of different images or themes in ancient Chinese poems.

[0042] from sklearn.cluster import KMeans; Merge the vectors for clustering: X = torch.cat(poem_vectors).numpy(); kmeans = KMeans(n_clusters=2, random_state=0).fit(X); print("Sentence clustering results:", kmeans.labels_).

[0043] Step S7: Visualize the analysis results. Use a heatmap to display the similarity matrix, and represent the high or low similarity by the depth of color; through the heatmap, it can be intuitively seen which sentences have a high similarity and which sentences have obvious image differences. Analyzing the images conveyed by sentences with high similarity can help understand the theme and style of ancient Chinese poems.

[0044] import matplotlib.pyplot as plt import seaborn as sns Visualize the similarity matrix plt.figure(figsize=(10, 8)); sns.heatmap(similarities, annot=True, cmap='coolwarm', xticklabels=text, yticklabels=text); plt.title("Sentence Similarity Matrix"); plt.show();

[0045] Step S8: Create the atmosphere of ancient Chinese poetry scenes, collect reference images and use Stable Diffusion to generate single-scene images; In step S8, creating the atmosphere of ancient Chinese poetry scenes specifically includes text prompt optimization, reference image collection, and generation of single ancient Chinese poetry scene images; text prompt optimization includes early emotional foreshadowing, picture logic arrangement, and precise restraint expression; Early emotional foreshadowing includes adding descriptions of the background story of the image to the prompt to enable the AI to understand the emotional tone behind the picture (for example, if you want to express the emotion of "homesickness", you can add elements such as "distant mountains", "setting sun glow", or "lonely traveler"), describing the emotional state and facial expressions of the characters (such as "with a hint of sadness", "a poet in deep thought", etc. to strengthen the emotions of the characters in the picture), and adding multiple sensory elements to the prompt. Sensory elements include visual elements, auditory elements, and tactile elements; In visual elements, describe the scene with light, shadow, and color tone (such as "early morning mist", "soft light of the autumn sunset"); in auditory elements, add sound elements (such as "gentle breeze caressing the willows" or "gurgling stream"); in tactile elements, use words to express the temperature and texture of the scene (such as "slightly cool night breeze", "fine sand beach"); Picture logic arrangement includes viewing order, parallel perspectives, guiding the line of sight, and focusing on details; In the viewing order, guide the line of sight by describing the layout of the picture (for example, from the large scene to the details, you can use "on the vast plain, there are small pavilions and towers in the near distance" to construct the hierarchy); in parallel perspectives, clarify the positions and relationships of different elements in the picture (for example, "on the left is an ancient pine tree, and on the right is a distant green mountain"); contrast between dynamic and static: add elements that guide the line of sight in guiding the line of sight (such as "fluttering petals" or "reflection in the water"); set static details at key positions in focusing on details (such as "isolated lantern", "person gazing into the distance", etc. to focus the picture's focus); Precise restraint in expression includes turning the abstract into the concrete, turning the static into the dynamic, and detailed portrayal; in turning the abstract into the concrete, abstract emotions are made concrete (for example, "sorrow" is specifically manifested as "a young girl lost in thought with her head down"); in turning the static into the dynamic, static objects are given motion (such as "a gentle breeze caressing the willow branches" or "a flickering candle flame. Such details can add vitality and atmosphere to the picture"); in detailed portrayal, detailed descriptions are added (such as "ancient trees covered with moss" and "ripples on the water surface", through these details to increase the sense of reality and atmosphere).

[0046] In the collection of reference images in step S8, a text-image retrieval model is used to automatically retrieve reference images that match the description from the image database according to the prompt; specifically, it includes the following steps: S81: Prepare a large image database, such as the public datasets LAION, Unsplash, or Getty Images; S82: Use the multi-modal model CLIP (Contrastive Language-Image Pretraining) with the ability of text-image matching to encode the prompt, and at the same time encode all the images in the database, enabling the computer to understand the semantic relationship between the text and the image; S83: Calculate the similarity between the prompt and each image, and select several images with the highest similarity as reference images.

[0047] Generation of a single ancient poetry scene image Painting style selection First, understand the creation background of the ancient poetry, conduct an in-depth analysis of the content of the ancient poetry, and understand its theme, emotion, historical background, etc. Study the cultural characteristics of the ancient poetry creation period, including clothing, architecture, natural scenery, etc. Then collect paintings that match the ancient poetry creation period, which will be used as style references, and digitally process the collected paintings, including image cropping, scaling, denoising, etc.

[0048] Use image processing technology to extract the main colors and tones in the paintings, and use deep learning models (such as convolutional neural network CNN) to extract the style features of the paintings, including brushstrokes, textures, compositions, etc. Select a suitable style conversion model (such as CycleGAN or Shuffle ControlNet). Use deep learning technology to train the model so that it can convert ordinary images into images with a specific era style. Input the scene image described by the ancient poetry into the trained model to generate a stylized image. Manually adjust the details of the generated image as needed to better conform to the artistic conception of the ancient poetry.

[0049] Definition of the scene corresponding to the poem In the process of defining the corresponding scenes of the verses, the composition of the scenery (i.e., time, place, people, scenery and their characteristics) and the characteristics of the picture (i.e., the characteristics or types of artistic conception, such as lonely and cold, quiet and beautiful, grand and magnificent, etc.) are used as the judgment basis.

[0050] Determine the time (such as season, time) and place (such as mountains and waters, cities, villages) described in the verses, identify the people and their actions or states (such as poets, travelers, fishermen, etc.) appearing in the verses, describe the natural landscapes (such as mountains, waters, flowers, grasses) and buildings (such as pavilions, towers, bridges) in the verses, extract the key scenery characteristics from the verses, such as colors, shapes, dynamics, etc., and reasonably combine the extracted elements according to the description of the verses to construct a complete scene, so as to realize the division of the scenes depicted in ancient Chinese poems.

[0051] Using Stable Diffusion to generate single-scene images specifically includes the following steps: Step 1: Generate an edge map or a depth map according to the reference image, load the ControlNet plug-in in Stable Diffusion, input the edge map or depth map of the reference image, and set the control intensity of ControlNet, and the control intensity is 0.5 - 0.8; Step 2: Generate an image in cooperation with the prompt words, and ControlNet controls the overall structure of the generated image according to the edge or depth map; Step 3: ControlNet maintains a specific image structure during the generation process to ensure that the picture composition meets the expectations; If the generated picture has a specific human pose or scene layout, use the pose detection OpenPose or edge detection Canny of ControlNet; If the ancient Chinese poem has a sense of atmosphere hierarchy or depth, use the depth map generated by the depth estimation model MiDaS to control the sense of space of the image.

[0052] Step S9: Build an image-guided ancient Chinese poem scene and complete the interaction between the player and the ancient Chinese poem scene.

[0053] In step S9, building an image-guided ancient Chinese poem scene includes a direct scene and an indirect scene, and the direct scene includes a texture map and an environment map.

[0054] Texture Map: AI can generate or optimize materials and textures, which can be directly applied to 3D models in Unity. It can quickly test and adjust the material effects, improving the realism and detail richness of the scene. Create a new material in Unity and apply the AI image as the Albedo (color texture) to the material. Import the image into Unity and then drag it into the Albedo slot of the material to complete. To enhance realism, use physically based rendering (PBR) materials. Perform necessary processing on the AI-generated image, such as adjusting the resolution and color correction. The AI image can provide maps such as color (Albedo), normal, roughness, metallic, and ambient occlusion, which can be adjusted through the PBR material settings in Unity to simulate the lighting and surface characteristics of the real world, increasing the realism and artistry of the scene.

[0055] Environment Map: AI can generate environment maps and skyboxes, providing a realistic background and ambient atmosphere for Unity scenes. These images can be used as the background or reflection map of the scene, enhancing the immersion of the scene. In the Inspector window, find the imported image and set its Texture Import Settings. Set the Texture Type to "Cubemap" and select appropriate Cubemap settings, such as "Auto Cubemap" or "From Six Strips". Apply the material to the corresponding object in the scene. For example, you can create an empty GameObject and add a "Skybox" component, then drag the material to the Skybox Material property of this component. In this way, the entire scene will use this environment map. To make the effect of the environment map more realistic, it may be necessary to adjust the lighting settings in the scene. Use "Lighting > Baked Indirect" or "Lighting > Realtime Global Illumination" to simulate the ambient light and make the reflection on the object surface more natural.

[0056] Indirect Use AI-generated ancient poetry artistic conception scene images can be used as concept maps for scene construction, referring to scene layout, color matching, lighting effects, and atmosphere, etc., to help quickly construct an ancient poetry artistic conception space in Unity. These images provide intuitive visual references and reduce the time from concept to implementation. The lighting effects in AI images can guide the lighting settings in the Unity scene. Analyze the light source direction, intensity, color, and the shape, length, and hardness of shadows in the image, set the main light source (such as directional light, point light, or spotlight) in Unity, adjust the resolution and quality of shadows, and set the time controller in Unity according to the time in the image (such as sunrise, sunset) to simulate the lighting effects at different times. This helps create a lighting atmosphere that conforms to the emotions of ancient poetry, such as a peaceful moonlit night, hazy morning mist, or gorgeous sunset, etc.

[0057] In step S9, the construction process of the teaching system includes initial contact and perception, emotional experience and resonance, rational thinking and understanding, knowledge consolidation and application, emotional internalization and expression, rational integration and innovation, and the integration of knowledge and emotion.

[0058] During the initial contact and perception process, it includes virtual enrollment, poetry overview, and basic recitation. Virtual enrollment: Players enter the virtual Chibi Academy and feel the living environment and state of mind of Su Shi when he was demoted to Huangzhou back then.

[0059] Poetry overview: Players browse the table of contents of "The Red Cliff Rhapsody" on the virtual bookshelf. By touching the pages, they can hear Su Shi's voice reciting the verses and feel the changes in his emotions.

[0060] Basic recitation: Players learn the recitation skills of Su Shi in "The Red Cliff Rhapsody" and simulate Su Shi's intonation and rhythm through VR equipment to experience the ups and downs of his emotions.

[0061] During the emotional experience and resonance process, it includes emotional immersion, role-playing, and interactive learning. Emotional immersion: Players experience scenes such as boating on the moonlit night, surging river water, and burning warships described in "The Red Cliff Rhapsody" by Su Shi through the changes in the VR environment, and feel his sighs about the cruelty of war and the suffering of the people.

[0062] Role-playing: Players act as Su Shi and simulate the boating scene after the Battle of Red Cliff through VR to feel his philosophical thoughts on the brevity of life and the eternity of the universe.

[0063] Interactive learning: Players interact with the environment in the 3D scene, such as simulating rowing a boat and touching the river water, to experience the natural beauty and the tragedy of war described in "The Red Cliff Rhapsody" by Su Shi.

[0064] During the rational thinking and understanding process, it includes clue exploration, logical reasoning, and creative practice. Clue Exploration: Players collect clues related to "The Red Cliff Rhapsody" in a 3D scene, such as Su Shi's manuscripts and historical documents, and explore its creation background and ideological changes.

[0065] Logical Reasoning: Players analyze the clues and infer Su Shi's profound understanding of the Battle of Red Cliffs and his philosophical thinking about life in "The Red Cliff Rhapsody".

[0066] Creative Practice: Players use virtual writing brushes and ink to try to create their own poems, practicing the literary techniques and emotional expressions demonstrated by Su Shi in "The Red Cliff Rhapsody".

[0067] During the process of knowledge consolidation and application, it includes scene review and knowledge application; Scene Review: Players review the learning process of "The Red Cliff Rhapsody" through the VR system, strengthening their understanding and memory of Su Shi's emotional changes and philosophical thinking.

[0068] Knowledge Application: Players apply the knowledge they have learned in a virtual environment, such as creating new poems in the scene of the Battle of Red Cliffs to express their thoughts on war and peace.

[0069] During the process of emotional internalization and expression, it includes emotional resonance and emotional expression; Emotional Resonance: Players further experience Su Shi's emotional changes by reviewing the records on the scroll, achieving emotional internalization.

[0070] Emotional Expression: Players share their learning experiences and emotional changes in a virtual community, communicating with other players about their insights into "The Red Cliff Rhapsody".

[0071] During the process of rational integration and innovation, it includes clue integration and innovative thinking; Clue Integration: Players integrate the clues collected on the scroll to form a comprehensive understanding of "The Red Cliff Rhapsody".

[0072] Innovative Thinking: Players conduct innovative thinking in a virtual environment, such as combining the elements of "The Red Cliff Rhapsody" with modern elements to create new works.

[0073] During the process of the integration of knowledge and emotion, it includes knowledge review and testing, and emotional experience sharing.

[0074] Knowledge Review and Testing: Players consolidate their understanding and memory of "The Red Cliff Rhapsody" through the tests provided by the system. The test content covers Su Shi's emotional changes and philosophical thinking. Emotional Experience Sharing: Players share their learning experiences and emotional changes, and through community interaction, achieve the integration of knowledge and emotion.

[0075] The interaction between players and the ancient poetry scene includes teaching system construction, character teleportation, click interaction, grab interaction, ray interaction, visual interaction, and emotional interaction; Character teleportation In the game, the player pushes the joystick of the controller forward and then releases it to achieve character teleportation. At the same time, some hot events are added on the ground for teleportation guidance. Implementation method: Hang the Locomotion Controller Interactor Group prefab in the official Unity library on the Controller Hand Interactors, find the Best Hover Interactor Group of the Controller Interactors, and drag the just-added child object Controller Ray Interactor into the list bar to replace the automatically added object. At the same time, create an empty object in the OVRPlayerController, name it Locomotion, and then add the official Unity script Player Locomotor. Open the Navigation panel and bake the ground where the teleportation function needs to be implemented.

[0076] Click interaction is achieved through controller interaction. Implementation method: The implementation of click interaction requires two steps. Add the Controller Hand Interactors in the OVRPlayerController to the Hand Poke Interactor classes of the left and right hands as child objects respectively to handle hand poking interaction. This interaction method allows users to penetrate or touch objects in the virtual environment with their hands. It can perform hand penetration detection, penetration interaction event processing, and parameter configuration. Subsequently, create and configure a Poke Interactable object on the page where the interaction is required, and add a Pointable Unity Event Wrapper component to it to handle finger contact events.

[0077] Grab interaction is achieved through the trigger button and the handshake button respectively. Implementation method: On the hand model, add the Controller Hand Interactors in the OVRPlayerController to the Controller Grab Interactor classes of the left and right hands as child objects respectively for item grabbing interaction. For the item to be interacted with, first create a blank object, add the object visual with a collider and the official Unity GrabInteractable prefab under the object to build the framework of the interacted object. At the same time, attach a Rigid body rigidbody, a Physics Grabbable script, and a Pointable Unity Event Wrapper script to the object itself to achieve the feedback of interaction events.

[0078] During the experience process of ray interaction, the handle ray needs to be used for page interaction. The implementation method is as follows: On the hand model, in the OVRPlayerController, the ray controllers of the Controller Ray Interactor representing the handle ray are respectively added as child objects under the Controller Interactors of the left and right hands. For the page that needs to be interacted with, after creating a blank object, add an Event System object as a child object and add a PointableCanvasModule script. After adding the canvas child object, the Ray Interactable script, Pointable Canvas script, and Pointable Canvas Unity Event Wrapper script need to be added as the main scripts to be used.

[0079] Visual interaction includes the determination of the scene art style and particle effects; Regarding the determination of the scene art style for the ancient poetry artistic conception space scene, for the unified visual effect, the principle of maintaining the same style and model type for multiple scenes under the same ancient poem should be adhered to. In addition, due to the large number and size of the models required for the scene, in order to reduce the resource consumption of the program, the LOD (Level of Detail) optimization technology is used in the scenery to improve game performance and reduce resource consumption. LOD reduces the level of detail by using simpler models or textures on objects far from the camera, thereby reducing the rendering overhead. A semi-transparent hand model is also selected for the player's hand model to enhance the player's immersion.

[0080] Particle effects: In the scene, the particle system is used to enhance the atmosphere and emotional expression of the scene to provide a better gaming experience and reduce external human voice interference. By adding particle systems such as snowfall and falling leaves to the scene, for example, the snowfall effect can be achieved by setting the "Size over Lifetime" and "Color over Lifetime" modules of the particle system to change the size and color of the snowflakes, simulating the scene of snowflakes falling and increasing the realism and dynamic beauty of the scene. Create effects such as smoke and clouds to add a mysterious and hazy atmosphere to the scenes of mountains, waters, pavilions, etc. in ancient poems. The particle system can also change dynamically according to the user's interaction, such as the dust raised when the player character moves and the ripples generated when touching the water surface. For special scenes described in ancient poems, such as fairylands and illusions, the "Particle System Curves Editor" can be used to control the color and transparency changes of the particles to create effects such as halos and phantoms.

[0081] For auditory interaction, according to the scenes and atmospheres of ancient poems, background sound effects of natural environments can be added, such as the sound of flowing water, wind, bird chirping, etc. These sounds can be played in a loop to create a peaceful or profound artistic conception; record or synthesize the recitation audio of ancient poems and play the recitation of poems when players enter specific scenes or trigger certain events to enhance the poetry experience and memory; according to the player's behavior or dynamic elements in the scene, such as when the player walks on different materials of the ground (stone slabs, grasslands, water surfaces), different footsteps can be added to increase the realism of the scene; add sound effects to the interactive elements in the scene, such as the creaking sound of opening a door, the sound of striking an ancient bell, etc. These sound effects can provide feedback and enhance interactivity. To make the sound effects more natural, fade-in and fade-out effects can be used to avoid abrupt starts and endings. To optimize performance, dynamic loading technology can be used to load and unload audio resources as needed, such as using Resources.Load or UnityWebRequestMultimedia to load external audio files.

[0082] Emotional interaction includes conversations with NPCs and multi-perspective interaction; Conversations with NPCs. In the scene, the conversations with NPCs are mainly realized through UI dialogue scripts and ray processing of the gamepad. In the conversations with NPCs, it mainly plays a role in guiding the player forward, while through conversations with key characters, it can trigger spiritual resonance, promote deep immersion, and help the player understand the truth in ancient poems, serving as a point of topic and theme sublimation. Add triggers (Colliders) to NPCs. When the player enters the trigger area, the dialogue system is called through the script to display the dialogue content. The OnTriggerEnter and OnTriggerExit scripts can be used to detect the player's entry and exit and start or end the conversation.

[0083] Multi-perspective interaction. During the experience, in addition to the first-person perspective of the player himself, multiple perspectives are added during exploration to interact with nature. For example, create a camera that simulates the perspective of wild geese flying, and set its position and orientation to simulate the perspective of wild geese, and write scripts to control the switching between different cameras. To make the perspective transition smoother, Unity's animation system or Lerp (linear interpolation) can be used to smoothly transition the position and rotation of the camera. Feeling different perspectives can give players a rest and buffer stage, and at the same time make players more immersed in the environment of ancient poems, making players become a part of the artistic conception space.

[0084] Therefore, the present invention adopts the above-mentioned teaching method for the artistic conception of ancient poems based on an open-source model architecture, constructs a lightweight deep learning model to realize the visualization and sceneization of ancient poems, improves the rationality and accuracy of model classification, creates a scene with the atmosphere of ancient poems, and further guides the construction of unity scenes and user interaction methods.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements do not make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A teaching method for the artistic conception of ancient poetry based on an open source model architecture, characterized by: The specific steps include: Step S1: using natural language processing (NLP) to pre-process ancient poems, including text cleaning, word segmentation, part-of-speech tagging, and stop word removal; Step S2: Use the reconstructed word segmentation results to train the Word2Vec model. The Word2Vec model captures the semantic relationship between words. The word vector is obtained through the Word2Vec model training, including two model structures: CBOW and Skip-gram. Step S3: Use the deep learning framework TensorFlow to build a long short-term memory network LSTM model to process the word vector sequence obtained in step S2. LSTM uses its memory units and gates to capture the long-distance dependencies in the poem, converts the segmented poem data into word indexes, and constructs the model's input sequence and target words. Step S4: Identify and classify concrete content. Concrete content is the specific knowledge information displayed. The classification standard is based on imagery, behavior and status. After completing corpus annotation and model design, train the NER model. Step S5: After the training is completed, named entity recognition is performed on the unlabeled ancient poetry text, and the recognized entities are classified and labeled; Step S6: Identify abstract concepts. The knowledge information that cannot be directly displayed in step S4 is converted into vector representations by BERT-CCPoem. The similarity of sentence vectors is used to analyze the relevance of images. By comparing the vectors of different sentences, metaphors and similes are identified. Step S7: Visualize the analysis results and display the similarity matrix using a heat map, with the depth of color indicating the degree of similarity; Step S8: Create an ancient poetry scene atmosphere, collect reference images and use Stable Diffusion to generate a single scene image; Step S9: Build an image-guided ancient poetry scene and complete the interaction between the player and the ancient poetry scene.

2. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 1, it is characterized by: In step S2, training the Word2Vec model specifically includes the following steps: S21: Use Python's gensim library for training and set the following key parameters: vector_size: the dimension supported by the word, the dimension is set to 100 or 200; Window: sliding window size, usually set to 3 to 5; min_count: minimum word frequency, words with a lower frequency will be ignored; sg: 0 means CBOW model, 1 means Skip-gram model, Skip-gram is suitable for small corpus; S22: Training Word2Vec model: model=Word2Vec(sentences,vector_size=100,window=3,min_count=1,sg=1); View word vector: print("'·' word vector:",model.wv["·"]); S23: After the training is completed, the words in the sentence are given instructions to obtain the representation of the entire sentence, and the orange or verbal is converted into expression; S24: The word vectors obtained through Word2Vec model training are used to perform word-to-word similarity calculation, word meaning analysis, clustering and subject analysis.

3. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 2, it is characterized by: In step S4, images include people, scenery and animals; behaviors include independent behaviors and dependent behaviors; and states include adjectives and adverbs.

4. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 3, it is characterized by: The specific application in step S5 includes using the trained NER model to identify images in poetry, identify the behaviors and states described by the poet, count the images and emotional words that appear in ancient poetry, generate word cloud charts, analyze the poet's common images and emotional themes, analyze the identified images, behaviors and states, and discover common themes in poetry.

5. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 4, it is characterized by: In step S8, creating the atmosphere of the ancient poetry scene specifically includes text prompt word optimization, reference image collection and single ancient poetry scene image generation; text prompt word optimization includes early emotional preparation, picture logic arrangement and precise and restrained expression; The early emotional preparation includes adding the background story description of the image to the prompt words, so that the AI ​​can understand the emotional tone behind the picture, describe the emotional state and facial expressions of the characters, and add multiple sensory elements to the prompt words, including visual elements, auditory elements, and tactile elements; In visual elements, light and shade are used to describe the scene, in auditory elements, sound elements are added; in tactile elements, words are used to express the temperature and texture of the scene; The logical arrangement of the picture includes the order of perception, juxtaposition of perspectives, guiding the line of sight and focusing on details; In the order of perception, guide the sight by describing the layout of the picture; clarify the position and relationship of different elements in the picture in the parallel perspective; add elements to guide the sight in the guiding sight; set static details in the key position of the focusing details; Precise and restrained expression includes turning the virtual into the real, turning stillness into movement and depicting details; turning the virtual into the real makes abstract emotions concrete; turning stillness into movement gives dynamics to static objects; and adding detailed descriptions to depicting details.

6. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 5, it is characterized by: In the reference image collection in step S8, a text-image retrieval model is used to automatically retrieve reference images that meet the description from the image database according to the prompt words; specifically, the following steps are included: S81: Prepare a large image database, such as the public datasets LAION, Unsplash or Getty Images; S82: Use CLIP, a multimodal model with image-text matching capabilities, to encode the prompt words and all the images in the database at the same time, so that the computer can understand the semantic relationship between text and images; S83: Calculate the similarity between the prompt word and each image, and select the images with the highest similarity as reference images.

7. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 5, it is characterized by: Generating a single scene image using Stable Diffusion includes the following steps: Step 1: Generate an edge map or depth map based on the reference image, load the ControlNet plug-in in Stable Diffusion, input the edge map or depth map of the reference image, and set the control strength of ControlNet to 0.5-0.8; Step 2: Generate an image with the prompt word. ControlNet controls the overall structure of the generated image based on the edge or depth map. Step 3: ControlNet maintains a specific image structure during the generation process to ensure that the image composition meets expectations; If the generated image has a specific character pose or scene layout, use ControlNet's pose detection OpenPose or edge detection Canny; If the atmosphere of ancient poetry has a sense of layering or depth, the depth map generated by the depth estimation model MiDaS is used to control the spatial sense of the image.

8. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 5, it is characterized by: In step S9, the image guidance ancient poetry scene is constructed including a direct scene and an indirect scene, and the direct scene includes a texture map and an environment map.

9. According to the teaching method of the artistic conception of ancient poetry based on the open source model architecture as described in claim 5, it is characterized by: In step S9, the player's interaction with the ancient poetry scene includes teaching system construction, character transmission, click interaction, grab interaction, ray interaction, visual interaction and emotional interaction; The process of building a teaching system includes initial contact and perception, emotional experience and resonance, rational thinking and understanding, knowledge consolidation and application, emotional internalization and expression, rational integration and innovation, and the integration of knowledge and emotion.

10. A teaching method for the artistic conception of ancient poetry based on an open source model architecture according to claim 9, characterized in that: During the initial contact and perception process, it includes virtual enrollment, poetry overview and basic recitation; The process of emotional experience and resonance includes emotional immersion, role-playing and interactive learning; The process of rational thinking and understanding includes clue exploration, logical reasoning and creative practice; The process of knowledge consolidation and application includes scenario review and knowledge application; The process of emotional internalization and expression includes emotional resonance and emotional expression; The rational integration and innovation process includes cue integration and innovative thinking; The process of integrating knowledge and emotion includes reviewing, testing, and sharing emotional experiences.

Images