Mind mapping online generation system based on AI assistance

Abstract

The invention relates to the technical field of AI education assistance, in particular to an AI assistance-based mind map online generation system, which comprises a data acquisition and preprocessing unit, an intelligent image association fusion unit, a mind map construction optimization unit and a user interaction feedback adjustment unit, the data acquisition and preprocessing unit acquires poem data and extracts key images, the intelligent image association fusion unit matches and fuses images for poem images based on a deep learning recognition model, and the mind map construction optimization unit constructs a mind map framework by applying a dynamic programming algorithm. According to the method, multiple types of information are integrated, layout is optimized, knowledge graph links are established, a user interaction feedback adjustment unit adjusts mind maps by analyzing user behavior data, the problems that in traditional poem learning, a logic structure is not visual, and cultural background knowledge is difficult to integrate are solved, and poem knowledge is visually and systematically presented with the help of the AI technology; and the understanding and memorizing efficiency of the learner on the poems is improved.

Description

Technical Field

[0001] The present invention relates to the field of AI educational assistance technology, and in particular to an AI-assisted mind map online generation system. Background Art

[0002] AI educational assistance is an important technology. In the field of poetry learning, poetry carries rich cultural connotations with its refined language and profound artistic conception, but traditional learning methods rely more on recitation and simple text interpretation. Learners often find it difficult to quickly grasp the semantic connection between words and sentences in poetry, the ups and downs of emotions, and the deep meaning behind the images, making the learning process relatively boring and inefficient.

[0003] With the development of information technology, although some electronic learning resources have emerged, they are still insufficient in helping learners build a systematic knowledge system. Ordinary text materials often present the content of poetry in a linear way, and are only arranged in the order of the verses. There is a lack of visual sorting of the internal logical relationship of the poetry, which makes it impossible to intuitively display the logical structure of the poetry. It is difficult for learners to clearly see the connection between different verses and images. At the same time, for the cultural background knowledge contained in the poetry, such as the author's life experience, creative background, and related historical events, learners need to spend a lot of time collecting and sorting from different channels, and it is difficult to organically combine this knowledge with the poetry itself. In order to solve this technical problem, we provide an AI-assisted mind map online generation system. Summary of the invention

[0004] The purpose of the present invention is to provide an AI-assisted online mind map generation system to solve the problems raised in the above background technology.

[0005] To achieve the above purpose, an AI-assisted mind map online generation system is provided, which includes a data acquisition and preprocessing unit, an intelligent image association fusion unit, a mind map construction optimization unit, and a user interaction feedback adjustment unit;

[0006] The data acquisition and preprocessing unit acquires poetry data through multiple source channels, preprocesses the poetry data, and then uses natural language processing technology to analyze the preprocessed poetry data to extract key image information;

[0007] The intelligent image association fusion unit extracts and analyzes the visual features and semantic information of the key images in the poems based on the deep learning image recognition model, and processes the selected images using the image semantic understanding algorithm and multimodal fusion technology to generate image fusion data;

[0008] The mind map construction optimization unit constructs a mind map framework according to the semantic structure, emotional context and image association of the poem. In the process of node construction, the poem text, key semantic analysis, emotional label and image fusion data are integrated, and the layout structure and node size of the mind map are adjusted by using intelligent layout algorithm and reinforcement learning mechanism, and then the knowledge graph technology is used to establish dynamic links;

[0009] The user interaction feedback adjustment unit builds an interactive interface between the user and the mind mapping system, collects the user's behavior data during the mind mapping operation in real time through the user behavior monitoring and analysis system, uses a machine learning algorithm to analyze and evaluate the behavior data, and adjusts the mind mapping according to the evaluation results.

[0010] As a further improvement of the technical solution, the intelligent image association fusion unit includes an intelligent image module, and the intelligent image module adopts a residual network structure pre-trained by self-supervised contrastive learning based on a deep learning image recognition model. The specific pre-training method is as follows:

[0011] First, self-supervised learning is performed on an unlabeled image dataset. For image enhancement, a combination of random cropping, horizontal flipping, and color jittering is used to generate different enhanced views of the same image.

[0012] Define the contrast loss function, and learn the universal feature representation of the image by minimizing the contrast loss function through back propagation, and then transfer the pre-trained model weights to the model for image recognition of key images in poetry.

[0013] As a further improvement of the technical solution, the intelligent image association fusion unit includes an association fusion module. When the association fusion module performs image screening in the image resource library for key images in poems, a fast retrieval method based on semantic hashing is adopted, as follows:

[0014] First, a semantic hash function is constructed to map the semantic information of the image and the text features of the key image into a fixed-length binary hash code. The semantic information is the description information extracted by the pre-trained image semantic understanding model, and the text features are obtained by word vector conversion.

[0015] By calculating the Hamming distance between hash codes, images with the same semantics as the key image are screened out and included in the preliminary candidate set, and then visual feature matching is performed on the candidate set.

[0016] As a further improvement of the technical solution, in the association fusion module, when processing the screened images using the image semantic understanding algorithm, a semantic enhancement method based on a graph convolutional network is adopted, as follows:

[0017] Construct an image semantic graph, take the objects and scene elements in the image as nodes, and the semantic relationships between the elements as edges, construct a connection matrix and a node feature matrix, where the node features are extracted through the pre-trained object recognition model and scene understanding model, and then propagate through the graph convolutional network to update the node features.

[0018] As a further improvement of the technical solution, the mind map construction optimization unit includes a mind map module. When constructing a mind map framework, the mind map module adopts a semantic structure mining algorithm based on dynamic programming, as follows:

[0019] Segment the poem text into sentences and construct a semantic dependency graph between the sentences, wherein the semantic dependency graph is used to represent a graphical structure of semantic dependency between texts;

[0020] And define the state transfer equation, which is a mathematical expression used to describe the law of system transition between different states. Then, the optimal semantic path is obtained by solving it through the dynamic programming algorithm, and the framework structure of the mind map is constructed based on this.

[0021] As a further improvement of the technical solution, the mind map module adopts a hybrid method based on the sentiment dictionary and the deep learning model to generate sentiment labels during the node construction process, as follows:

[0022] First, we use the sentiment dictionary to annotate the sentiment polarity of the words in the poems. For the words not included in the dictionary, we map them to a low-dimensional vector space through a pre-trained word vector model, and then input them into a sentiment classifier based on a long short-term memory network to judge the sentiment polarity. The output of the classifier is the sentiment label of the word.

[0023] When constructing nodes, the sentiment tendency of each node is determined based on the sentiment tag statistics of the vocabulary and the semantic analysis of the sentences, and the sentiment tag is integrated as one of the attributes of the node.

[0024] As a further improvement of the technical solution, the mind map construction optimization unit includes a construction optimization module. When the construction optimization module uses the intelligent layout algorithm to adjust the layout structure of the mind map, the optimization algorithm based on simulated annealing is used, as follows:

[0025] Define the objective function of the layout, which is the number of edge intersections between nodes, and initialize the layout parameters, which are the coordinates of the nodes. In each iteration, obtain new layout parameters by randomly perturbing the current layout parameters, and calculate the difference of the objective function accordingly, and adjust the mind map layout according to the difference of the objective function.

[0026] As a further improvement of the technical solution, the construction optimization module adopts a knowledge graph compression method based on knowledge distillation when establishing dynamic links using knowledge graph technology, as follows:

[0027] First, a knowledge graph is constructed, which contains entities and relationships of people, places, events, and images related to poetry, and a graph neural network is used to encode the knowledge graph to obtain vector representations of entities and relationships.

[0028] Then, this knowledge graph is used as the teacher model to construct a student knowledge graph. Through the knowledge distillation technology, the difference between the teacher model and the student model in entity prediction and relationship prediction tasks is minimized, and relative entropy is used as the loss function;

[0029] After distillation training, dynamic links are established in the mind map based on the students’ knowledge graph.

[0030] As a further improvement of the technical solution, in the user interaction feedback adjustment unit, when analyzing and evaluating the user behavior data using a machine learning algorithm, a hybrid model based on time series decomposition and convolutional neural network is adopted, as follows:

[0031] First, the user behavior data is decomposed into a time series, which is divided into a trend term, a seasonal term, and a residual term. These three components are extracted by moving average and seasonal decomposition methods.

[0032] The decomposed time series data is reassembled into a two-dimensional image form to construct a multi-channel image data, and then the image data is input into a convolutional neural network for feature extraction and classification. The convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer, wherein the convolutional layer is used to extract local features of the image, the pooling layer is used for downsampling, and the fully connected layer is used for the final classification task;

[0033] And adjust the content display, layout structure and prompt information of the mind map according to the classification results.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] In an AI-assisted online mind map generation system, the mind map construction optimization unit adopts a semantic structure mining algorithm based on dynamic programming when constructing the mind map framework. It can segment the poetry text and construct a semantic dependency graph. The optimal semantic path is obtained by solving it, and the framework is constructed based on this. The logical structure of the poetry is clearly presented, and the connection between different verses and images is clear at a glance. The poetry text, semantic analysis, emotional label and image fusion data are integrated in the node construction, and dynamic links are established using knowledge graph technology to closely connect poetry with cultural background knowledge, so that learners can acquire rich knowledge without collecting and organizing it by themselves. The user interaction feedback adjustment unit monitors and analyzes user behavior data, uses a hybrid model to evaluate and adjust the mind map, so that it continuously meets user needs, further optimizes the learning experience, and effectively assists learners in building a systematic poetry knowledge system and improving learning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is an overall block diagram of the present invention.

[0037] The meaning of each number in the figure is:

[0038] 1. Data acquisition and preprocessing unit; 2. Intelligent image association and fusion unit; 21. Intelligent image module; 22. Association and fusion module; 3. Mind map construction and optimization unit; 31. Mind map module; 32. Construction optimization module; 4. User interaction feedback adjustment unit. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The present invention provides an AI-assisted mind map online generation system, please refer to Figure 1 As shown, it includes a data acquisition and preprocessing unit 1, an intelligent image association fusion unit 2, a mind map construction optimization unit 3 and a user interaction feedback adjustment unit 4.

[0041] The data collection and preprocessing unit 1 obtains poetry data through multiple source channels, preprocesses it, and then uses natural language processing technology to analyze the preprocessed poetry data to extract key image information.

[0042] The intelligent image association fusion unit 2 is based on a deep learning image recognition model. It extracts and performs association analysis on the visual features and semantic information of the key images in the poems in the image resource library, and uses the image semantic understanding algorithm and multimodal fusion technology to process the screened images and generate image fusion data.

[0043] The intelligent image association fusion unit 2 includes an intelligent image module 21. The intelligent image module 21 adopts a residual network structure pre-trained by self-supervised contrastive learning based on a deep learning image recognition model. The specific pre-training method is as follows:

[0044] Collect a large-scale unlabeled image dataset that covers various scenes, objects, and visual elements to ensure that rich image features can be learned. Perform multiple enhancement operations on each image in the dataset to generate different enhanced views. Through the combination of the above enhancement operations, two different enhanced views are generated for each original image. These two enhanced views have certain visual differences, but still retain the key information and semantic content of the original image, providing rich data pairs for subsequent self-supervised contrastive learning.

[0045] Build a deep learning model architecture based on residual network so that the network can effectively learn deep features. The input layer of the model receives the enhanced image, passes through a series of convolutional layers, pooling layers and residual blocks, and finally outputs a fixed-length feature vector to represent the characteristics of the image.

[0046] The contrast loss function is used for self-supervised learning, so that the model can learn the similarities and differences between different enhanced views, and thus learn the general feature representation of the image. The contrast loss function is defined as:

[0047]

[0048] Among them, z ij is the image x ij The feature vector after residual network encoding, represents cosine similarity, which is used to measure the similarity between two feature vectors. τ is the temperature parameter and N is the batch size. During the training process, N pairs of enhanced images are randomly selected from the data set for training each time. In this way, more sample information can be used to update the model parameters in each iteration, thereby improving the training efficiency and stability of the model.

[0049] During the training process, the generated image enhancement pairs (x i1 , x i2 ) is input into the residual network model, and the corresponding feature vector z is obtained through forward propagation. i1 and z i2, the loss value is calculated according to the defined contrast loss function, and then the gradient of the loss function to the model parameters is calculated through the back propagation algorithm, so that the model gradually reduces the contrast loss value during the training process, thereby learning the general feature representation of the image.

[0050] The training process involves multiple training cycles. In each cycle, the entire data set is iterated multiple times. In each iteration, a batch of image enhancement pairs are randomly selected for forward propagation, loss calculation, and back propagation to update parameters. As the training progresses, the model gradually learns various features and patterns of the image, and can effectively extract and represent features of different images, providing a powerful basic model for subsequent key image recognition tasks in poetry.

[0051] After completing the pre-training of self-supervised contrastive learning, the weights of the pre-trained residual network model are transferred to the model for image recognition of key images in poetry.

[0052] The intelligent image association fusion unit 2 includes an association fusion module 22. When the association fusion module 22 performs image screening in the image resource library for key images in poems, it adopts a fast retrieval method based on semantic hashing, as follows:

[0053] Use the pre-trained image semantic understanding model to process each image in the image resource library, extract the semantic description information of the image, and then input these features into the recurrent neural network. By identifying and understanding elements such as objects, scenes, colors, and actions in the image, a text information describing the semantics of the image is generated, such as "a natural landscape image depicting green mountains and clear waters, a small bridge across the river, and several willow trees swaying in the wind on the shore."

[0054] For the key images in the poems, their texts are converted into word vectors of fixed dimensions through the word vector conversion model, and a semantic hash function is designed, whose purpose is to map the semantic description information of the image and the word vector of the key image into a binary hash code of fixed length. In this way, both the image and the key image are converted into binary hash codes of the same length for subsequent fast similarity comparison.

[0055] For each key image in the poem, the Hamming distance between its hash code and the hash codes of all images in the image resource library is calculated. The Hamming distance refers to the number of different characters in corresponding positions of two equal-length strings. For binary hash codes, it is the number of different corresponding bits. Then a Hamming distance threshold is set. When the Hamming distance between the image and the hash code of the key image is less than the threshold, the image is included in the preliminary candidate set. This can quickly eliminate most of the images that have a large semantic difference with the key image, greatly reducing the amount of calculation for subsequent precise matching and improving the efficiency of image screening.

[0056] For each image in the preliminary candidate set, its visual features are extracted using an image feature extraction model based on a convolutional neural network. These models can effectively extract visual features such as color, texture, and shape of the image by learning from a large number of images, and represent them as a feature vector of fixed length.

[0057] The similarity between the visual feature vector of the candidate image and the standard visual feature vector of the key image is calculated. The standard visual feature vector of the key image can be obtained by averaging the features of a large number of high-quality images related to the image. According to the calculation result of the visual similarity, a visual similarity threshold is set.

[0058] When the visual similarity between the candidate image and the key image is greater than the threshold, the image is determined as the final image that matches the key image and is included in the final image set for subsequent operations such as image fusion and mind map construction. In this way, on the basis of rapid screening based on semantic hashing, visual feature matching is further used to ensure that the selected image is not only semantically related to the key image, but also has a high degree of visual fit, which can better provide high-quality, artistic conception-fitting image resources for the construction of poetry mind maps, enhance the visualization and expressiveness of mind maps, and help users understand the connotation and artistic conception of poetry more intuitively and deeply.

[0059] In the association fusion module 22, when processing the screened images using the image semantic understanding algorithm, a semantic enhancement method based on a graph convolutional network is adopted, as follows:

[0060] For each screened image, the pre-trained object recognition model and scene understanding model are used to identify and annotate the objects and scene elements in the image, and these elements are used as nodes of the image semantic graph.

[0061] The connectivity and weight of the edges are determined based on the semantic relationships between objects and scene elements. Semantic relationships can include spatial position relationships, semantic similarity relationships, and functional relationships. Spatial adjacent elements are given higher edge weights, semantically related elements are given medium edge weights, and the edge weights of other relationships can be set to lower values ​​based on the specific situation. The adjacency matrix is ​​constructed in this way.

[0062] The pre-trained object recognition model and scene understanding model are used to extract the feature vector of each node. These feature vectors can include the category information of the object, the shape, color, texture visual features of the object, and the semantic features of the scene. The feature vectors of all nodes are combined into a node feature matrix, where the number of rows of the matrix is ​​equal to the number of nodes, and the number of columns is equal to the dimension of the feature vector.

[0063] A multi-layer graph convolutional network is used to propagate and update node features. The propagation formula of one layer of the graph convolutional network is:

[0064]

[0065] I is the identity matrix, yes , which ensures that each node has self-connection. W is a learnable weight matrix whose dimension is the product of the input feature dimension and the output feature dimension. σ is the activation function.

[0066] A multi-layer graph convolutional network is used to gradually enhance the semantic expression of node features. Each layer propagates and updates features through the above-mentioned graph convolution formula, and activation functions are used between each layer to enable the model to learn the multi-level semantic relationship between objects and scene elements in the image, thereby enhancing the semantic information of the entire image. Through multi-layer feature propagation and updating, the final node feature matrix provides richer and more accurate image semantic information for the subsequent integration with key images of poetry and the construction of mind maps, so that the generated mind maps can better show the artistic conception and connotation of poetry.

[0067] Mind map construction optimization unit 3 constructs a mind map framework based on the semantic structure, emotional context and image association of poetry. In the process of node construction, the poem text, key semantic analysis, emotional labels and image fusion data are integrated, and the layout structure and node size of the mind map are adjusted using intelligent layout algorithms and reinforcement learning mechanisms, and then knowledge graph technology is used to establish dynamic links.

[0068] The mind map construction optimization unit 3 includes a mind map module 31. When constructing a mind map framework, the mind map module 31 adopts a semantic structure mining algorithm based on dynamic programming, as follows:

[0069] First, obtain the poem text for which a mind map needs to be constructed, and clean it to remove punctuation marks, special characters, and extra spaces in the text, and convert the text into a pure text sequence.

[0070] The sentence segmentation technology in natural language processing is used to divide the poetry text into semantically complete sentences. For each sentence, a pre-trained word vector model is used to convert each word in the sentence into a word vector representation, thereby obtaining the word vector sequence of the sentence. At the same time, the pre-trained syntactic analysis model is used to perform syntactic analysis on the sentence to extract the syntactic relationship and part-of-speech information between the words in the sentence.

[0071] Each sentence is regarded as a node in the semantic dependency graph. For edge construction, if there is a semantic association between two sentences, an edge is added between the corresponding nodes. The judgment of semantic association can be based on vocabulary overlap, that is, if the same key words exist in two sentences, they are considered to have a certain semantic association.

[0072] For each edge in the semantic dependency graph, calculate its weight, which represents the strength of the semantic association between the two sentences. The edge weight can be calculated based on the vocabulary overlap ratio, that is, the ratio of the number of overlapping words in the two sentences to the total number of words is calculated as part of the edge weight.

[0073] The definition state represents the optimal semantic path score ending with a node, that is, the maximum value of the sum of all edge weights in the semantic path from the starting node to the node. The starting node can be determined according to the theme of the poem or a common semantic starting point.

[0074] The state transfer equation is where ρ(i) is the set of predecessor nodes of node i, i.e., the nodes that are connected to node i by edges and are semantically located before node i, and ω ji is the edge weight from node j to node i. By continuously applying this state transfer equation, starting from the starting node, the optimal semantic path score of each node is gradually calculated until all nodes have been calculated.

[0075] After calculating the optimal semantic path scores of all nodes, the optimal semantic path is determined by backtracking. Starting from the end node, select the predecessor node with the maximum score, and then backtrack forward in sequence until it backtracks to the starting node. The path obtained in this way is the optimal semantic path from the starting node to the end node. The node sequence and connection relationship on this path can reflect the main semantic structure and logical context of the poem. On this basis, the framework structure of the mind map is constructed. The nodes on the path are used as the main branch nodes of the mind map, and they are expanded in sequence, thereby constructing a mind map framework that can accurately reflect the semantic hierarchy and logical relationship of the poem, providing a reasonable structural basis for subsequent node content filling and layout optimization, helping users better understand the connotation and structure of the poem, and improving the readability and practicality of the mind map.

[0076] In the process of node construction, the mind map module 31 uses a hybrid method based on the sentiment dictionary and the deep learning model to generate sentiment labels, as follows:

[0077] First, select a HowNet sentiment dictionary. HowNet sentiment dictionary usually divides words into different sentiment categories, such as positive, negative, and neutral, and assigns corresponding sentiment polarity values. Perform word segmentation on the poem text and split the poem into independent words. Then, traverse these words and search for matching words in the selected sentiment dictionary. For words that can be found in the dictionary, directly assign corresponding sentiment labels according to the sentiment polarity annotations given in the dictionary.

[0078] For words that are not included in the sentiment dictionary, a pre-trained word vector model is used to map them to a low-dimensional vector space. This vector can reflect the semantics and context-related information of the word to a certain extent, and collect a large-scale annotated sentiment text dataset. The dataset can contain various text forms such as poetry, prose, and modern texts, and each word has a clear sentiment polarity annotation.

[0079] Extract words and their corresponding word vectors from these texts as input features, use sentiment polarity annotation as output labels, build a training sample set, and build a long short-term memory network architecture consisting of input layer, hidden layer, and output layer. The input layer receives word vectors as input, and its number of neurons is the same as the dimension of word vectors. The hidden layer can be set to multiple layers. The long short-term memory network can effectively process long-term dependencies in sequence data and capture the impact of semantic associations of words in text on sentiment. The output layer uses a fully connected layer, and outputs the probability distribution of words belonging to different sentiment polarity categories through the softmax function.

[0080] The network is trained using the training sample set, and the cross entropy loss function is defined to minimize the loss function. Through multiple iterative training, the model can learn the mapping relationship between the word vector features of the vocabulary and the sentiment polarity, so that it has the ability to judge the sentiment polarity of unincluded vocabulary.

[0081] The vector representation of the unincluded vocabulary after mapping by the word vector model is input into the trained long short-term memory network sentiment classifier. After forward propagation calculation, the probability distribution of the vocabulary belonging to different sentiment polarity categories is obtained, thereby completing the sentiment polarity judgment of all unincluded vocabulary.

[0082] For each node in the mind map construction, the sentiment labels of all the words it contains are counted. In addition to the simple sentiment label statistics of the words, it is also necessary to analyze them in combination with the overall semantics of the sentence. Because sometimes the sentiment label combination of words may be more complex, it is necessary to judge its sentiment tendency from the overall context of the sentence, and add the determined sentiment tendency as an attribute to the corresponding mind map node. In the subsequent visualization of the mind map or user interaction process, this sentiment label attribute can be presented in different colors, icons, etc., to help users better understand the emotional tone of the poem and improve user experience and learning effects.

[0083] The mind map construction optimization unit 3 includes a construction optimization module 32. When the construction optimization module 32 uses the intelligent layout algorithm to adjust the layout structure of the mind map, it uses an optimization algorithm based on simulated annealing, which is as follows:

[0084] For the layout of the mind map, the goal is to reduce the number of edge crossings between nodes, because too many edge crossings will make the mind map look messy and affect readability. To determine whether they cross, we can determine whether the straight line equations where the two line segments are located have an intersection and whether the intersection is within the range of the two line segments. After testing all edge pairs, the total number of edge crossings is obtained, which is used as the main part of the objective function.

[0085] First, you need to determine the initial coordinates for each node in the mind map. Take the center of the mind map as the origin, place the root node near the origin according to the hierarchy and semantic relationship of the nodes, and then assign coordinates to the child nodes in sequence according to the order of breadth-first search or depth-first search. These initial coordinates constitute the initial layout parameters.

[0086] By setting the initial temperature and cooling rate, the temperature will gradually decrease during the iteration process, controlling the algorithm's acceptance of new solutions. The higher the temperature during simulated annealing, the easier it is to accept poorer solutions, thereby avoiding falling into local optimal solutions; the lower the temperature, the more the algorithm tends to accept better solutions, gradually converging to the global optimal solution or a region close to the global optimal solution.

[0087] In each iteration t, the current layout parameters Perform random perturbations to obtain new layout parameters X t+1 The perturbation method can be to add a random small offset to the coordinates of each node, and add a similar random number to the y coordinate to obtain the new node coordinates where Δx i and Δy i is a randomly generated offset, which generates a new layout solution.

[0088] Use the new layout parameter X t+1Calculate the objective function value F t+1 , while using the current layout parameters X t Calculate the objective function value F t , calculate the difference of the objective function ΔF = F t+1 -F t According to the simulated annealing criterion, if ΔF < 0, that is, the new layout scheme reduces the objective function value, then the new layout parameter X is accepted. t+1 As the basis for the next iteration, if ΔF ≥ 0, then with a certain probability p = exp(-ΔF / T t ) accepts new layout parameters, where T t is the current temperature. The calculation of this probability is the core of the simulated annealing algorithm. It enables the algorithm to accept a worse solution with a higher probability at high temperatures, thereby jumping out of the trap of the local optimal solution. At low temperatures, it is more inclined to accept a better solution and gradually converge to a better layout solution.

[0089] A maximum number of iterations is set. When the number of iterations reaches the maximum number of iterations, the algorithm terminates. The layout parameters at this time are the optimized mind map layout parameters. The final coordinates of each node are determined according to these parameters to complete the layout optimization of the mind map, and a mind map with fewer edge crossings and a more reasonable and beautiful layout is obtained. The visualization effect of the mind map and the clarity of information display are improved, which makes it easier for users to intuitively understand the structure and content of poetry and enhances the user experience.

[0090] When constructing the optimization module 32 to establish dynamic links using the knowledge graph technology, a knowledge graph compression method based on knowledge distillation is adopted, as follows:

[0091] Collect people, places, events, and images related to poetry from multi-source data, label and classify these entities, determine their types and attributes, analyze the semantic relationships between entities, extract these relationships through a combination of natural language processing technology and manual annotation, define the type and direction of the relationships, build a network structure of entities and relationships, and form a complete knowledge graph.

[0092] Select a graph neural network architecture to encode the knowledge graph and build a graph neural network model consisting of an input layer, multiple hidden layers, and an output layer. The input layer receives the node feature matrix X and adjacency matrix A of the knowledge graph.

[0093] By performing multiple rounds of message passing and feature updates on the knowledge graph, the model learns the semantic representation of entities and relationships. After multiple layers of feature updates, a vector representation of each entity and relationship is obtained. This vector can represent the semantic features of the Li Bai entity in the entire knowledge graph and the association information with other entities in a low-dimensional space, thereby completing the encoding process of the large knowledge graph and obtaining rich entity and relationship vector representations, providing a basis for subsequent knowledge distillation.

[0094] According to actual application requirements and resource constraints, a student knowledge graph with a more compact structure and fewer entities and relationships is constructed. A graph neural network architecture similar to the teacher model is used, but the number of layers and nodes can be appropriately reduced to adapt to the simplified student knowledge graph structure. The parameters of the student model are randomly initialized to prepare for subsequent distillation training.

[0095] Relative entropy is used as the loss function of knowledge distillation to measure the difference between the teacher model and the student model in entity prediction and relationship prediction tasks. For the entity prediction task, it is assumed that the predicted probability distribution of the teacher model for the entity is p te , the predicted probability distribution of the student model for the same entity is p se , then the distillation loss formula for entity prediction is Among them, E s is the entity set in the student knowledge graph, It is the relative entropy formula, which is used to calculate the degree of difference between two probability distributions. The purpose of this loss function is to make the predicted probability distribution of the student model as close as possible to the predicted probability distribution of the teacher model, so that the student model can learn the knowledge and semantic representation ability mastered by the teacher model, and can accurately predict and classify entities under limited resources, thereby improving the application efficiency and accuracy of the knowledge graph.

[0096] Similarly, for the relationship prediction task, the relative entropy loss L between the teacher model and the student model on relationship prediction is calculated r , by minimizing the distillation loss of relationship prediction, the student model can learn the teacher model's accurate judgment and representation ability of the relationship between entities, so as to better construct the semantic associations in the knowledge graph and provide reliable relationship information for the dynamic links in the mind map.

[0097] Use the same training data as the large knowledge graph, or extract a portion of representative data from it as the dataset for distillation training, including instances of entities and relationships and corresponding label information, to supervise the learning of the student model during the training process so that it can correctly predict the relationship type between entities and the category of the entity.

[0098] During the training process, the training data is input into the teacher model and the student model at the same time. The teacher model outputs its prediction results for entities and relationships, and the student model also outputs the corresponding prediction results. According to the entity prediction loss and relationship prediction loss defined above, the total distillation loss L is calculated. KD , use the optimization algorithm to minimize the distillation loss function, and through multiple iterative training, the student model gradually learns the knowledge and capabilities of the teacher model, and its performance in entity prediction and relationship prediction tasks continues to approach the teacher model, thereby completing the knowledge distillation process and obtaining a compressed and optimized student knowledge graph model, which can reduce computing resource requirements and model complexity while maintaining key knowledge, providing strong support for quickly and efficiently establishing dynamic links in mind maps.

[0099] In the process of constructing the mind map, after a node is determined, the student knowledge graph model is used to find other entities and relationships related to the node. By searching for other nodes that have a direct or indirect relationship with the image node in the student knowledge graph, a dynamic link from the current node to the related nodes is created in the mind map based on the found entity and relationship information.

[0100] When the user clicks on the link, the relevant information will be displayed, providing the user with rich knowledge expansion and related information, enhancing the knowledge content and interactivity of the mind map, helping the user to understand the cultural background and artistic value of poetry more deeply, and improving the user's exploration and learning efficiency of poetry knowledge.

[0101] The user interaction feedback adjustment unit 4 builds an interactive interface between the user and the mind mapping system, collects the user's behavior data during the mind mapping operation in real time through the user behavior monitoring and analysis system, uses a machine learning algorithm to analyze and evaluate the behavior data, and adjusts the mind mapping according to the evaluation results.

[0102] User behavior data often contains a variety of different patterns over time, such as long-term growth or downward trends, and periodic repetitive behaviors. Through time series decomposition, these components of different natures can be separated, providing a clearer insight into the essential laws of user behavior, which will help to accurately analyze the characteristics of user interaction with mind maps in the future.

[0103] The decomposed time series data is reassembled into a two-dimensional image form to construct a multi-channel image data. Then, this image data is input into a convolutional neural network for feature extraction and classification. The convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer, wherein the convolutional layer is used to extract local features of the image, the pooling layer is used for downsampling, and the fully connected layer is used for the final classification task. The convolutional neural network has a strong feature extraction capability in processing image data, and can automatically learn the complex local and global feature patterns in the image. The time series data is reconstructed into an image form. With the advantage of the convolutional neural network, the hidden associations and features between different components of user behavior data can be mined, and these features may be difficult to effectively obtain through traditional data analysis methods.

[0104] The content display, layout structure and prompt information of the mind map are adjusted according to the classification results. Different users have different experiences and needs for using mind maps. Through targeted adjustments based on classification results, the presentation of mind maps can be optimized according to actual user feedback, such as satisfaction level, to better suit each user's usage preferences and enhance the interaction between users and the system.

[0105] In the present invention, the poetry data is acquired and key images are extracted through the data collection and preprocessing unit 1, the intelligent image association fusion unit 2 matches and fuses images for poetry images based on the deep learning recognition model, the mind map construction optimization unit 3 uses the dynamic programming algorithm to construct the mind map framework, integrates multiple types of information and optimizes the layout and establishes knowledge graph links, and the user interaction feedback adjustment unit 4 adjusts the mind map by analyzing the user behavior data, which solves the problems of non-intuitive logical structure and difficult integration of cultural background knowledge in traditional poetry learning, and uses AI technology to realize the visualization and systematic presentation of poetry knowledge, thereby improving learners' understanding and memory efficiency of poetry.

[0106] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An AI-assisted mind map online generation system, characterized in that: It includes a data acquisition and preprocessing unit (1), an intelligent image association fusion unit (2), a mind map construction optimization unit (3) and a user interaction feedback adjustment unit (4); The data acquisition and preprocessing unit (1) acquires poetry data through multiple source channels, preprocesses the poetry data, and then uses natural language processing technology to analyze the preprocessed poetry data to extract key image information; The intelligent image association fusion unit (2) extracts and analyzes the visual features and semantic information of the key images in the poem based on the deep learning image recognition model, and processes the selected images using the image semantic understanding algorithm and multimodal fusion technology to generate image fusion data; The mind map construction optimization unit (3) constructs a mind map framework according to the semantic structure, emotional context and image association of the poem. In the process of node construction, the poem text, key semantic analysis, emotional label and image fusion data are integrated, and the layout structure and node size of the mind map are adjusted by using an intelligent layout algorithm and a reinforcement learning mechanism, and then a dynamic link is established by using knowledge graph technology; The user interaction feedback adjustment unit (4) builds an interactive interface between the user and the mind mapping system, collects the user's behavior data during the mind mapping operation in real time through the user behavior monitoring and analysis system, analyzes and evaluates the behavior data using a machine learning algorithm, and adjusts the mind mapping according to the evaluation results.

2. The AI-assisted mind map online generation system according to claim 1, characterized in that: The intelligent image association fusion unit (2) comprises an intelligent image module (21), wherein the intelligent image module (21) adopts a residual network structure pre-trained by self-supervised contrastive learning based on a deep learning image recognition model, and the specific pre-training method is as follows: First, self-supervised learning is performed on an unlabeled image dataset. For image enhancement, a combination of random cropping, horizontal flipping, and color jittering is used to generate different enhanced views of the same image. Define the contrast loss function, and learn the universal feature representation of the image by minimizing the contrast loss function through back propagation, and then transfer the pre-trained model weights to the model for image recognition of key images in poetry.

3. The AI-assisted mind map online generation system according to claim 2, characterized in that: The intelligent image association fusion unit (2) comprises an association fusion module (22). When the association fusion module (22) performs image screening in an image resource library for key images in poems, a fast retrieval method based on semantic hashing is adopted, which is specifically as follows: First, a semantic hash function is constructed to map the semantic information of the image and the text features of the key image into a fixed-length binary hash code. The semantic information is the description information extracted by the pre-trained image semantic understanding model, and the text features are obtained by word vector conversion. By calculating the Hamming distance between hash codes, images with the same semantics as the key image are screened out and included in the preliminary candidate set, and then visual feature matching is performed on the candidate set.

4. The AI-assisted mind map online generation system according to claim 3, characterized in that: In the association fusion module (22), when processing the screened images using the image semantic understanding algorithm, a semantic enhancement method based on a graph convolutional network is adopted, which is specifically as follows: Construct an image semantic graph, take the objects and scene elements in the image as nodes, and the semantic relationships between the elements as edges, construct a connection matrix and a node feature matrix, where the node features are extracted through the pre-trained object recognition model and scene understanding model, and then propagate through the graph convolutional network to update the node features.

5. The AI-assisted mind map online generation system according to claim 4, characterized in that: The mind map construction optimization unit (3) comprises a mind map module (31). When constructing a mind map framework, the mind map module (31) adopts a semantic structure mining algorithm based on dynamic programming, which is as follows: Segment the poem text into sentences and construct a semantic dependency graph between the sentences, wherein the semantic dependency graph is used to represent a graphical structure of semantic dependency between texts; And define the state transfer equation, which is a mathematical expression used to describe the law of system transition between different states. Then, the optimal semantic path is obtained by solving it through the dynamic programming algorithm, and the framework structure of the mind map is constructed based on this.

6. The AI-assisted mind map online generation system according to claim 5, characterized in that: In the process of node construction, the mind map module (31) generates emotional labels by adopting a hybrid method based on the emotional dictionary and the deep learning model, as follows: First, we use the sentiment dictionary to annotate the sentiment polarity of the words in the poems. For the words not included in the dictionary, we map them to a low-dimensional vector space through a pre-trained word vector model, and then input them into a sentiment classifier based on a long short-term memory network to judge the sentiment polarity. The output of the classifier is the sentiment label of the word. When constructing nodes, the sentiment tendency of each node is determined based on the sentiment tag statistics of the vocabulary and the semantic analysis of the sentences, and the sentiment tag is integrated as one of the attributes of the node.

7. The AI-assisted mind map online generation system according to claim 6, characterized in that: The mind map construction optimization unit (3) comprises a construction optimization module (32). When the construction optimization module (32) uses an intelligent layout algorithm to adjust the layout structure of the mind map, it uses an optimization algorithm based on simulated annealing, which is as follows: Define the objective function of the layout, which is the number of edge intersections between nodes, and initialize the layout parameters, which are the coordinates of the nodes. In each iteration, obtain new layout parameters by randomly perturbing the current layout parameters, and calculate the difference of the objective function accordingly, and adjust the mind map layout according to the difference of the objective function.

8. The AI-assisted mind map online generation system according to claim 7, characterized in that: When the construction optimization module (32) uses the knowledge graph technology to establish dynamic links, a knowledge graph compression method based on knowledge distillation is adopted, which is as follows: First, a knowledge graph is constructed, which contains entities and relationships of people, places, events, and images related to poetry, and a graph neural network is used to encode the knowledge graph to obtain vector representations of entities and relationships. Then, this knowledge graph is used as the teacher model to construct a student knowledge graph. Through the knowledge distillation technology, the difference between the teacher model and the student model in entity prediction and relationship prediction tasks is minimized, and relative entropy is used as the loss function; After distillation training, dynamic links are established in the mind map based on the students’ knowledge graph.

9. The AI-assisted mind map online generation system according to claim 8, characterized in that: In the user interaction feedback adjustment unit (4), when analyzing and evaluating the user behavior data using a machine learning algorithm, a hybrid model based on time series decomposition and convolutional neural network is adopted, as follows: First, the user behavior data is decomposed into a time series, which is divided into a trend term, a seasonal term, and a residual term. These three components are extracted by moving average and seasonal decomposition methods. The decomposed time series data is reassembled into a two-dimensional image form to construct a multi-channel image data, and then the image data is input into a convolutional neural network for feature extraction and classification. The convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer, wherein the convolutional layer is used to extract local features of the image, the pooling layer is used for downsampling, and the fully connected layer is used for the final classification task; And adjust the content display, layout structure and prompt information of the mind map according to the classification results.

Images