Knowledge graph-based regional culture visual element dynamic reconstruction method and system

Abstract

The application discloses a regional culture visual element dynamic reconstruction method and system based on a knowledge graph, relates to the technical field of knowledge graphs, and calculates and judges whether user behavior data can accurately reflect the real semantics of culture elements, so that the algorithm does not blindly perform visual presentation of regional culture according to the user behavior data, reduces the phenomenon that part of one-sided or misleading information is regarded as an effective signal, ensures stable semantic representation of culture elements in the knowledge graph, reduces deviation accumulation, and makes the meaning of culture visual elements gradually deviate from the real historical background and cultural connotation, so that the result of the visual presentation of regional culture is consistent with the actual regional culture, and a serious culture deviation phenomenon does not occur.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph, in particular to a method and system for dynamic reconstruction of regional culture visual elements based on knowledge graph. BACKGROUND

[0002] With the continuous progress of technology, more and more unique regional culture is presented to the public in the form of vision; through knowledge graph, dynamic reconstruction of regional culture visual elements is gradually becoming an important means of protecting and inheriting these cultures. This method systematically collects and analyzes various data such as symbols, patterns, inscriptions, and folk stories in regional culture, and converts them into a knowledge graph, a structured network composed of cultural element nodes and their semantic association relationships. Knowledge graph can not only store rich cultural information, but also reveal and understand the complex relationships between cultural elements; based on this, advanced algorithms can be used to dynamically reconstruct cultural visual elements, that is, automatically adjust the combination of cultural elements according to the latest data changes and user interaction behaviors, and update their visual presentation in real time. This not only helps to protect and inherit culture, but also provides structured and semantically supported material resources for design creation; at the same time, knowledge graph can continuously update and adjust its structure and weight according to user behavior data, so that regional culture can reflect the understanding and preferences of contemporary users for regional cultural elements in time when it is presented visually, enhancing the interactivity and individualization of cultural performance. At the same time, this dynamic mechanism helps to discover potential cultural evolution trends, promote the innovation and integration of cultural elements, and promote the living inheritance and development of regional culture, avoiding the rigidity and singleness of cultural expression.

[0003] The existing dynamic reconstruction method mainly relies on user behavior feedback to optimize the structure and weight of the knowledge graph; however, the problem is that users' behavior has a strong personal subjectivity, and these data often cannot accurately and comprehensively reflect the real semantics of cultural elements, and when the algorithm adopts these feedbacks without distinction, it will gradually change the semantic representation of some cultural elements in the knowledge graph by taking some one-sided or misleading information as effective signals. With the passage of time, this unconscious deviation accumulates, causing the meaning of cultural visual elements to gradually deviate from their real historical background and cultural connotation, resulting in a serious deviation between the results of regional culture visual presentation and the actual regional culture. SUMMARY

[0004] The purpose of the present application is to solve the above-mentioned problems, and to provide a method and system for dynamic reconstruction of regional culture visual elements based on knowledge graph.

[0005] In the first aspect of the present application, a method for dynamic reconstruction of regional culture visual elements based on knowledge graph is first proposed, which comprises:

[0006] S1: Collect behavior data of a user in an interaction process of regional culture visual elements, and map the behavior data into a semantic vector based on a preset multi-modal semantic analysis model, denoted as a user semantic behavior vector;

[0007] S2: Select a corresponding core culture element and its semantic relationship from the user semantic behavior vector, construct a baseline semantic weight vector, and calculate a semantic shift gradient based on a difference between the user semantic behavior vector and the baseline semantic weight vector; and determine whether to perform semantic space reconstruction in a knowledge graph based on the semantic shift gradient;

[0008] S3: If semantic space reconstruction in the knowledge graph is needed, input the semantic shift gradient into a preset variational autoencoder model, perform semantic space reconstruction in combination with the baseline semantic weight vector, generate a user behavior semantic vector after deviation removal, and update the node weight and relationship weight value of the regional culture knowledge graph;

[0009] S4: Calculate an evaluation index of semantic space reconstruction based on the updated knowledge graph, and determine whether to perform semantic space reconstruction in the knowledge graph again based on the evaluation index; if not, use the reconstructed knowledge graph as a benchmark to support dynamic reconstruction of subsequent regional culture visual elements.

[0010] Optionally, the step of calculating a semantic shift gradient based on a difference between a user semantic behavior vector and a baseline semantic weight vector, and determining whether to perform semantic space reconstruction in a knowledge graph based on the semantic shift gradient is:

[0011] Subtract the core element weight values corresponding to the user semantic behavior vector and the baseline semantic weight vector to obtain a semantic shift gradient of each core element, forming a semantic shift gradient vector;

[0012] Calculate the norm of the semantic shift gradient vector, and compare the norm of the semantic shift gradient vector with a preset threshold; if the norm of the semantic shift gradient vector is less than the preset threshold, it indicates that the knowledge graph updated automatically according to user behavior does not need to be reconstructed in semantic space; and if the norm of the semantic shift gradient vector is not less than the preset threshold, it indicates that the knowledge graph updated automatically according to user behavior needs to be reconstructed in semantic space.

[0013] Optionally, the step of inputting the semantic shift gradient into a preset variational autoencoder model, performing semantic space reconstruction in combination with the baseline semantic weight vector, and generating a user behavior semantic vector after deviation removal is:

[0014] Normalize the semantic shift gradient vector, and concatenate the normalized semantic shift gradient vector and the baseline semantic weight vector to form a joint input vector;

[0015] Input the joint input vector into the encoder of the preset variational autoencoder model to obtain a mean vector and a variance of a latent variable, and sample the latent variable vector based on reparameterization;

[0016] Input the latent variable vector into the decoder of the variational autoencoder model to output a debiased user semantic vector with the same dimension as the user semantic behavior vector; the purpose is to make the debiased vector closer to the baseline semantic weight vector while retaining the effective information in the user behavior;

[0017] The debiased user semantic vector is updated as a new user semantic representation to the weight value and the semantic relationship weight value of the corresponding node in the regional cultural knowledge graph.

[0018] Optionally, the step of calculating the evaluation index of semantic space reconstruction is:

[0019] The evaluation index of semantic space reconstruction includes an element correction rationality index of the updated knowledge graph and a multimodal fusion consistency index of the user behavior, the element correction rationality index and the multimodal fusion consistency index are normalized to map to the interval of 0-1, and the element correction rationality index and the multimodal fusion consistency index after normalization are weighted and summed to obtain the evaluation index of semantic space reconstruction.

[0020] Optionally, the calculation step of the element correction rationality index is:

[0021] The semantic weight vector corresponding to the core element from the updated knowledge graph is extracted, denoted as the reconstructed semantic weight vector, and the baseline semantic weight vector and the reconstructed semantic weight vector are normalized to obtain the baseline probability distribution and the reconstructed probability distribution, respectively; the corresponding Shannon entropy is calculated according to the baseline probability distribution and the reconstructed probability distribution, and the absolute difference value of the two Shannon entropies is calculated, and the absolute difference value is divided by the Shannon entropy corresponding to the baseline probability distribution, and the result of subtracting the absolute difference value from the value 1 divided by the Shannon entropy corresponding to the baseline probability distribution is obtained. The structure entropy difference value is obtained.

[0022] The values corresponding to the baseline semantic weight vector are reordered in descending order to obtain the baseline semantic ordering sequence , the values corresponding to the reconstructed semantic weight vector are reordered in descending order to obtain the reconstructed semantic ordering sequence , and the Kendall rank correlation coefficient distance between the baseline semantic ordering sequence and the reconstructed ordering sequence is calculated to obtain the ordering correlation index , and the formula is: , wherein represents the number of inconsistent pairs in the ordering sequence, is the number of elements.

[0023] The structural entropy difference Multiply by the ranking correlation index The element correction rationality index is obtained.

[0024] Optionally, the calculation steps of the multimodal fusion consistency index include:

[0025] Extract semantic embedding vector sets of different modalities from the updated knowledge graph; for any two modalities A and B, denote them as a modality pair, and denote the corresponding semantic embedding vector sets as modality set A and modality set B, count the joint occurrence frequency of corresponding core elements in the two modalities, and calculate the joint probability distribution. ,in, Represents the first mode in mode set A. The element and the modality set B. The probability of all elements appearing at the same time;

[0026] Calculate the marginal probability distribution of mode set B Each edge probability for , Indicates the second mode. The probability of each element appearing;

[0027] Calculate the conditional probability distribution Among them, conditional probability for ,

[0028] In the modality set A, the th Given that the nth element appears, the modality set B contains the nth element. The probability of each element appearing ensures that for each... satisfy .

[0029] Optionally, the calculation steps of the multimodal fusion consistency index further include:

[0030] Based on conditional probability distribution and marginal probability distribution Calculate the conditional entropy of mode A and mode B. and marginal entropy The formula for calculating conditional entropy is: The formula for calculating marginal entropy is: ;

[0031] The marginal entropy of mode A and mode B Subtract conditional entropy Then divide the result of the subtraction by the marginal entropy. The fusion consistency index of modal A and modal B is obtained, the fusion consistency indexes of all modal pairs are added, and a multi-modal fusion consistency index is obtained.

[0032] Optionally, the step of judging whether the semantic space reconstruction of the knowledge graph needs to be performed again according to the evaluation index comprises:

[0033] The evaluation index is compared with the preset threshold value, if the evaluation index is not less than the preset threshold value, it indicates that the semantic space reconstruction effect of the knowledge graph in step S3 is qualified, and the semantic space reconstruction of the knowledge graph does not need to be performed again, and the reconstructed knowledge graph in S3 is directly taken as a reference to support the dynamic reconstruction of the regional culture visual elements.

[0034] Optionally, the step of judging whether the semantic space reconstruction of the knowledge graph needs to be performed again according to the evaluation index further comprises:

[0035] If the evaluation index is less than the preset threshold value, it indicates that the semantic space reconstruction effect of the knowledge graph in step S3 is unqualified, and the semantic space reconstruction of the knowledge graph in step S3 needs to be performed again until the evaluation index is not less than the preset threshold value to stop the reconstruction, and the reconstructed knowledge graph is taken as a reference to support the dynamic reconstruction of the regional culture visual elements.

[0036] In the second aspect of the embodiment of the present application, a regional culture visual element dynamic reconstruction system based on a knowledge graph is provided, and the system comprises:

[0037] A user semantic behavior module: collects behavior data of a user in a regional culture visual element interaction process, and maps the behavior data into a semantic vector based on a preset multi-modal semantic analysis model, and the semantic vector is recorded as a user semantic behavior vector;

[0038] A preliminary judgment module: selects a corresponding core culture element and a semantic relationship thereof from the user semantic behavior vector, constructs a baseline semantic weight vector, and calculates a semantic offset gradient according to a difference between the user semantic behavior vector and the baseline semantic weight vector; and judges whether the semantic space reconstruction of the knowledge graph is needed according to the semantic offset gradient;

[0039] A reconstruction module: if the semantic space reconstruction of the knowledge graph is needed, the semantic offset gradient is input into a preset variational autoencoder model, the semantic space is reconstructed in combination with the baseline semantic weight vector, a user behavior semantic vector after deviation is removed is generated, and the user behavior semantic vector after deviation is removed is updated into a node weight and a relationship weight value of the regional culture knowledge graph;

[0040] A re-determination module: based on the updated knowledge graph, an evaluation index of semantic space reconstruction is calculated, and it is determined whether the knowledge graph needs to be reconstructed for semantic space reconstruction again according to the evaluation index, if not, the reconstructed knowledge graph is taken as a benchmark to support the dynamic reconstruction of regional culture visual elements.

[0041] The beneficial effects of the present application are:

[0042] The present application provides a regional culture visual element dynamic reconstruction method and system based on a knowledge graph, which can determine whether user behavior data can accurately reflect the real semantics of cultural elements, and the algorithm does not blindly present regional culture visually according to user behavior data, reduces the phenomenon of regarding some one-sided or misleading information as effective signals, ensures stable semantic representation of cultural elements in the knowledge graph, reduces the deviation accumulation that causes the meaning of cultural visual elements to gradually deviate from the real historical background and cultural connotation, ensures that the result of regional culture visual presentation is consistent with the actual regional culture, and serious cultural deviation phenomenon does not occur. BRIEF DESCRIPTION OF DRAWINGS

[0043] The present application will be further described below with reference to the accompanying drawings.

[0044] Figure 1 A flowchart of the regional culture visual element dynamic reconstruction based on the knowledge graph;

[0045] Figure 2 A framework diagram of the regional culture visual element dynamic reconstruction system based on the knowledge graph. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0047] The present application provides a regional culture visual element dynamic reconstruction based on a knowledge graph. Referring to Figure 1 , Figure 1 A flowchart of the regional culture visual element dynamic reconstruction based on the knowledge graph provided by the present application. The method comprises the following steps:

[0048] S1: Collecting user behavior data in the process of interacting with regional culture visual elements, and mapping the behavior data into a semantic vector based on a pre-set multi-modal semantic analysis model, denoted as a user semantic behavior vector;

[0049] S2: selecting corresponding core cultural elements and semantic relationships thereof from the user semantic behavior vector, constructing a baseline semantic weight vector, and calculating a semantic shift gradient according to a difference between the user semantic behavior vector and the baseline semantic weight vector; and judging whether semantic space reconstruction is needed in the knowledge graph according to the semantic shift gradient;

[0050] S3: if semantic space reconstruction is needed in the knowledge graph, inputting the semantic shift gradient into a preset variational autoencoder model, combining the baseline semantic weight vector to perform semantic space reconstruction, generating a user behavior semantic vector after deviation elimination, and updating the node weight and relationship weight value of the regional culture knowledge graph;

[0051] S4: calculating an evaluation index of semantic space reconstruction based on the updated knowledge graph, and judging whether semantic space reconstruction is needed in the knowledge graph again according to the evaluation index, if not, taking the reconstructed knowledge graph as a benchmark to support dynamic reconstruction of subsequent regional culture visual elements.

[0052] Based on the regional culture visual element dynamic reconstruction method based on the knowledge graph provided in the embodiment of the application, through the above-mentioned manner, it can be judged whether the user behavior data can accurately reflect the real semantics of the cultural elements, the algorithm does not blindly perform visual presentation of regional culture according to the user behavior data, reduces the phenomenon that part of one-sided or misleading information is regarded as an effective signal, ensures stable semantic representation of the cultural elements in the knowledge graph, reduces deviation accumulation to cause the meaning of the cultural visual elements to gradually deviate from the real historical background and cultural connotation, ensures that the result of the visual presentation of the regional culture is consistent with the actual regional culture, and serious cultural deviation phenomenon does not occur.

[0053] In one embodiment, S1: collecting behavior data of a user in a regional culture visual element interaction process, and mapping the behavior data into a semantic vector based on a preset multi-modal semantic analysis model, as a user semantic behavior vector set;

[0054] Specifically, the user's behavior on the regional culture visual element interaction platform is diverse, generally including: clicking / viewing a certain cultural element (such as clicking to view high-definition pictures of Lingling stele carvings), collecting / liking a certain pattern (such as collecting a Yao pattern), searching for keywords (such as inputting "Nüshu blessing words"), combination operation (such as in the design interface, splicing a new visual scheme by combining Yao patterns and Nüshu symbols), dwell time, scroll track, zoom behavior, and other fine-grained interaction data; These data may be: visual modalities: pictures and videos browsed or uploaded by users; text modalities: search terms, comments, and tags input by users; structured modalities: element combination methods selected by users in the design interface; These data can be collected on the regional culture visual element interaction platform to collect user behavior data; In addition, the preset multi-modal semantic analysis model is used to map different types (text, image, video, structured data, etc.) of user interaction behaviors to a semantic vector space, so that they can be directly mapped to cultural element nodes in the knowledge graph.

[0055] Specifically, the process implemented by the preset multi-modal semantic analysis model is as follows:

[0056] Image semantic analysis: Use visual coding models (such as CLIP, ViT, etc.) to identify and extract labels for cultural elements in pictures. For example, identify the "Yunlei pattern" of "Yao pattern" in the picture and extract its feature vector;

[0057] Text semantic analysis: Use pre-trained language models such as BERT and RoBERTa to vectorize user input text (such as search term "Nüshu love poem") and map it to a cultural semantic space;

[0058] Cross-modal alignment: Use multi-modal alignment models (such as CLIP or self-trained cross-modal Transformers) to map visual feature vectors and text feature vectors to the same semantic space, so that "image + text + behavior trajectory" can be directly compared and integrated.

[0059] Behavior sequence encoding: Use sequence modeling (such as TransformerEncoder or LSTM) to encode a series of user behaviors into a comprehensive semantic vector, preserving interaction order and context information.

[0060] In addition, the regional cultural knowledge graph is composed of nodes (cultural elements) and edges (semantic relationships between elements). For example: nodes: Lingling Epigraphy (cultural relics), Nvshu symbols (language and writing), Yao patterns (national patterns); edges: “Lingling Epigraphy” — historical association — “Nvshu symbols”, “Yao patterns” — style integration — “Nvshu symbols”, etc. By mapping user behavior semantic vectors to the knowledge graph, it is determined which cultural elements are related to user behavior and how strong the relationship is by calculating the vector similarity between user behavior semantic vectors and each node in the graph. For example: assume that a user’s interaction process on the platform is: 1. search for “ancient love culture in Yongzhou”, 2. click to view a Lingling Epigraphy rubbing containing Nvshu symbols, 3. embed a Yao pattern into the frame of the epigraph image and save it as a personal work, 4. like another digital manuscript of a Nvshu love poem; the multi-modal semantic analysis process: the text “ancient love culture in Yongzhou” is encoded by BERT, and the obtained vector is highly similar to the nodes “Yongzhou love customs” and “Nvshu love poems” in the knowledge graph; the clicked epigraph image identifies the “Lingling Epigraphy” node and the “Nvshu symbols” node through a visual model. The combination design behavior is recorded as a new user-established decoration relationship between “Yao patterns” and “Lingling Epigraphy” through structured coding, and the like behavior strengthens the association weight between the user and the “Nvshu love poems” node. Finally, the system obtains the user semantic behavior vector of the user, for example: [Lingling Epigraphy: 0.82, Nvshu symbols: 0.90, Yao patterns: 0.65, Yongzhou love customs: 0.78], these values represent the user’s attention and association degree to each cultural element in the interaction, and correspond to the nodes in the knowledge graph one by one; through the above method, the user semantic behavior vector can be obtained according to the user’s behavior data.

[0061] In one embodiment, S2: select the corresponding core cultural elements and their semantic relationships from the user semantic behavior vector, construct a baseline semantic weight vector, and calculate the difference between the user semantic behavior vector and the baseline semantic weight vector to obtain a semantic shift gradient;

[0062] Specifically: according to the user semantic behavior vector obtained in S1, identify the cultural elements that the user has recently interacted with; for example, in the above example, the user’s interaction is focused on Yongzhou culture-related elements such as Lingling Epigraphy, Nvshu symbols, Yao patterns, etc., which are taken as the core elements corresponding to the user semantic behavior vector;

[0063] Using a graph database (such as Neo4j) or other graph query tools, the existing knowledge graph is queried for the adjacency structure, and nodes and semantic relationships directly or indirectly related to the core elements are found, including history, function, style, etc. For example: find nodes and semantic relationships (historical association, functional similarity, style inheritance, etc.) directly or indirectly connected to the core elements in the knowledge graph; for example: Lingling epitaph → association: epitaph era background, epitaph calligraphy style, historical events; female book symbol → association: women's communication culture, love custom, writing tool; Yao nationality pattern → association: clothing pattern, religious symbol, festival celebration; and filter core nodes and core relationships, core nodes are elements that can represent the essence of cultural themes and are irreplaceable in the cultural system (such as female book symbol, Lingling epitaph); core relationships are semantic relationships that can reflect the logic chain of culture (such as "carry", "originated from", "used for") and the like;

[0064] Construct a baseline semantic weight vector, which is a multi-dimensional vector, each dimension corresponds to a core cultural element, and the numerical value represents the standard semantic weight of the element in the real cultural context; the specific construction method of the baseline semantic weight vector can be:

[0065] Obtain the semantic weight of each node (core element) through multiple channels, and perform weighted fusion and normalization processing on all weights to obtain the baseline semantic value of each node. The baseline semantic value of each core element is used as the baseline semantic weight vector corresponding to the user semantic behavior vector;

[0066] Specifically, the multi-channel acquisition of the semantic weight of each node (core element) includes expert scoring weight, literature statistical weight, and knowledge graph historical version weight;

[0067] The expert scoring weight is manually scored by cultural experts and scholars according to the importance of the element (usually ranging from 0 to 1); for example: female book symbol: 0.95, Lingling epitaph: 0.85, Yao nationality pattern: 0.40.

[0068] The literature statistical weight is obtained by analyzing the frequency and relevance of elements in historical literature, papers, and museum data; for example: the female book symbol appears 80% in related literature, and appears 65% with love custom, with a weight of 0.9.

[0069] The knowledge graph historical version weight is to use the average weight of nodes and relations in the past version of the knowledge graph to reflect the long-term stability of the cultural elements. Among them, the expert scoring weight is derived from the existing systematic investigation and evaluation of experts and scholars in the field of organizational culture. Questionnaires, interviews or scoring meetings are used to quantitatively score the importance of core cultural elements. The literature statistics weight relies on natural language processing technology to automatically extract and count the frequency and co-occurrence relationship of cultural elements in a large number of historical literature, academic papers, museum collection databases and related digital resources, and calculates the weight value by combining text mining and semantic analysis. The knowledge graph historical version weight is obtained by saving and comparing the historical data of different time nodes of the knowledge graph, and counting the weight change and average value of each core element and its semantic relationship in the past version, so as to reflect the long-term stability and credibility of the cultural elements. The specific obtaining method is not limited and elaborated. The baseline semantic weight vector represents the standard expression of each core element and its relationship in an ideal cultural context. For example, in the baseline semantic weight vector of Yongzhou culture, the weight of female book symbols may be 0.95, the weight of Lingling inscriptions may be 0.85, and the weight of Yao nationality patterns may be 0.40. These values reflect the cultural core status and correlation strength of each element. Finally, the baseline semantic weight vector is used as a reference for comparison with the user semantic behavior vector, and the difference between the two is calculated to obtain the semantic shift gradient, which guides the subsequent semantic space correction and dynamic reconstruction of the knowledge graph.

[0070] In one implementation, the step of calculating the difference between the user semantic behavior vector and the baseline semantic weight vector to obtain the semantic shift gradient, and judging whether to perform semantic space reconstruction in the knowledge graph according to the semantic shift gradient is as follows:

[0071] The user semantic behavior vector and the baseline semantic weight vector are subtracted from the corresponding core element weight values to obtain the semantic shift gradient of each core element , wherein , ; and a semantic shift gradient vector is formed.

[0072] The norm of the semantic shift gradient vector is calculated , , and the norm of the semantic shift gradient vector is compared with a preset threshold. If the norm of the semantic shift gradient vector is less than the preset threshold, it indicates that the knowledge graph updated automatically according to the user behavior does not need to be reconstructed in the semantic space. If the norm of the semantic shift gradient vector is not less than the preset threshold, it indicates that the knowledge graph updated automatically according to the user behavior needs to be reconstructed in the semantic space.

[0073] It should be noted that the core of the above process is to determine whether the difference between the user's current cultural understanding and the baseline cultural semantics is large enough to correct the knowledge graph. Specifically, the system compares the user semantic behavior vector with the baseline semantic weight vector on each core element to obtain the semantic offset value of each element. These offset values combined are the semantic offset gradient vector. This vector reflects the overall deviation between the user's understanding and the real cultural context. Then, the system calculates the overall size of the offset gradient and compares it with the preset threshold: if the overall size is less than the threshold, it means that the knowledge graph changes caused by user behavior are small in amplitude, and the cultural understanding is basically correct, and there is no need to reconstruct the semantic space; if the overall size is not less than the threshold, it means that the user's current cultural cognition has deviated significantly from the baseline, and the reconstruction process needs to be started to calibrate the knowledge graph. For example, under the theme of "Yongzhou Love Culture", assuming the baseline vector is [0.95, 0.90, 0.85, 0.40] (corresponding to "Nvshu symbol", "Yongzhou love custom", "Lingling epitaph", "Yao nationality pattern"), and the user vector is [0.60, 0.40, 0.90, 0.75], then the difference vector is [−0.35, −0.50, 0.05, 0.35], and the norm is about 0.74. If the system sets the threshold to be 0.5, then because 0.74>0.5, it will trigger the semantic space reconstruction of the knowledge graph to reduce the deviation that "Nvshu symbol" and "love custom" are underestimated, and "Yao nationality pattern" is overemphasized. In this way, the ability of dynamic learning of the graph can be maintained, and it can also prevent it from gradually distorting in long-term automatic updating.

[0074] In one embodiment, S3: if semantic space reconstruction is needed in the knowledge graph, input the semantic offset gradient into a preset variational autoencoder model, combine the baseline semantic weight vector for semantic space reconstruction, generate a user behavior semantic vector after deviation elimination, and update it to the regional cultural knowledge graph;

[0075] In one implementation, the step of inputting the semantic offset gradient into a preset variational autoencoder model, combining the baseline semantic weight vector for semantic space reconstruction, and generating a user behavior semantic vector after deviation elimination is:

[0076] The semantic offset gradient vector is normalized, and the normalized semantic offset gradient vector is spliced with the baseline semantic weight vector to form a joint input vector .

[0077] Wherein, , represents the first element in the normalized semantic offset gradient vector . The semantic offset of each core element; , Indicates will and Dimensionally stitched;

[0078] Joint input vector Input to the preset variational autoencoder The encoder of the model obtains the mean vector of the latent variables. With variance vector And obtain the latent variable vector based on reparameterized sampling. ; In the formula, and Let these represent the mean vector and standard deviation vector of the latent variables, respectively. This indicates element-wise multiplication. It indicates that it follows a standard normal distribution. random noise, This represents the unit covariance matrix (i.e., each dimension is independent and has a variance of 1); by multiplying it with the standard deviation vector σ, randomness is introduced, allowing the sampling process of the VAE to be backpropagated for training (reparameterization technique).

[0079] latent variable vector The input is fed into the decoder of the variational autoencoder model, and the output is a debiased user semantic vector with the same dimension as the user semantic behavior vector. The purpose is to make the debiased vector closer to the baseline semantic weight vector while retaining the effective information in the user behavior.

[0080] The debiased user semantic vector is used as the new user semantic representation, and the weight values ​​and semantic relationship weights of the corresponding nodes in the regional cultural knowledge graph are updated.

[0081] It should be noted that inputting the semantic offset gradient into the preset variational autoencoder (VAE) model and combining it with the baseline semantic weight vector for semantic space reconstruction aims to leverage the powerful nonlinear modeling and bias removal capabilities of VAE to generate a more objective user semantic representation that eliminates biases caused by user behavior, thereby ensuring the stability and authenticity of the regional cultural knowledge graph. The specific steps are as follows:

[0082] Firstly, the semantic shift gradient vector is normalized to limit its numerical range within a unified scale interval, which can avoid training instability caused by excessively large or small values, and make it easier for the model to capture the relative differences in shift gradient. The normalized semantic shift gradient vector and the baseline semantic weight vector are concatenated in dimension to form a joint input vector. Here, the normalized shift gradient provides the deviation information between the current user behavior and the true culture, while the baseline semantic weight vector serves as a standard semantic reference to help the model understand the correct semantic meaning of cultural elements. The combination of the two enables the model to perceive the shift and grasp the core features of the culture.

[0083] Next, the joint input vector is input into the encoder part of the pre-set variational autoencoder model, which extracts the semantic features from the input and outputs the mean vector and standard deviation vector of the latent variable. Here, the mean vector reflects the latent, de-biased semantic center position, and the standard deviation vector is used to introduce certain random noise to make the model have certain flexibility and generalization ability when generating. Using the reparameterization technique, the latent variable vector is generated by linear combination of the mean and standard deviation plus random noise following the standard normal distribution. This not only makes the sampling process of VAE derivable, facilitating backpropagation optimization, but also ensures that the model will not be over-fitted, effectively avoiding over-adjustment caused by the contingency or bias of user behavior data.

[0084] Then, the latent variable vector is input into the decoder part of the variational autoencoder, which maps the latent variable back to the semantic space with the same dimension as the user semantic behavior vector, and outputs the de-biased user semantic vector. This vector, compared to the original user behavior vector, eliminates noise and bias, and more accurately reflects the true semantic meaning of cultural elements.

[0085] Finally, the generated de-biased user semantic vector is used as the new user semantic representation, and the system automatically updates the weights of the corresponding nodes and the semantic relationship weights between nodes in the regional cultural knowledge graph based on the de-biased user semantic vector. In this way, the knowledge graph not only integrates the latest information from user behavior, but also avoids semantic drift caused by one-sided or misleading behavior through the VAE de-biasing mechanism, ensuring the stability and authority of cultural semantics. For example, suppose the user's attention to "Nü Shu symbols" is abnormally high, causing the semantic shift gradient to show that the element weight exceeds the baseline by a large margin. After normalization, the shift and the baseline semantic weight vector are input into VAE, the encoder estimates the latent de-biased center, adds random noise for sampling, and the decoder outputs a vector that adjusts the weight of "Nü Shu symbols" back to a more reasonable range, preventing the continuous spread of cultural misunderstandings caused by single user behavior in the knowledge graph. In this way, even in the face of individual user preferences or behavior abnormalities, the authenticity and stability of cultural element semantics can be maintained.

[0086] It should be noted that the above-mentioned fusion of user behavior bias and baseline cultural semantics, and the use of nonlinear mapping and probability generation capability of variational autoencoder effectively removes the bias, optimizes the dynamic update of cultural knowledge graph, and ensures that the dynamic reconstruction of cultural visual elements not only reflects user demand, but also is faithful to the essence of culture.

[0087] In one embodiment, S4: based on the updated knowledge graph, calculate the evaluation index of semantic space reconstruction, and determine whether the knowledge graph needs to continue semantic space reconstruction according to the evaluation index of semantic space reconstruction, if not, the reconstructed knowledge graph is taken as the benchmark to support the dynamic reconstruction of regional cultural visual elements;

[0088] In one implementation, the step of calculating the evaluation index of semantic space reconstruction is:

[0089] The evaluation index of semantic space reconstruction includes the element correction reasonable index of the updated knowledge graph and the multimodal fusion consistency index of user behavior, the element correction reasonable index and the multimodal fusion consistency index are normalized to map to the interval of 0-1, and the normalized element correction reasonable index and multimodal fusion consistency index are weighted and summed to obtain the evaluation index of semantic space reconstruction. The formula for calculating the evaluation index of semantic space reconstruction is: , wherein is the evaluation index, and are the normalized element correction reasonable index and multimodal fusion consistency index, respectively, and represent the preset weight values of the normalized element correction reasonable index and multimodal fusion consistency index, respectively, and are both greater than 0;

[0090] It should be noted that the above-mentioned normalization dimension removal method includes Min-Max normalization, Z-Score standardization, etc., which will not be described here; According to the actual situation, generally are equal and the sum is 1, for example, may be 0.5, 0.5.

[0091] In one embodiment, the calculation step of the element correction reasonable index is:

[0092] The semantic weight vector corresponding to the core element is extracted from the updated knowledge graph, denoted as the reconstructed semantic weight vector , and the baseline semantic weight vector and the reconstructed semantic weight vector are normalized to obtain the baseline probability distribution and the reconstruction probability distribution ; and based on the baseline probability distribution With reconstruction probability distribution Calculate the corresponding Shannon entropy and And calculate the two Shannon entropies. and The absolute difference is calculated and divided by the Shannon entropy corresponding to the baseline probability distribution. And subtract the absolute difference from the value of 1 and divide by the Shannon entropy corresponding to the baseline probability distribution. The result yields the structural entropy difference. The calculation formula is: In the formula, Shannon entropy , In the formula, Indicates the first Baseline probability distribution of each core element Indicates the first The reconstruction probability distribution of each core element , , and They represent the first Baseline semantic weights and reconstructed semantic weights of each core element; The total number of core elements;

[0093] The values ​​corresponding to the baseline semantic weight vectors are reordered in descending order to obtain the baseline semantic ranking sequence. The values ​​corresponding to the reconstructed semantic weight vectors are then reordered in descending order to obtain the reconstructed semantic sorting sequence. And calculate the baseline semantic ordering sequence. With the reconstructed sorted sequence The distance between the Kendall's Tau rank correlation coefficients is used to obtain the ordinal correlation index. The calculation formula is: In the formula, This indicates the number of inconsistent pairs in the sorted sequence. The number of elements The value range is from 0 to 1. A larger value indicates a more consistent sorting.

[0094] The structural entropy difference Multiply by the ranking correlation index The element correction rationality index is obtained.

[0095] It should be noted that the reconstructed semantic weight can be automatically updated by the knowledge graph after the semantic space reconstruction, and the semantic weight vector of the core cultural elements is directly extracted from the latest reconstruction output of the reconstruction model (such as variational autoencoder).

[0096] It should be noted that the element correction rationality index is used to measure the degree of coincidence of the baseline semantics of the reconstructed knowledge graph in the two key dimensions of information structure and element hierarchy, and its calculation considers both structure entropy difference and sorting correlation. The structure entropy difference reflects the closeness of the distribution of the semantic weight of the core elements after reconstruction to the baseline distribution in terms of overall information quantity and diversity. If the weight distribution is not significantly imbalanced during the reconstruction process, the change in Shannon entropy is small, the structure entropy difference is close to 1, and the overall information structure of the cultural semantics is effectively preserved. The sorting correlation measures the consistency of the importance sorting of the core elements after reconstruction with the baseline sorting through the Kendall rank correlation coefficient distance. If the sorting is almost not disturbed, the correlation index is close to 1, indicating that the semantic hierarchy structure is stable. The element correction rationality index is the product of the structure entropy difference and the sorting correlation index. Only when both of them remain at a high level, the index is close to 1, meaning that the reconstruction effectively corrects the deviation and does not damage the cultural semantic structure. When the index value is large, it means that the reconstruction has achieved the dual goals of removing the influence of abnormal user behavior and maintaining the stability of cultural semantics. Re-executing the semantic space reconstruction of step S3 not only wastes computational resources, but also may introduce new disturbances or overfitting, thereby reducing the stability and reliability of the knowledge graph. On the contrary, when the index value is low, it means that the reconstructed knowledge graph has a large deviation from the baseline in terms of distribution information structure or element sorting hierarchy, and may not be able to fully correct the deviation or has damaged the cultural semantics, so it needs to be re-executed S3 for secondary reconstruction. For example, if the semantic weight distribution of "Nüshu symbols, Yongzhou love customs, Lingling inscriptions, and Yao nationality patterns" in the baseline is balanced and the sorting is stable, and after reconstruction, the entropy value and the sorting are highly consistent, the element correction rationality index is close to 1, and there is no need to repeat S3. But if the weight distribution is concentrated on a single element and the sorting is disturbed after reconstruction due to abnormal data, the index will decrease significantly, triggering the reconstruction again to restore the semantic rationality and structural stability.

[0097] It should be noted that the advantages of calculating the element correction reasonable index by the above method are that the method quantitatively evaluates the reconstruction effect from two complementary dimensions of global information structure and local element importance, the structure entropy difference can capture the deviation of the semantic weight distribution after reconstruction in the overall information quantity and diversity, and prevent the stability of the overall semantic structure from being ignored by only focusing on individual elements; the sorting correlation index measures the degree of maintaining the importance level of the core element, avoiding the implicit damage of overturning the semantic primary and secondary relationship in the case of close entropy value. The index formed by multiplying the two has a "short board effect", that is, any dimension significantly deviates, which will cause the overall score to decrease, thereby ensuring the reliability and sensitivity of the evaluation result, avoiding the misjudgment risk caused by using a single index, and more accurately reflecting the fidelity of cultural semantics and the effectiveness of reconstruction than the conventional weighted sum or single dimension evaluation.

[0098] In one embodiment, the calculation steps of the multi-modal fusion consistency index are:

[0099] Extracting a set of semantic embedding vectors of different modalities from the updated knowledge graph; for example, modality A is a set of text description vectors of regional culture, such as the text semantic vector of "Zerling Monument", and modality B is a set of visual symbol vectors corresponding to the image feature vector of the text of "Zerling Monument".

[0100] For any two modalities A and B, denoted as a modality pair, the corresponding set of semantic embedding vectors is denoted as modality set A and modality set B, the joint occurrence frequency of the corresponding core elements in the two modalities is counted, and the joint probability distribution is calculated wherein represents the probability of the th element in modality set A and the th element in modality set B appearing simultaneously, satisfying , for example, the proportion of the number of times that a certain keyword in the text of "Zerling Monument" and a certain pattern in the image of the monument appear together to the number of times of all text and image combinations appears;

[0101] Calculate the marginal probability distribution of modality set B wherein each marginal probability is , represents the probability of the th element in modality B appearing; for example, the probability of a certain pattern appearing in the image of "Zerling Monument" is the cumulative probability of the pattern in modality B in all joint probability of appearance;

[0102] Calculate the conditional probability distribution wherein the conditional probability is ,

[0103] In the modality set A, the th Given that the nth element appears, the modality set B contains the nth element. The probability of each element appearing ensures that for each... satisfy For example, given a keyword in the text "Lingling Stele Inscriptions", calculate the probability distribution of different image patterns appearing under the condition of that keyword;

[0104] Based on conditional probability distribution and marginal probability distribution Calculate the conditional entropy of mode A and mode B (mode pair AB). and marginal entropy The formula for calculating conditional entropy is: ; represents the uncertainty of mode B given information about mode A; the marginal entropy calculation formula is: ; indicates the uncertainty of information in mode B itself; for example, when the text information of "Lingling Stele Inscription" clearly corresponds to certain image patterns, the conditional entropy is low; if the image patterns are diverse and evenly distributed, the edge entropy is high;

[0105] The marginal entropy of mode A and mode B Subtract conditional entropy Then divide the result of the subtraction by the marginal entropy. The fusion consistency index of modality A and modality B is obtained. This index reflects the proportion of uncertainty reduction in modality B information given modality A information. The closer the value is to 1, the stronger the fusion consistency between modalities, and the higher the semantic complementarity and sharing. For example, if the text and image of "Lingling Stele Inscription" highly correspond, the index is close to 1, indicating that the text and visual symbol are well integrated; conversely, if the index is low, there may be semantic disconnect or deviation.

[0106] The fusion consistency index of all modal pairs is obtained by summing the fusion consistency indices.

[0107] It should be noted that the "core element" mentioned in the above multi-modal fusion consistency index calculation step refers to those representative and key nodes or entities in the regional culture knowledge graph. These nodes have corresponding expressions or features in different modal data. Specifically, the core element can be an important cultural symbol, a landmark visual element, a typical text description, or a sound symbol in a certain regional culture, etc. They are the basic objects of multi-modal fusion. For example, in Yongzhou culture, "Liling Epigraphy" as a core element, has corresponding epigraphic texts (text modality), corresponding epigraphic images (visual modality), and possibly corresponding sound symbols or music (sound modality). The extraction of core elements ensures the correspondence and semantic relevance of cross-modal data, so that the joint probability and conditional probability calculated have clear semantic direction, thus effectively evaluating the consistency of multi-modal fusion. In short, the core element is those key cultural symbols or entities in multi-modal data that are simultaneously focused on and analyzed, and is the basic unit for cross-modal fusion calculation.

[0108] It should be noted that the data involved in the above multi-modal fusion consistency index calculation process is mainly derived from the updated regional culture knowledge graph and its corresponding multi-modal original data. First, through the identification association of knowledge graph nodes and entities, the multi-modal data related to core cultural elements such as text description, visual symbol, and sound feature are extracted to form a corresponding semantic embedding vector set. Second, these vector sets are matched through structured mapping relationships to identify the corresponding core elements and their frequencies in each modality, and then to count the joint occurrence times and marginal occurrence probabilities. These data acquisition processes rely on semantic analysis, image feature extraction, sound signal analysis, and other technical means to ensure that semantic nodes between different modalities can be effectively corresponded and jointly analyzed, thus providing accurate basic data for calculating joint probability distribution, marginal probability distribution, and conditional probability distribution.

[0109] It should be noted that the multi-modal fusion consistency index refers to a numerical index for measuring the semantic fusion quality and consistency between different modal data (such as text description, visual symbols, sound symbols, etc.) in the updated knowledge graph. Through the index, the user can intuitively understand the complementarity and coordination degree of information between different modalities, i.e., whether different modalities express the same or related regional cultural elements, and whether the semantics are highly matched and fused. The larger the index, the closer the semantic correspondence of multi-modal information in the knowledge graph, and the more mutual confirmation and support between text and image, sound and other forms of expression, reducing information conflict and modal bias, and reflecting the stability and accuracy of the semantic space of the knowledge graph. This means that the current knowledge graph structure has better reflected the user's multi-modal interaction behavior and regional cultural connotation, and the necessity of further reconstruction is reduced. For example, if the key symbols and historical background mentioned in the text description of "Liling Stele" can be fully reflected and matched in the corresponding stele image and related sound recording, the multi-modal fusion consistency index will be higher, indicating that the multi-modal content fusion effect of text, vision, sound, etc. is good, the semantic space of the knowledge graph is stable, and the semantic space reconstruction of step S3 does not need to be frequently performed; otherwise, if there is a large semantic difference or mismatch between the text and the visual symbols, the index is low, indicating that the fusion effect is poor, and the knowledge graph needs to be adjusted through step S3 to improve the semantic consistency and completeness. In summary, the larger the multi-modal fusion consistency index, the higher the depth and accuracy of semantic fusion, and the more reasonable and robust the semantic space structure of the knowledge graph, thereby reducing the need for repeated reconstruction, ensuring that the dynamic reconstruction of regional cultural visual elements can be based on a reliable and highly fused knowledge base, and achieving more accurate and efficient cultural expression and application.

[0110] It should be noted that the advantage of calculating the multi-modal fusion consistency index in the above manner is that it is based on the relative relationship between conditional entropy and marginal entropy, can accurately quantify the reduction of information uncertainty between different modalities, and directly reflects the semantic complementarity and fusion tightness of one modality given the information of another modality. Compared with simple similarity or correlation coefficient calculation, this method pays more attention to the actual sharing and supplement of information, avoiding the surface consistency misjudgment caused by simple numerical matching. In addition, the introduction of conditional entropy makes the calculation consider the dependency relationship and information transmission between modalities, which can reveal hidden semantic associations and improve the discrimination accuracy of fusion consistency.

[0111] In one embodiment, the step of judging whether the knowledge graph needs to be reconstructed again according to the evaluation index includes:

[0112] The evaluation index is compared with the preset threshold value. If the evaluation index is not less than the preset threshold value, it indicates that the semantic space reconstruction effect of the knowledge graph in step S3 is qualified, and it is not necessary to continue to repeatedly perform semantic space reconstruction of the knowledge graph. The reconstructed knowledge graph in S3 is directly taken as a reference to support dynamic reconstruction of subsequent regional culture visual elements.

[0113] In an implementation manner, the step of judging whether the knowledge graph needs to be reconstructed again according to the evaluation index further includes:

[0114] If the evaluation index is less than the preset threshold value, it indicates that the semantic space reconstruction effect of the knowledge graph in step S3 is unqualified, and it is necessary to continue to repeatedly perform the semantic space reconstruction process of the knowledge graph in step S3 until the evaluation index is not less than the preset threshold value to stop the reconstruction, and the reconstructed knowledge graph is taken as a reference to support dynamic reconstruction of subsequent regional culture visual elements.

[0115] It should be noted that, according to the comparison result of the evaluation index and the preset threshold value, it can be dynamically judged whether the knowledge graph needs to be further reconstructed. When the evaluation index reaches or exceeds the preset threshold value, it indicates that the semantic space reconstruction in step S3 has reached the expected rationality and multi-modal fusion consistency, which means that the current knowledge graph is relatively perfect in structure and semantic expression, and can accurately reflect the behavior characteristics of the user and the diversity of the regional culture, so it is not necessary to continue to repeatedly reconstruct, and the knowledge graph at this time can be taken as a stable reference to provide reliable support for the dynamic reconstruction of subsequent regional culture visual elements. Conversely, if the evaluation index is lower than the preset threshold value, it means that the current knowledge graph has not reached sufficient rationality or modal fusion effectiveness in the semantic space adjustment, and there is information deviation or fusion deficiency. At this time, step S3 needs to be restarted to continue the semantic space reconstruction based on the latest semantic shift gradient, to optimize the node weight and relationship weight of the knowledge graph, so as to better adapt to the complex diversity of user behavior and cultural elements. The reconstruction process will continue to iterate until the evaluation index meets or exceeds the threshold value, so as to ensure that the final knowledge graph can efficiently and accurately reflect the actual semantic relationship, and avoid model instability or misjudgment caused by premature stopping. For example, assuming that the evaluation index of the "Liling Stele Carving" element of a certain regional culture after initial reconstruction is 0.65, and the threshold value is set to 0.8. Since it does not meet the standard, it automatically triggers reconstruction again, and by correcting the weight and relationship, the fusion consistency is improved, and finally the evaluation index reaches 0.85, confirming that the knowledge graph is sufficiently reasonable, so the reconstruction process is stopped to support the accurate dynamic reconstruction of subsequent visual elements. Through this adaptive judgment mechanism based on the evaluation index, not only the effectiveness of the knowledge graph reconstruction is improved, but also unnecessary repeated calculation is avoided, and the overall response speed and stability are improved.

[0116] It should be noted that the reason for reconstruction instead of directly depriving the update of step S3 in the knowledge graph which has been automatically updated according to user behavior data before, and returning to the knowledge graph before the update, is that the benefits of the present application are as follows: first, the updating process of the knowledge graph is based on real user behavior data and dynamic feedback of multi-modal information, and directly falling back will lose these valuable personalized and time-sensitive semantic information, resulting in the system being unable to accurately reflect the latest development trend of current user demand and regional culture. Through semantic space reconstruction, the system can make detailed corrections to the semantic deviation or unreasonable part that may exist in the update, so that the knowledge graph not only retains the effective information brought by the user behavior, but also eliminates noise and deviation, and improves the overall semantic expression rationality and accuracy. Secondly, the semantic space reconstruction adopts a deep learning model such as a variational autoencoder, which can capture complex nonlinear relationships and potential semantic structures, and realize fine adjustment of the node weights and relationship weights of the knowledge graph. This way is more flexible and adaptive than simple rollback, can dynamically optimize the knowledge graph structure, avoid information loss, and at the same time improve the performance of the knowledge graph in multi-modal fusion and semantic consistency, and enhance the effect and stability of subsequent dynamic reconstruction of visual elements. For example, suppose that "Lingling Stele" shows new cultural attention in recent user interactions, such as preference for certain stele patterns, and step S3 automatically updates the knowledge graph to reflect this preference, but part of the update may introduce false associations due to data noise. If the update is directly revoked, the latest behavior information of the user cannot be utilized; and through semantic space reconstruction, the system can effectively filter noise and adjust weights, so that the final knowledge graph can reflect user preferences while ensuring semantic structure rationality, improving overall knowledge quality and user experience. In summary, through semantic space reconstruction, the present scheme realizes the preservation of dynamic update information while strengthening semantic correction and optimization, avoids information loss and system performance degradation caused by simple rollback, and guarantees the accuracy and continuity of dynamic reconstruction of regional culture visual elements

[0117] Based on the same inventive concept, the present application also provides a knowledge graph-based dynamic reconstruction system of regional culture visual elements. Referring to Figure 2 , Figure 2 The framework diagram of the knowledge graph-based dynamic reconstruction system of regional culture visual elements provided by the present application embodiment, the system comprises:

[0118] User semantic behavior module: collect user behavior data in the process of interacting with regional culture visual elements, and map the behavior data into a semantic vector based on a pre-set multi-modal semantic analysis model, denoted as a user semantic behavior vector;

[0119] The preliminary judgment module selects the corresponding core cultural elements and semantic relationships from the user semantic behavior vector, constructs a baseline semantic weight vector, and calculates the difference between the user semantic behavior vector and the baseline semantic weight vector to obtain a semantic shift gradient; and judges whether semantic space reconstruction is needed in the knowledge graph according to the semantic shift gradient;

[0120] The reconstruction module inputs the semantic shift gradient into the preset variational autoencoder model if semantic space reconstruction is needed in the knowledge graph, combines the baseline semantic weight vector to perform semantic space reconstruction, generates a user behavior semantic vector after deviation elimination, and updates the node weight and relationship weight value of the regional culture knowledge graph;

[0121] The re-judgment module calculates an evaluation index of semantic space reconstruction based on the updated knowledge graph, and judges whether semantic space reconstruction is needed in the knowledge graph again according to the evaluation index, and if not, the reconstructed knowledge graph is taken as a benchmark to support the dynamic reconstruction of regional culture visual elements.

[0122] The regional culture visual element dynamic reconstruction system based on the knowledge graph provided by the embodiment of the present application can judge whether the user behavior data can accurately reflect the real semantics of the cultural elements through the above-mentioned manner, the algorithm does not blindly perform visual presentation of regional culture according to user behavior data, reduces the phenomenon of regarding some one-sided or misleading information as effective signals, ensures stable semantic representation of cultural elements in the knowledge graph, reduces the deviation accumulation that causes the meaning of cultural visual elements to gradually deviate from the real historical background and cultural connotation, ensures that the result of regional culture visual presentation is consistent with the actual regional culture, and serious cultural deviation phenomenon does not occur.

[0123] The above describes one embodiment of the present application in detail, but the content is only the preferred embodiment of the present application, and cannot be considered as limiting the scope of the present application. Any equivalent changes and improvements made according to the scope of the present application should still belong to the patent coverage range of the present application.

Claims

1. A method for dynamic reconstruction of regional culture visual elements based on a knowledge graph, characterized in that, Comprise the following steps: S1: Collect the behavior data of the user in the process of interacting with the regional culture visual elements, and map the behavior data to a semantic vector based on a pre-set multi-modal semantic analysis model, denoted as a user semantic behavior vector; S2: Select the corresponding core cultural elements and their semantic relationships from the user semantic behavior vector, construct a baseline semantic weight vector, and calculate the difference between the user semantic behavior vector and the baseline semantic weight vector to obtain a semantic shift gradient; And according to the semantic shift gradient, judge whether to perform semantic space reconstruction on the knowledge graph, including: Subtract the core element weight values corresponding to the user semantic behavior vector and the baseline semantic weight vector to obtain the semantic shift gradient of each core element, forming a semantic shift gradient vector; Calculate the norm of the semantic shift gradient vector, and compare the norm of the semantic shift gradient vector with a pre-set threshold value; if the norm of the semantic shift gradient vector is less than the pre-set threshold value, it indicates that the knowledge graph updated automatically according to the user behavior does not need to be reconstructed in the semantic space; and if the norm of the semantic shift gradient vector is not less than the pre-set threshold value, it indicates that the knowledge graph updated automatically according to the user behavior needs to be reconstructed in the semantic space; S3: If the semantic space reconstruction needs to be performed on the knowledge graph, normalize the semantic shift gradient vector, and concatenate the normalized semantic shift gradient vector with the baseline semantic weight vector to form a joint input vector; Input the joint input vector into the encoder of the pre-set variational autoencoder model to obtain the mean vector and variance vector of the latent variable, and obtain the latent variable vector based on the reparameterization sampling; Input the latent variable vector into the decoder of the variational autoencoder model to output a user semantic vector after deviation that has the same dimension as the user semantic behavior vector; the purpose is to make the user semantic vector after deviation closer to the baseline semantic weight vector while retaining the effective information in the user behavior; Use the user semantic vector after deviation as a new user semantic representation to update the weight values of the corresponding nodes and their semantic relationship weight values in the regional culture knowledge graph; S4: Based on the updated knowledge graph, calculate an evaluation index of semantic space reconstruction, and according to the evaluation index, judge whether the semantic space reconstruction of the knowledge graph needs to be performed again; if not, use the reconstructed knowledge graph as a benchmark to support the dynamic reconstruction of subsequent regional culture visual elements; the evaluation index of semantic space reconstruction includes an element correction rationality index of the updated knowledge graph and a multi-modal fusion consistency index of the user behavior, the element correction rationality index is used to measure the degree of consistency of the updated knowledge graph with the baseline semantic in the two key dimensions of information structure and element level; the multi-modal fusion consistency index is used to measure the numerical index of the semantic fusion quality and consistency between different modal data in the updated knowledge graph.

2. The knowledge graph-based regional culture visual element dynamic reconstruction method according to claim 1, characterized in that, The steps for calculating the evaluation index of semantic space reconstruction are: Normalize the element correction rationality index and the multi-modal fusion consistency index to map them to the interval of 0-1, and weight-sum the normalized element correction rationality index and the multi-modal fusion consistency index to obtain the evaluation index of semantic space reconstruction. 3.The knowledge graph-based regional culture visual element dynamic reconstruction method according to claim 2, characterized in that, The element correction reasonable index calculation step is: The core element corresponding semantic weight vector is extracted from the updated knowledge graph, denoted as a reconstructed semantic weight vector, and the baseline semantic weight vector and the reconstructed semantic weight vector are normalized to obtain a baseline probability distribution and a reconstructed probability distribution; the corresponding Shannon entropy is calculated according to the baseline probability distribution and the reconstructed probability distribution, the absolute difference of the two Shannon entropies is calculated, the absolute difference is divided by the corresponding Shannon entropy of the baseline probability distribution, and the result of subtracting the absolute difference from the value 1 divided by the corresponding Shannon entropy of the baseline probability distribution is obtained to obtain a structure entropy difference value; reordering the numerical values corresponding to the baseline semantic weight vector in descending order to obtain a baseline semantic ranking sequence reordering the numerical values corresponding to the reconstructed semantic weight vector in descending order to obtain a reconstructed semantic ranking sequence , and calculating the Kendall rank correlation coefficient distance between the baseline semantic ranking sequence and the reconstructed ranking sequence to obtain a ranking correlation index The formula for calculating is: , wherein, the number of inconsistent pairs in the ranking sequence, the number of elements; The structural entropy difference value is multiplied by the ranking correlation index to obtain the element correction rationality index.

4. The knowledge graph-based dynamic reconstruction method of regional culture visual elements according to claim 2, characterized in that, The multi-modal fusion consistency index calculation step includes: extract a set of semantic embedding vectors of different modalities from the updated knowledge graph; for any two modalities A and B, denoted as a modal pair, and the corresponding set of semantic embedding vectors is denoted as modal set A and modal set B, the joint occurrence frequency of the corresponding core elements in the two modalities is counted, and the joint probability distribution is calculated wherein, denotes the probability of the simultaneous occurrence of the th element in the modal set A and the th element in the modal set B; Calculate the marginal probability distribution of mode set B Each edge probability for , Indicates the second mode. The probability of each element appearing; Computing conditional probability distributions where the conditional probability is , represents the probability of the occurrence of the element of the modal set B in the condition that the element of the modal set A satisfies .​​ 5. The knowledge graph-based dynamic reconstruction method of regional culture visual elements according to claim 4, characterized in that, The multi-modal fusion consistency index calculation step further includes: Based on conditional probability distributions and marginal probability distributions , the conditional entropy of modality A and modality B is calculated and the marginal entropy , the conditional entropy calculation formula is: ; the marginal entropy calculation formula is: ; The marginal entropy of mode A and mode B Subtract conditional entropy Then divide the result of the subtraction by the marginal entropy. We obtain the fusion consistency index of mode A and mode B, and then sum the fusion consistency indices of all mode pairs to obtain the multimodal fusion consistency index. 6.The knowledge graph-based regional culture visual element dynamic reconstruction method according to claim 1, characterized in that, The step of judging whether the semantic space reconstruction of the knowledge graph needs to be performed again according to the evaluation index includes: The evaluation index is compared with the preset threshold value, if the evaluation index is not less than the preset threshold value, it indicates that the semantic space reconstruction effect of the knowledge graph in step S3 is qualified, and the semantic space reconstruction of the knowledge graph does not need to be repeated, and the reconstructed knowledge graph in S3 is directly used as a reference to support the dynamic reconstruction of the regional culture visual elements.

7. The knowledge graph-based dynamic reconstruction method of regional culture visual elements according to claim 6, characterized in that, The step of judging whether the semantic space reconstruction of the knowledge graph needs to be performed again according to the evaluation index includes: If the evaluation index is less than the preset threshold value, it indicates that the semantic space reconstruction effect of the knowledge graph in step S3 is not qualified, and the semantic space reconstruction process of the knowledge graph in step S3 needs to be repeated, until the evaluation index is not less than the preset threshold value to stop the reconstruction, and the reconstructed knowledge graph is used as a reference to support the dynamic reconstruction of the regional culture visual elements.

8. A knowledge graph-based dynamic reconstruction system of regional culture visual elements, used to implement the knowledge graph-based dynamic reconstruction method of regional culture visual elements according to any one of claims 1-7, characterized in that, The system includes: A user semantic behavior module: collects behavior data of a user in the process of interacting with regional culture visual elements, and maps the behavior data into a semantic vector based on a preset multi-modal semantic analysis model, denoted as a user semantic behavior vector; A preliminary judgment module: selects corresponding core culture elements and their semantic relationships from the user semantic behavior vector, constructs a baseline semantic weight vector, and calculates the semantic deviation gradient according to the difference between the user semantic behavior vector and the baseline semantic weight vector; and judges whether the semantic space reconstruction of the knowledge graph is needed according to the semantic deviation gradient; A reconstruction module: if the semantic space reconstruction of the knowledge graph is needed, the semantic deviation gradient is input into a preset variational autoencoder model, the semantic space is reconstructed in combination with the baseline semantic weight vector, a user behavior semantic vector after deviation is generated, and the node weight and relationship weight of the regional culture knowledge graph are updated; A second judgment module: based on the updated knowledge graph, an evaluation index of the semantic space reconstruction is calculated, and whether the semantic space reconstruction of the knowledge graph needs to be performed again is judged according to the evaluation index, if not, the reconstructed knowledge graph is used as a reference to support the dynamic reconstruction of the regional culture visual elements.

Images