Method and system for discovering time-space dependence rule of cavern wall painting
By performing feature fusion and confidence score calculation on cave mural data, the shortcomings of temporal knowledge graph models in learning spatiotemporal dependent features are addressed, thereby improving prediction accuracy.
Patent Information
- Application Number
- CN202511086703.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-05
- Publication Date
- 2026-01-23
AI Technical Summary
Existing temporal knowledge graph models struggle to fully learn the spatiotemporal dependencies of historical events, resulting in insufficient accuracy in predicting future events.
A temporal knowledge graph reasoning model is trained using cave mural data. Relational embeddings and entity embeddings are obtained through relation encoders and entity encoders. A time vector generator is used to generate time vectors. Visual features are extracted through graph convolutional networks. Feature fusion and confidence score calculation are performed to train the model.
It improves the model's inference accuracy on temporal knowledge graphs, enabling it to better learn spatiotemporal dependency features, thereby enhancing the accuracy of future fact predictions.
Smart Images
Figure CN121390133A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of temporal knowledge graph reasoning in knowledge graph, in particular to a method and system for discovering spatiotemporal dependency law of cave murals. BACKGROUND
[0002] Temporal knowledge graphs have been widely applied in various fields to model the dynamics of facts on a timeline. In the extrapolation setting of temporal knowledge graph reasoning, since future facts are completely unknown, understanding history is the key to predicting future facts.
[0003] However, it is still a great challenge for existing models, as they are difficult to fully learn the spatiotemporal dependency features of historical events. From the perspective of historical development, considering the spatiotemporal law of historical facts is beneficial to predict future facts.
[0004] However, there is no effective solution to the problems in the related art. SUMMARY
[0005] To overcome the above technical problems existing in the prior art, the present application provides a method and system for discovering spatiotemporal dependency law of cave murals.
[0006] To this end, the specific technical solutions adopted by the present application are as follows:
[0007] In a first aspect, the present application provides a method for discovering spatiotemporal dependency law of cave murals, comprising the following steps:
[0008] Obtain the cave mural data required for initial training, and input the text data in the cave mural data into an initial temporal knowledge graph reasoning model for training;
[0009] Input the text data in the trained cave mural data into a relation encoder and an entity encoder respectively to obtain corresponding relation embeddings and entity embeddings, and input the time information trained by the initial temporal knowledge graph reasoning model into a time vector generator to obtain a corresponding time vector;
[0010] Extract visual features from the image data in the obtained cave mural data;
[0011] Fuse the relation embeddings, entity embeddings, time vectors and visual features to generate a comprehensive feature vector;
[0012] Input the comprehensive feature vector into a decoder to calculate a confidence score; calculate a loss according to the confidence score to further train the temporal knowledge graph reasoning model.
[0013] Further, the text data in the trained cave painting data is input into the relation encoder and the entity encoder respectively to obtain corresponding relation embedding and entity embedding, including:
[0014] The trained cave painting data is processed to extract the relation embedding therefrom;
[0015] The trained cave painting data is processed by using a graph convolution network to obtain an entity embedding matrix, and further obtain the entity embedding.
[0016] Further, the time information of the initial time knowledge graph reasoning model training is input into the time vector generator to obtain the corresponding time vector, and further including:
[0017] The time information of the time knowledge graph reasoning model training is input into the time vector generator to generate a periodic time vector and a non-periodic time vector respectively.
[0018] Further, the image data in the obtained cave painting data is subjected to visual feature extraction, including:
[0019] The trained cave painting image data is processed by using a convolution layer to obtain visual features.
[0020] Further, the relation embedding, the entity embedding, the time vector and the visual features are subjected to feature fusion to generate a comprehensive feature vector, including:
[0021] The decoder performs one-dimensional convolution operation on the concatenated relation embedding, entity embedding, periodic time vector, non-periodic time vector and visual features to generate a comprehensive feature vector.
[0022] Further, the one-dimensional convolution operation is:
[0023]
[0024] In the formula, denotes the comprehensive feature vector, m c denotes the output vector, c denotes the number of convolution kernels, K denotes the kernel width, n∈[0,d] denotes the element in the output vector, w c denotes the learnable kernel parameter,
[0025] and denote the padding version of the entity embedding, the relation embedding, the periodic time vector, the non-periodic time vector and the visual feature respectively, and τ denotes a specific time point.
[0026] Further, the formula for calculating the confidence score p is:
[0027]
[0028] wherein, Ψ represents a one-dimensional convolution operator, h s,t t , respectively represent entity embedding, relation embedding, time vector and visual feature, ves represents a feature mapping operator, M conv represents a convolution operation, W represents a linear transformation matrix, H t represents entity embedding, ReLu and softmax respectively represent two kinds of activation functions.
[0029] Further, the formula for calculating the loss is:
[0030]
[0031] wherein, represents a loss function, G represents a set of (s, r, o, t), represents a label vector of this task, if it occurs, the element is 1, otherwise 0, log represents logarithmic operation, p(o|s, r, t) represents a class to be identified by the current model.
[0032] In a second aspect, the present application further provides a cave mural spatiotemporal dependence law discovery system, which comprises:
[0033] A data acquisition training module is configured to acquire cave mural data required for initial training, and input text data in the cave mural data to an initial time sequence knowledge graph reasoning model for training.
[0034] A data encoding module is configured to input text data in the trained cave mural data to a relation encoder and an entity encoder respectively, acquire corresponding relation embedding and entity embedding, input time information trained by the initial time sequence knowledge graph reasoning model to a time vector generator, and obtain a corresponding time vector.
[0035] A feature extraction module is configured to perform visual feature extraction on image data in the acquired cave mural data.
[0036] A feature fusion module is configured to perform feature fusion on the relation embedding, the entity embedding, the time vector and the visual feature, and generate a comprehensive feature vector.
[0037] A model training module is configured to input the comprehensive feature vector to a decoder, calculate a confidence score, calculate a loss according to the confidence score, and further train the time sequence knowledge graph reasoning model.
[0038] Further, the text data in the trained cave mural data is input to the relation encoder and the entity encoder respectively to acquire the corresponding relation embedding and the entity embedding, which comprises:
[0039] The trained cave painting data is processed to extract the relationship embedding;
[0040] The trained cave painting data is processed by using a graph convolution network to obtain an entity embedding matrix, and further obtain the entity embedding.
[0041] The beneficial effects of the present application are:
[0042] In the present application, the gated recurrent unit is used to learn the relationship embedding when the relationship embedding is learned, the graph convolution network and the gated recurrent unit are used to learn the entity embedding when the entity embedding is learned, and the time information is encoded and the visual features are learned by using the image feature extraction module. Thus, the model can learn the relationship embedding, the entity embedding and the visual features to obtain the spatio-temporal dependency, so that the excellent effect of the model on the reasoning accuracy of the time sequence knowledge graph is achieved. BRIEF DESCRIPTION OF DRAWINGS
[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0044] Figure 1 is one of the flowcharts of a cave painting spatio-temporal dependency rule discovery method according to an embodiment of the present application;
[0045] Figure 2 is a structural block diagram of a cave painting spatio-temporal dependency rule discovery system according to an embodiment of the present application;
[0046] Figure 3 is the second flowchart of a cave painting spatio-temporal dependency rule discovery method according to an embodiment of the present application;
[0047] Figure 4 is a time sequence knowledge graph of a cave painting spatio-temporal dependency rule discovery method according to an embodiment of the present application. DETAILED DESCRIPTION
[0048] To further illustrate the embodiments, the present application provides drawings, which are part of the disclosure of the present application, mainly used to illustrate the embodiments, and can be used to explain the operating principle of the embodiments in conjunction with the related description of the specification. Those skilled in the art should understand other possible embodiments and advantages of the present application by referring to these contents. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0049] According to an embodiment of the present application, a cave painting space-time dependence rule discovery method and system are provided.
[0050] The present application will be further described in conjunction with the drawings and specific embodiments, as shown in Figure 1 The cave painting space-time dependence rule discovery method according to an embodiment of the present application comprises:
[0051] Step S1, obtaining cave painting data required for initial training, and inputting text data in the cave painting data into an initial time sequence knowledge graph reasoning model for training.
[0052] Step S2, inputting the text data in the trained cave painting data into a relationship encoder and an entity encoder respectively, obtaining corresponding relationship embedding and entity embedding, and inputting time information trained by the initial time sequence knowledge graph reasoning model into a time vector generator to obtain a corresponding time vector.
[0053] Step S3, performing visual feature extraction on the image data in the obtained cave painting data.
[0054] Step S4, performing feature fusion on the relationship embedding, the entity embedding, the time vector and the visual feature to generate a comprehensive feature vector.
[0055] Step S5, inputting the comprehensive feature vector into a decoder to calculate a confidence score; calculating a loss according to the confidence score to further train the time sequence knowledge graph reasoning model.
[0056] It needs to be explained that, as shown in Figure 3 The text data in the cave painting data is input into the initial time sequence knowledge graph reasoning model for training; and the text data in the trained cave painting data is input into the relationship encoder and the entity encoder respectively. In the relationship encoder, the relationship embedding is obtained through the gated recurrent unit; and in the entity encoder, the entity embedding is obtained through the graph convolution network and the gated recurrent unit.
[0057] The text information in the cave painting data is input into the initial time sequence knowledge graph reasoning model for training; after the training, the text data is sent into the relationship encoder and the entity encoder. In the relationship encoder, the relationship feature is extracted through the gated recurrent unit to obtain the relationship embedding; and in the entity encoder, the entity data is extracted through the graph convolution network and the gated recurrent unit to obtain the entity embedding.
[0058] The time information trained by the time sequence knowledge graph reasoning model is input into the time vector generator to generate a periodic time vector and a non-periodic time vector respectively.
[0059] The image data in the obtained cave painting data is processed using a convolutional layer to obtain visual features.
[0060] The relationship embedding, entity embedding, time vector and visual feature are fused to generate a comprehensive feature vector.
[0061] The comprehensive feature vector is input into the decoder to calculate a confidence score.
[0062] As shown in Figure 4 , the training data set is 800+ cave painting data collected in the laboratory, including entities, relationships, and timestamp information. In terms of entities, the name of the painting is included. In terms of relationships, the temporal evolution relationship between paintings is recorded. The timestamp information marks the creation period of each painting. A quadruple example: {Tiangxingtu, contains, Jinfengniao, 4th century to 5th century}. By integrating these multi-dimensional information, our data set can comprehensively and detailedly reflect the complex characteristics of the painting and its change rule over time, providing a solid and detailed data foundation for the training of the initial temporal knowledge graph reasoning model.
[0063] In one embodiment, the text data in the trained cave painting data is input into the relationship encoder and the entity encoder respectively to obtain the corresponding relationship embedding and entity embedding, including:
[0064] Step S21, processing the trained cave painting data to extract the relationship embedding therefrom;
[0065] It should be explained that the trained cave painting data is processed using a gated recurrent unit to extract the relationship embedding therefrom;
[0066] The cave painting data at time t is input, which contains relationship data r t and entity data h s,t , h o,t The relationship data r t is processed using a gated recurrent unit to obtain a relationship embedding R t , and the calculation formula is:
[0067] R t = GRU(R t-1 , r t );
[0068] In the formula, GRU represents the gated recurrent unit operation, R t represents the relationship embedding at time t, and R t-1 represents the relationship embedding at time t-1.
[0069] Step S22, the trained cave painting data is processed by using the graph convolution network to obtain the entity embedding matrix, and further obtain the entity embedding.
[0070] It needs to be explained that the cave painting data at the input time t contains relationship data r t and entity data h s,t , h o,t , for relationship data r t and entity data h s,t , h o,t , first use the graph convolution network to process it, obtain The calculation formula of the lth layer in the graph convolution network is:
[0071]
[0072] In the formula, and respectively represent the embedding of entities s and o at the lth layer in the graph convolution network, and represent the learnable weight, c0 represents a normalization factor, which is equal to the in-degree of o, σ represents the ReLU activation function, and ψ represents the one-dimensional convolution operator; The number of layers of the graph convolution network is l=2.
[0073] Then, the entity embedding H t is obtained, and the formula is:
[0074]
[0075] In the formula, GPU represents the gating cycle unit operation, H t represents the entity embedding at time t, H t-1 represents the entity embedding at time t-1, represents the entity embedding matrix after processing by the graph convolution network at time t-1.
[0076] In one embodiment, the time information of the initial time sequence knowledge graph reasoning model training is input to the time vector generator to obtain the corresponding time vector, which further includes:
[0077] Step S23, the time information of the time sequence knowledge graph reasoning model training is input to the time vector generator to generate the periodic time vector and the aperiodic time vector respectively.
[0078] It needs to be explained that the time information is input to the time vector generator to obtain the time vector, and the calculation formula is respectively:
[0079]
[0080] In the formula, and Let w represent the periodic and non-periodic time vectors, respectively. p , w np , Let f represent the learnable parameters, and let f represent a periodic excitation function sin.
[0081] In one embodiment, visual feature extraction from the image data in the acquired cave mural data includes:
[0082] Step S31: Process the trained cave mural image data using convolutional layers to obtain visual features.
[0083] It should be explained that the cave mural image data at input time t is processed using a convolutional layer to obtain visual features X. t The output feature map of the q-th convolutional layer is The calculation formula is as follows:
[0084]
[0085] In the formula, W represents the output feature map of the (q-1)th convolutional layer. q Let σ represent the weight matrix of the q-th convolutional kernel, and b represent the activation function. q This indicates the bias; the number of convolutional network layers is q = 3.
[0086] In one embodiment, feature fusion of relation embedding, entity embedding, temporal vector, and visual features to generate a comprehensive feature vector includes:
[0087] Step S41: The decoder performs a one-dimensional convolution operation on the concatenated relation embedding, entity embedding, periodic time vector, non-periodic time vector, and visual features to generate a comprehensive feature vector.
[0088] It needs to be explained that the decoder embeds the entity h. s,t Relational embedding r t Periodic time vector nonperiodic time vector and visual features X t The concatenation of the vectors is then subjected to a one-dimensional convolution operation to obtain a vector.
[0089] In one embodiment, the one-dimensional convolution operation is as follows:
[0090]
[0091] In the formula, Denotes the comprehensive feature vector, m cdenotes an output vector, c denotes the number of convolution kernels, K denotes the kernel width, n [0, d] denotes an element in the output vector, w c denotes a learnable kernel parameter,
[0092] and denote the padding version of entity embedding, relation embedding, periodic time vector, aperiodic time vector and visual feature respectively, and τ denotes a specific time point.
[0093] It needs to be explained that the embedding dimension d = 200, the number of convolution kernels c = 50, and the kernel width K = 12.
[0094] In one embodiment, the formula for calculating the confidence score p is:
[0095]
[0096] In the formula, Ψ denotes a one-dimensional convolution operator, h s,t ,r t , denote entity embedding, relation embedding, time vector and visual feature respectively, ves denotes a feature mapping operator, M conv denotes a convolution operation, W denotes a linear transformation matrix, H t denotes entity embedding, ReLu and softmax denote two kinds of activation functions respectively.
[0097] In one embodiment, the formula for calculating the loss is:
[0098]
[0099] In the formula, denotes a loss function, G denotes a set of (s, r, o, t), denotes a label vector of this task, the element is 1 if it occurs, otherwise 0, log denotes logarithmic operation, and p (o | s, r, t) denotes the class to be identified by the current model.
[0100] As Figure 2 shown, according to another embodiment of the present application, the present application also provides a cave mural spatiotemporal dependence law discovery method system, which comprises:
[0101] The data acquisition and training module 201 is used to acquire cave mural data required for initial training, and input text data in the cave mural data to an initial time sequence knowledge graph reasoning model for training.
[0102] The data coding module 202 is configured to input text data in the trained cave mural data into a relation encoder and an entity encoder respectively, to obtain corresponding relation embedding and entity embedding, and to input time information of an initial temporal knowledge graph reasoning model into a time vector generator to obtain a corresponding time vector.
[0103] The feature extraction module 203 is configured to perform visual feature extraction on the obtained image data in the cave mural data.
[0104] The feature fusion module 204 is configured to perform feature fusion on the relation embedding, the entity embedding, the time vector and the visual feature to generate a comprehensive feature vector.
[0105] The model training module 205 is configured to input the comprehensive feature vector into a decoder to calculate a confidence score, to calculate a loss according to the confidence score, and to further train the temporal knowledge graph reasoning model.
[0106] In one embodiment, the inputting of the text data in the trained cave mural data into the relation encoder and the entity encoder to obtain the corresponding relation embedding and the entity embedding comprises:
[0107] The trained cave mural data is processed to extract the relation embedding therefrom.
[0108] The trained cave mural data is processed by using a graph convolution network to obtain an entity embedding matrix, and the entity embedding is further obtained.
[0109] In summary, by means of the above technical solutions of the present application, the gated recurrent unit is used to learn the relation embedding in the relation embedding, the graph convolution network and the gated recurrent unit are used to learn the entity embedding in the entity embedding, and the time information is encoded and the image feature extraction module is used to learn the visual feature. Thus, the model can learn the relation embedding, the entity embedding and the visual feature to obtain the spatio-temporal dependency, thereby achieving the excellent effect of the reasoning accuracy of the model on the temporal knowledge graph.
[0110] The above only describes the preferred embodiments of the present application and should not be used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A method for discovering spatiotemporal dependence rules of cave paintings, characterized in that, The method comprises the following steps: S1, obtaining cave painting data required for initial training, and inputting text data in the cave painting data into an initial time sequence knowledge graph reasoning model for training; S2, inputting the text data in the trained cave painting data into a relationship encoder and an entity encoder respectively to obtain corresponding relationship embedding and entity embedding, and inputting time information trained by the initial time sequence knowledge graph reasoning model into a time vector generator to obtain a corresponding time vector; S3, performing visual feature extraction on the image data obtained from the cave painting data; S4, performing feature fusion on the relationship embedding, entity embedding, time vector and visual feature to generate a comprehensive feature vector; S5, inputting the comprehensive feature vector into a decoder to calculate a confidence score; calculating a loss according to the confidence score to further train the time sequence knowledge graph reasoning model.
2. The method according to claim 1, wherein, The step of inputting the text data in the trained cave painting data into the relationship encoder and the entity encoder to obtain the corresponding relationship embedding and entity embedding comprises the following steps: processing the trained cave painting data to extract relationship embedding therefrom; processing the trained cave painting data by using a graph convolution network to obtain an entity embedding matrix and further obtain entity embedding.
3. The method of claim 1, wherein the method is characterized by, The step of inputting the time information trained by the initial time sequence knowledge graph reasoning model into the time vector generator to obtain the corresponding time vector further comprises the following steps: inputting the time information trained by the time sequence knowledge graph reasoning model into the time vector generator to generate a periodic time vector and a non-periodic time vector respectively.
4. The method of claim 1, wherein the method is characterized by, The step of performing visual feature extraction on the image data obtained from the cave painting data comprises the following step: processing the trained cave painting image data by using a convolution layer to obtain visual features.
5. The method of claim 4, wherein the method further comprises: determining a time period of the cave painting; and determining a space period of the cave painting. The step of performing feature fusion on the relationship embedding, entity embedding, time vector and visual feature to generate a comprehensive feature vector comprises the following step: S41, performing one-dimensional convolution operation on the concatenated relationship embedding, entity embedding, periodic time vector, non-periodic time vector and visual feature by using a decoder to generate a comprehensive feature vector.
6. The method of claim 5, wherein the method is characterized by, The one-dimensional convolution formula is as follows: wherein denotes the comprehensive feature vector, m c denotes the output vector, c denotes the number of convolution kernels, K denotes the kernel width, n ∈ [0, d] denotes the element in the output vector, w c denotes the learnable kernel parameter, and denote the entity embedding, the relation embedding, the periodic time vector, the aperiodic time vector and the padded version of the visual feature, respectively, τ denotes a specific time point.
7. The method of claim 1, wherein the method is characterized by, The formula for calculating the confidence score p is as follows: wherein Ψ denotes a one-dimensional convolution operator, h s,t , t , denote entity embedding, relation embedding, time vector and visual feature respectively, ves denotes a feature mapping operator, M conv denotes a convolution operation, W denotes a linear transformation matrix, H t denotes an entity embedding, ReLu and softmax denote activation functions respectively.
8. The method according to claim 7, wherein the method is characterized by, The formula for calculating the loss is as follows: wherein represents a loss function, G represents a set of (s, r, o, t), represents a label vector of the task, the element is 1 if it occurs, otherwise 0, log represents a logarithmic operation, and p(o|s, r, t) represents a class to be recognized by the current model.
9. A method system for discovering time-space dependent rules of cave paintings, for implementing the method for discovering time-space dependent rules of cave paintings according to any one of claims 1-8, characterized in that, The method comprises the following steps: a data acquisition and training module for obtaining cave painting data required for initial training, and inputting text data in the cave painting data into an initial time sequence knowledge graph reasoning model for training; a data encoding module for inputting the text data in the trained cave painting data into a relationship encoder and an entity encoder respectively to obtain corresponding relationship embedding and entity embedding, and inputting time information trained by the initial time sequence knowledge graph reasoning model into a time vector generator to obtain a corresponding time vector; a feature extraction module for performing visual feature extraction on the image data obtained from the cave painting data; a feature fusion module for performing feature fusion on the relationship embedding, entity embedding, time vector and visual feature to generate a comprehensive feature vector; a model training module for inputting the comprehensive feature vector into a decoder to calculate a confidence score; calculating a loss according to the confidence score to further train the time sequence knowledge graph reasoning model. 10.The system of claim 9, wherein, The text data in the trained cave mural data is respectively input into the relation encoder and the entity encoder, and the corresponding relation embedding and entity embedding are obtained, including: The trained cave mural data is processed to extract the relation embedding therein; The trained cave mural data is processed by using a graph convolution network to obtain an entity embedding matrix, and further obtain the entity embedding.