Target action prediction method based on time sequence knowledge graph

Abstract

The invention provides a time sequence knowledge graph-based target action prediction method, which comprises the following steps of: acquiring multi-source data which comprises a plurality of entities; based on a relationship classification model, obtaining an association relationship between the entities, and completing redundancy check and alignment de-duplication of the association relationship, the relationship classification model being obtained by classification learning; after target entities in the multiple entities are determined according to the positions and the feature information of the entities, an association tree of association relationships among the target entities is dynamically constructed to complete construction of the time sequence knowledge graph; based on the time sequence knowledge graph, modeling and classifying the time sequence nonlinear incidence relation between the target entities by using a deep neural network to generate a target space evolution model; and predicting a behavior pattern and a motion track of the target entity based on the target spatial evolution model.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph application technology, and in particular to a target action prediction method based on a time series knowledge graph. Background Art

[0002] With the continuous advancement of artificial intelligence and big data technologies, knowledge graphs have entered a period of rapid development. Knowledge graphs have been widely used in search engines, intelligent question-answering, recommendation systems, and other fields. Knowledge graphs not only provide more accurate and rich search results but also support applications in natural language understanding and machine learning. However, a significant drawback of static knowledge graphs is that they ignore the temporal information of facts and fail to account for the fact that some facts may change over time.

[0003] In the current field of intelligent surveillance and target recognition, automatic analysis of target associations and patterns based on temporal knowledge graphs is becoming a research hotspot. With the advancement of sensor technology and the advent of the big data era, effectively extracting useful information from massive amounts of data and accurately analyzing and predicting target behavior have become key issues that need to be addressed. However, practical applications of this technology still face two challenges.

[0004] First, the problem of misassociation between target entities seriously affects the reliability of analysis results. Existing target association algorithms are mostly based on target location information and make judgments based on location similarity. While this approach can effectively identify targets in certain situations, it still has significant limitations. For example, the nearest neighbor (NN) algorithm relies on similarity evaluation criteria and is easily affected by noise or interference, leading to misassociation. Furthermore, while multi-hypothesis tracking algorithms (MHT) and fuzzy logic-based methods have shown some improvement in handling complex scenes, their adaptability in highly dynamic environments still needs to be improved.

[0005] Secondly, the accuracy of analyzing and predicting typical target behavior patterns is generally low. Traditional models often rely on static models to analyze target trajectories and states, making it difficult to accurately predict the behavior of target entities in rapidly changing dynamic environments. If the model is not updated in a timely manner to reflect these dynamic changes, the predicted results will often deviate significantly from the actual situation. Summary of the Invention

[0006] The present invention provides a target action prediction method based on a time series knowledge graph, comprising: acquiring multi-source data, the multi-source data including multiple entities; obtaining the association relationship between entities based on a relationship classification model, and completing redundancy checking and alignment deduplication of the association relationship, wherein the relationship classification model is obtained by classification learning; after determining the target entity among multiple entities based on the location and feature information of the entity, dynamically constructing an association tree of the association relationship between the target entities to complete the construction of the time series knowledge graph; based on the time series knowledge graph, modeling and classifying the time series nonlinear association relationship between target entities by using a deep neural network to generate a target space evolution model; based on the target space evolution model, predicting the behavior pattern and movement trajectory of the target entity.

[0007] In the above scheme, based on the temporal knowledge graph, the temporal nonlinear association relationship between target entities is modeled and classified by using a deep neural network to generate a target space evolution model, including: extracting the spatiotemporal attributes of the target entities from real-time information and forming a temporal nonlinear association relationship between the target entities; mapping the temporal nonlinear association relationship between the target entities into a spatial high-dimensional projection vector through a recursive neural network; and establishing a target space evolution model based on the spatial high-dimensional projection vector and the random process correlation model.

[0008] In the above scheme, based on the target space evolution model, the behavior pattern and movement trajectory of the target entity are predicted, including: based on the target space evolution model, determining the predicted state of the target entity on the evolution path, and forming decision information of the target entity.

[0009] In the above scheme, based on the target space evolution model, the behavior pattern and movement trajectory of the target entity are predicted, which also includes: obtaining historical data to form historical action law cases of the target entity; based on the predicted state of the target entity on the evolution path, the current state of the target entity is determined by comparing the historical action law cases of the target entity.

[0010] In the above scheme, the behavior pattern and movement trajectory of the target entity are predicted based on the target space evolution model, which also includes: based on the spatiotemporal attributes, evolutionary state and historical action law cases of the target entity, the behavior pattern and movement trajectory of the target entity are determined by dynamically sampling the historical trajectory data of the target entity and performing nonlinear function fitting.

[0011] In the above scheme, the relationship classification model is obtained by classification learning, and the classification learning process includes: obtaining a relationship type training set; constructing a decision tree model, and learning the decision tree model based on the relationship type training set to obtain a relationship classification model.

[0012] In the above scheme, the redundancy check and alignment deduplication of association relationships include: redundancy check and alignment deduplication of association relationships through a semi-supervised learning entity alignment algorithm and an unsupervised scoring and ranking relationship disambiguation method.

[0013] In the above scheme, the target entity among multiple entities is determined based on the location and feature information of the entity, including: using the location correlation of the entity and the calculation of the feature information membership to analyze the association relationship between multiple entities to identify the target entity among multiple entities.

[0014] In the above scheme, an association tree of association relationships between target entities is dynamically constructed, including: generating a time association dataset and a spatial association dataset of target entities based on time matching and spatial association measurement; performing track matching between target entities based on the DTW algorithm to generate an association relationship set between target entities; matching and searching for target entities with target association relationships in the association relationship set based on the spanning tree algorithm, determining the target association relationships between multiple target entities, and generating an association tree.

[0015] In the above scheme, the multi-source data includes the category and model of the entity, the time of discovery and disappearance, the coordinates of the track coverage area, the direction, altitude, distance and speed.

[0016] The technical solution of the embodiment of the present invention has at least the following beneficial effects:

[0017] (1) By dynamically constructing an association tree between target entities, this method can more accurately capture the temporal nonlinear associations between entities and resolve erroneous associations between target entities, thereby significantly improving the accuracy of predicting the behavior patterns and motion trajectories of target entities.

[0018] (2) Based on the constructed temporal knowledge graph data, this method uses the associations between target entities and the associations between target entities and the environment to complete the analysis of the behavioral patterns of typical target entities, thereby improving the accuracy of behavior prediction.

[0019] (3) This method monitors and predicts the behavior of the target entity in real time, can respond to emergencies or changes in a timely manner, and improve the adaptability and stability to dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for predicting target actions based on a time series knowledge graph according to an embodiment of the present invention is schematically shown;

[0021] Figure 2 Schematically shows a flow chart of obtaining a relationship classification model through classification learning according to an embodiment of the present invention;

[0022] Figure 3 Schematically shows a flow chart of dynamically building a relationship tree according to an embodiment of the present invention;

[0023] Figure 4 The flowchart of establishing the target space evolution model according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0025] Figure 1 The flowchart of the target action prediction method based on the time series knowledge graph according to an embodiment of the present invention is schematically shown.

[0026] In the embodiment of the present invention, the target action prediction method based on the time series knowledge graph is divided into two parts for explanation: the construction of the time series knowledge graph and the prediction and analysis of the spatiotemporal evolution of the target entity. First, the construction of the time series knowledge graph is explained in detail.

[0027] Please refer to Figure 1 The specific process of the target action prediction method based on the time series knowledge graph in an embodiment of the present invention includes operations S110~S150.

[0028] In operation S110 , multi-source data is acquired, where the multi-source data includes a plurality of entities.

[0029] For example, the receiving device can acquire data related to multiple moving entities, such as aircraft or ships. This data can include the type and model of the entities, the time of discovery and disappearance, the coordinates of the track coverage area, the position, altitude, distance, and speed, and other related data.

[0030] In operation S120 , based on the relationship classification model, the association relationship between entities is obtained, and redundancy check and alignment deduplication of the association relationship are completed, wherein the relationship classification model is obtained by classification learning.

[0031] For example, the acquired multi-source data is input into an established relationship classification model to obtain the association relationship between entities. The relationship classification model is obtained by classification learning of a decision tree model, which is described in detail below.

[0032] Figure 2 The flowchart of obtaining a relationship classification model through classification learning according to an embodiment of the present invention is schematically shown.

[0033] Please refer to Figure 2In an embodiment of the present invention, the relational classification model is obtained through classification learning, and the classification learning process specifically includes operations S210 to S220.

[0034] In operation S210 , a relationship category training set is obtained.

[0035] In operation S220 , a decision tree model is constructed, and the decision tree model is learned based on the relationship category training set to obtain a relationship classification model.

[0036] For example, after acquiring multi-source data, a training set for machine classification learning is generated based on designed relationship type data collection. After obtaining the training set, the collected data is subjected to correlation feature extraction. The correlation features can be numerical, categorical, or textual. After extraction, they need to be standardized and normalized to facilitate subsequent model classification learning.

[0037] Furthermore, a decision tree model is constructed, and the decision tree model is learned based on the relationship type training set to obtain a relationship classification model.

[0038] Specifically, in an embodiment of the present invention, classification learning is performed using the XGBoost implementation of the decision forest algorithm, which is most effective for classifying discrete data. The XGBoost implementation is an ensemble learning algorithm whose core is to construct multiple decision tree models and optimize them in the direction with the largest gradient descent of the loss function of the preceding decision tree models, thereby achieving rapid convergence and improved classification and generalization performance.

[0039] Furthermore, after obtaining the association relationships between these entities, these association relationships are checked for redundancy and aligned for deduplication.

[0040] In an embodiment of the present invention, the redundancy check and alignment deduplication of association relationships include: redundancy check and alignment deduplication of association relationships through a semi-supervised learning entity alignment algorithm and an unsupervised scoring and ranking relationship disambiguation method.

[0041] Specifically, after constructing relationships between entities, a large number of duplicate edges often appear, which can interfere with data analysis. Therefore, it is necessary to check for redundancy in these relationships and merge them. Relationship redundancy checking determines whether there are duplicate relationships between two entities. First, a semi-supervised learning-based entity alignment algorithm is used to solve the entity alignment problem in the graph. Then, an unsupervised relationship disambiguation method based on scoring and ranking is used to align and remove duplicate relationships between entities.

[0042] Based on the above, after completing the alignment and deduplication of the association relationships, the target entity among multiple entities is determined based on the entity's location and feature information. The target entity here is the entity that will be used to monitor and predict the action trajectory.

[0043] In an embodiment of the present invention, a target entity among multiple entities is determined based on the location and feature information of the entity, including: using the location correlation of the entity and the calculation of the feature information membership to analyze the association relationship between the multiple entities to identify the target entity among the multiple entities.

[0044] Specifically, target association based on the electronic characteristic information of the radiation source is adopted, and the potential relationship between the radiation source information and the target is analyzed by using the method of position correlation and calculation of characteristic information membership to perform target identification. The results obtained can be directly used in subsequent comprehensive processing.

[0045] In operation S130, after determining the target entity among multiple entities based on the location and feature information of the entity, an association tree of the association relationships between the target entities is dynamically constructed to complete the construction of the temporal knowledge graph.

[0046] Figure 3 The flowchart of dynamically constructing a relationship tree according to an embodiment of the present invention is schematically shown.

[0047] Please refer to Figure 3 In an embodiment of the present invention, dynamically constructing an association tree of association relationships between target entities includes operations S310 to S330.

[0048] In operation S310 , a time correlation dataset and a spatial correlation dataset of the target entity are generated based on the time matching and spatial correlation metrics.

[0049] In operation S320 , track matching between target entities is performed based on the DTW algorithm to generate a set of association relationships between the target entities.

[0050] In operation S330 , a target entity having a target association relationship is matched and searched in the association relationship set based on a spanning tree algorithm, target association relationships between multiple target entities are determined, and an association tree is generated.

[0051] In an embodiment of the present invention, after the target entity is determined, the relevant data of the target entity also includes the category and model of the target entity, the time of discovery and disappearance, the coordinates of the track coverage area, the position, altitude, distance and speed.

[0052] For example, taking a certain type of target entity as an example, the data of the target entity is sorted to obtain a track data set S={N1, N2, ⋯, N m}, N(C, M, t s, t f , t, X) represents the track information of a single target, where C represents the target category, M represents the target model, and t s Indicates the initial target discovery time, t f represents the target disappearance time, t=t f -t s is the target duration, X{(x1, y1), ⋯, (x n ,y n )} is the coordinate data set of all points of a single track.

[0053] Furthermore, the data of the target entity is preprocessed by adopting data cleaning methods such as given spatial range, limited movement duration, deleting duplicate data, and correcting error track points to obtain a data set sorted by the target departure time. Secondly, a multi-target activity time correlation matching degree measurement is designed, and the time correlation data set is screened and generated as follows:

[0054]

[0055] Where: p i ' is the target set that has a time association with the i-th target entity, that is

[0056] .

[0057] Furthermore, on this basis, we design a spatial correlation matching metric and filter the generated spatial correlation dataset as follows:

[0058]

[0059] Where: p i ″ is the target set that has spatial association relationship with the i-th target entity, that is,

[0060] .

[0061] Finally, the DTW algorithm is used to match the obtained spatial correlation data sets in pairs to obtain the track overlap between the two target entities, that is, the correlation relationship target set is

[0062]

[0063] Where: is the target set associated with target i, that is,

[0064] .

[0065] Finally, the spanning tree method is used to find multiple target entities with related relationships, generate an association tree, extract time-related information from the data, integrate time and relationships, and form a temporal knowledge graph of the target entity.

[0066] It can be understood that the above-mentioned tree generation process is also a multiple screening process. Through time correlation screening, a time correlation data set is generated; the time correlation data set is screened for spatial correlation to generate a spatial correlation data set; the spatial correlation data set is screened for track overlap using the DTW algorithm to generate a correlation relationship data set, and finally, the same branch target is found according to the tree generation method to generate a correlation tree.

[0067] In an embodiment of the present invention, a temporal knowledge graph of target entities is used to mine the relationships between multiple target entities, helping to identify target scale. By using massive amounts of target entity information, including typical target-related attribute information, feature information, historical behavior data, and temporal graph knowledge data, the knowledge graph can be organized, summarized, analyzed, and mined to identify relationships between typical target elements and between targets, enabling target state prediction and action-assisted decision-making analysis.

[0068] Through the embodiments of the present invention, first, an association relationship classification model is obtained through automatic classification learning of association relationships based on decision forests, then the association relationship redundancy check technology is used to eliminate duplicate edges, and finally, a temporal knowledge graph is established using the association tree generated by multiple screening, thereby solving the problem of incorrect associations between target entities.

[0069] Therefore, based on the temporal knowledge graph that has been established above, the spatiotemporal correlation and evolution prediction analysis of the target entity is explained in detail below.

[0070] In operation S140 , based on the temporal knowledge graph, a target space evolution model is generated by modeling and classifying the temporal nonlinear association relationship between target entities using a deep neural network.

[0071] Figure 4 The flowchart of establishing the target space evolution model according to an embodiment of the present invention is schematically shown.

[0072] Please refer to Figure 4 In an embodiment of the present invention, based on the temporal knowledge graph, the temporal nonlinear association relationship between target entities is modeled and classified by using a deep neural network to generate a target space evolution model, including operations S410~S430.

[0073] In operation S410 , spatiotemporal attributes of target entities are extracted from real-time information, and temporal nonlinear association relationships between target entities are formed.

[0074] In operation S420, the temporal nonlinear association relationship between target entities is mapped into a spatial high-dimensional projection vector through a recursive neural network.

[0075] In an embodiment of the present invention, according to the rules set in advance or cluster analysis, the dynamic attributes of the target entity related to the time series input are extracted, and then the spatiotemporal attribute data of the target entity are extracted from the real-time information in combination with the clustering or deep neural network feature space mapping method, and then a new temporal nonlinear association relationship between the target entities is formed by a shallow or deep classifier. At the same time, based on the latest temporal association relationship between each target entity and its adjacent associated target entity, the adjacent association relationship with a non-fixed number and non-fixed length is mapped into a deep feature vector with a fixed length through a recursive neural network in the form of edge length input sentence classification, and thus serves as a high-dimensional projection vector of the target entity in the state space.

[0076] In operation S430 , a target space evolution model is established based on the spatial high-dimensional projection vector and the random process correlation model.

[0077] In an embodiment of the present invention, for each type of target entity, the dynamic evolution process of the target entity is descriptively modeled using a random process related model based on the previously acquired information and the high-dimensional projection vector of the target entity in the state space, and is trained using a pre-collected training set to form a target space evolution model to estimate the possible future state of the target entity.

[0078] In operation S150 , a behavior pattern and a movement trajectory of the target entity are predicted based on the target space evolution model.

[0079] In an embodiment of the present invention, based on the target space evolution model, the behavior pattern and movement trajectory of the target entity are predicted, including: based on the target space evolution model, determining the predicted state of the target entity on the evolution path, and forming decision information of the target entity.

[0080] Specifically, based on the target space evolution model, the evolution law of each target entity can be obtained, so that the state of the target entity at future moments can be predicted and analyzed from the various possible states of the target entity on the evolution path, and the final decision information with high confidence can be formed.

[0081] Furthermore, in an embodiment of the present invention, predicting the behavior pattern and movement trajectory of the target entity based on the target space evolution model also includes: obtaining historical data to form historical action law cases of the target entity; based on the predicted state of the target entity on the evolution path, determining the current state of the target entity by comparing the historical action law cases of the target entity.

[0082] Specifically, historical data is obtained from the historical case library, and the historical data includes historical cases. Each historical case includes a problem and the response of the target entity to solve the problem. It can be understood that the problem in each historical case refers to the occurrence of a historical case at a certain moment, as well as the response or action taken by the target entity related to the problem to be solved. In addition, it also includes the response or action taken by other target entities to the event. In this way, a historical action law case of the target entity is formed through historical data. Further, based on the predicted state of the target entity obtained above, the current state of the target entity is determined by comparing it with the historical action law case of the target entity.

[0083] Furthermore, in an embodiment of the present invention, predicting the behavior pattern and motion trajectory of the target entity based on the target space evolution model also includes: based on the spatiotemporal attributes, evolutionary state and historical action law cases of the target entity, dynamically sampling the historical trajectory data of the target entity and performing nonlinear function fitting to determine the behavior pattern and motion trajectory of the target entity.

[0084] Specifically, based on the aforementioned examples of the target entity's spatiotemporal attributes, evolutionary state, and historical behavioral patterns, a dynamic sampling method is used to obtain the target entity's historical trajectory data sequence. A complex nonlinear function, including polynomial and transcendental functions, is then fitted to the target trajectory time series data sequence to determine the current target entity's behavior pattern and motion trajectory. In this embodiment of the present invention, the function fitting method is highly targeted, interpretable, adaptable, and generalizable, enabling accurate prediction of the target entity's behavior pattern and motion trajectory.

[0085] Through the embodiments of the present invention, based on the constructed time series knowledge graph data, the associations between target entities and the associations between target entities and the environment recorded therein are utilized to complete the analysis of information such as spatiotemporal distribution rules, spatiotemporal association rules, spatiotemporal clustering rules, and spatiotemporal feature rules, thereby effectively improving the analysis and prediction accuracy of the behavior pattern rules of the target entities.

[0086] The above specific embodiments further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A target action prediction method based on temporal knowledge graph, characterized in that: The method comprises: Acquire multi-source data, where the multi-source data includes a plurality of entities; Based on a relational classification model, the association relationships between the entities are obtained, and redundancy checking and alignment deduplication of the association relationships are completed, wherein the relational classification model is obtained by classification learning; After determining the target entity among the multiple entities based on the location and feature information of the entity, dynamically constructing an association tree of association relationships between the target entities to complete the construction of the temporal knowledge graph; Based on the temporal knowledge graph, a target space evolution model is generated by modeling and classifying the temporal nonlinear association relationship between the target entities using a deep neural network; Based on the target space evolution model, the behavior pattern and movement trajectory of the target entity are predicted.

2. The target action prediction method based on time series knowledge graph according to claim 1 is characterized in that: The method of generating a target space evolution model by modeling and classifying the temporal nonlinear association relationship between the target entities based on the temporal knowledge graph using a deep neural network includes: Extracting the spatiotemporal attributes of the target entities from the real-time information and forming a temporal nonlinear association relationship between the target entities; Mapping the temporal nonlinear association relationship between the target entities into a spatial high-dimensional projection vector through a recursive neural network; Based on the spatial high-dimensional projection vector and the random process correlation model, a target space evolution model is established.

3. The target action prediction method based on time series knowledge graph according to claim 1 is characterized in that: The predicting of the behavior pattern and movement trajectory of the target entity based on the target space evolution model includes: Based on the target space evolution model, the predicted state of the target entity on the evolution path is determined, and decision information of the target entity is formed.

4. The target action prediction method based on time series knowledge graph according to claim 1 or 3 is characterized in that: The predicting of the behavior pattern and movement trajectory of the target entity based on the target space evolution model further includes: Obtain historical data to form cases of historical behavior patterns of target entities; Based on the predicted state of the target entity on the evolution path, the current state of the target entity is determined by comparing the historical action regularity cases of the target entity.

5. The target action prediction method based on time series knowledge graph according to claim 4 is characterized in that: The predicting of the behavior pattern and movement trajectory of the target entity based on the target space evolution model further includes: Based on the spatiotemporal attributes, evolutionary state and historical action law cases of the target entity, the behavior pattern and movement trajectory of the target entity are determined by dynamically sampling the historical trajectory data of the target entity and performing nonlinear function fitting.

6. The target action prediction method based on time series knowledge graph according to claim 1 is characterized in that: The relationship classification model is obtained by classification learning, and the classification learning process includes: Obtain a training set of relationship categories; A decision tree model is constructed, and the decision tree model is learned based on the relationship type training set to obtain the relationship classification model.

7. The target action prediction method based on time series knowledge graph according to claim 1 is characterized in that: The redundancy check and alignment deduplication of the association relationship includes: The redundancy of the association relationships is checked and aligned to remove duplicates through a semi-supervised learning entity alignment algorithm and an unsupervised scoring and ranking relationship disambiguation method.

8. The target action prediction method based on time series knowledge graph according to claim 1 is characterized in that: The determining of a target entity among the multiple entities based on the location and feature information of the entity includes: The relationship between the multiple entities is analyzed by using the location association of the entities and the calculation of the membership degree of the feature information, so as to identify the target entity among the multiple entities.

9. The target action prediction method based on time series knowledge graph according to claim 1 or 8, characterized in that: The dynamically constructing an association tree of association relationships between the target entities includes: Based on time matching and spatial correlation metrics, a time correlation dataset and a spatial correlation dataset of the target entity are generated; Perform track matching between the target entities based on the DTW algorithm to generate a set of association relationships between the target entities; Based on the spanning tree algorithm matching, a target entity with a target association relationship is searched in the association relationship set, the target association relationships between the multiple target entities are determined, and an association tree is generated.

10. The target action prediction method based on time series knowledge graph according to claim 1 is characterized in that: The multi-source data includes the type and model of the entity, the time of discovery and disappearance, the coordinates of the track coverage area, the position, altitude, distance and speed.

Images