A target action prediction method based on a time sequence knowledge graph

Abstract

The application provides a target action prediction method based on a time sequence knowledge graph, comprising: acquiring multi-source data, the multi-source data comprising a plurality of entities; obtaining the correlation between the entities based on a relationship classification model, and completing the redundancy check and alignment deduplication of the correlation, wherein the relationship classification model is obtained through classification learning; after determining a target entity in the plurality of entities according to the position and characteristic information of the entity, dynamically constructing a correlation tree of the correlation between the target entities to complete the construction of the time sequence knowledge graph; based on the time sequence knowledge graph, modeling and classifying the time sequence nonlinear correlation between the target entities by using a deep neural network to generate a target space evolution model; and predicting the behavior mode and motion trajectory of the target entity based on the target space evolution model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph application, and particularly relates to a target action prediction method based on a time sequence knowledge graph. BACKGROUND

[0002] With the continuous development of artificial intelligence and big data technology, knowledge graph has ushered in a period of rapid development. Knowledge graph has been widely used in search engines, intelligent question answering, recommendation systems and other fields. Knowledge graph not only provides more accurate and rich search results, but also supports natural language understanding, machine learning and other applications. However, the static knowledge graph has a relatively obvious shortcoming, which is that it ignores the time sequence information of facts and does not consider that some facts will change with time.

[0003] In the current field of intelligent monitoring and target recognition, target association and rule automatic analysis technology based on time sequence knowledge graph is gradually becoming a research hotspot. With the development of sensor technology and the arrival of the big data era, how to effectively extract useful information from massive data and realize accurate analysis and prediction of target behavior has become a key problem to be solved. However, this technology still faces two challenges in practical application.

[0004] Firstly, the error association problem between target entities seriously affects the reliability of the analysis results. The existing target association algorithms are mostly based on target location information and determine by location similarity. Although this method can effectively identify targets in some cases, it still has great limitations. For example, the nearest neighbor algorithm (NN) relies on similarity evaluation criteria and is easily affected by noise or interference, resulting in false association. In addition, although the multiple hypothesis tracking algorithm (MHT) and the method based on fuzzy logic have improved in handling complex scenes, their adaptability in high dynamic environments still needs to be improved.

[0005] Secondly, the accuracy of typical target behavior mode rule analysis and prediction is generally low. In traditional models, the analysis of target trajectory and state usually relies on static models, which is difficult to meet the high-precision prediction and analysis of target entities in a rapidly changing dynamic environment, because if the model fails to update in time to reflect these dynamic changes, the predicted results will often deviate greatly from the actual situation. SUMMARY

[0006] The application provides a target action prediction method based on a time sequence knowledge graph, comprising: acquiring multi-source data, the multi-source data comprising a plurality of entities; obtaining the correlation between the entities based on a relationship classification model, and completing the redundancy check and alignment of the correlation, wherein the relationship classification model is obtained through classification learning; after determining a target entity in the plurality of entities according to the position and feature information of the entity, dynamically constructing a correlation tree of the correlation between the target entities to complete the construction of the time sequence knowledge graph; based on the time sequence knowledge graph, modeling and classifying the time sequence nonlinear correlation between the target entities by using a deep neural network to generate a target space evolution model; and predicting the behavior mode and motion trajectory of the target entity based on the target space evolution model.

[0007] In the above scheme, based on the time sequence knowledge graph, the target space evolution model is generated by using the deep neural network to model and classify the time sequence nonlinear correlation between the target entities, comprising: extracting the spatiotemporal attributes of the target entities from real-time information, and forming the time sequence nonlinear correlation between the target entities; mapping the time sequence nonlinear correlation between the target entities to a spatial high-dimensional projection vector through a recurrent neural network; and establishing the target space evolution model based on the spatial high-dimensional projection vector and a random process related model.

[0008] In the above scheme, based on the target space evolution model, the behavior mode and motion trajectory of the target entity are predicted, comprising: determining the predicted state of the target entity on the evolution path based on the target space evolution model, and forming the decision information of the target entity.

[0009] In the above scheme, based on the target space evolution model, the behavior mode and motion trajectory of the target entity are predicted, further comprising: acquiring historical data to form the historical action rule case of the target entity; and determining the current state of the target entity by comparing the historical action rule case of the target entity based on the predicted state of the target entity on the evolution path.

[0010] In the above scheme, based on the target space evolution model, the behavior mode and motion trajectory of the target entity are predicted, further comprising: based on the spatiotemporal attributes, evolution state and historical action rule case of the target entity, determining the behavior mode and motion trajectory of the target entity by dynamically sampling the historical trajectory data of the target entity and performing nonlinear function fitting.

[0011] In the above scheme, the relationship classification model is obtained through classification learning, and the classification learning process comprises: acquiring a relationship category training set; constructing a decision tree model, and learning the decision tree model based on the relationship category training set to obtain the relationship classification model.

[0012] In the scheme, the redundancy checking and alignment deduplication of the association relationship include: redundancy checking and alignment deduplication of the association relationship through an entity alignment algorithm based on semi-supervised learning and a scoring and sorting relationship disambiguation method based on unsupervised learning.

[0013] In the scheme, the target entity among the multiple entities is determined according to the position and feature information of the entity, and the association relationship among the multiple entities is analyzed to identify the target entity among the multiple entities by using the position correlation and feature information membership degree of the entity.

[0014] In the scheme, the association tree of the association relationship among the target entities is dynamically constructed, and the association relationship set among the target entities is generated based on the DTW algorithm, and the target entity with the target association relationship is matched and searched in the association relationship set based on the spanning tree algorithm, the target association relationship among the multiple target entities is determined, and the association tree is generated.

[0015] In the scheme, the multi-source data includes the category, model, discovery and disappearance time of the entity, and the coordinates, direction, height, distance and speed of the track coverage area.

[0016] The technical scheme of the embodiment of the application has at least the following beneficial effects:

[0017] (1) The method can more accurately capture the time sequence nonlinear association relationship between entities by dynamically constructing the association relationship tree among the target entities, solve the false association between the target entities, and thus significantly improve the prediction accuracy of the behavior pattern and motion trajectory of the target entity.

[0018] (2) The method uses the association between the target entities and the association between the target entity and the environment to complete the behavior pattern rule analysis of the typical target entity based on the constructed time sequence knowledge graph data, thereby improving the accuracy of behavior prediction.

[0019] (3) The method can timely respond to emergencies or changes by monitoring and predicting the behavior of the target entity in real time, thereby improving the adaptability and stability to the dynamic environment. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of a target action prediction method based on a time sequence knowledge graph according to an embodiment of the application is schematically shown;

[0021] Figure 2 A flowchart of obtaining a relationship classification model through classification learning according to an embodiment of the application is schematically shown;

[0022] Figure 3 Fig. 1 schematically shows a flow chart of dynamically constructing a relationship tree according to an embodiment of the present application;

[0023] Figure 4 Fig. 4 schematically shows a flow chart of establishing a target space evolution model according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the embodiments and the accompanying drawings.

[0025] Figure 1 Fig. 6 schematically shows a flow chart of a target action prediction method based on a time-series knowledge graph according to an embodiment of the present application.

[0026] In the embodiment of the present application, the target action prediction method based on the time-series knowledge graph is divided into two parts, construction of the time-series knowledge graph and prediction analysis of the target entity space-time evolution, for description. First, the construction of the time-series knowledge graph is described in detail.

[0027] For details, please refer to Figure 1 The specific process of the target action prediction method based on the time-series knowledge graph according to the embodiment of the present application includes operations S110-S150.

[0028] In operation S110, multi-source data is acquired, and the multi-source data includes multiple entities.

[0029] For example, the relevant data of multiple moving entities can be acquired through a receiving device, and the entities can be airplanes or ships. The data can be the categories, types, discovery and disappearance times, coordinates, directions, altitudes, distances and speeds of the track coverage areas of the multiple entities.

[0030] In operation S120, the correlation between the entities is obtained based on a relationship classification model, and the redundancy check and alignment deduplication of the correlation are completed, wherein the relationship classification model is obtained through classification learning.

[0031] For example, the acquired multi-source data is input into the established relationship classification model, and the correlation between the entities can be obtained. The relationship classification model is obtained through classification learning of a decision tree model, which is described in detail below.

[0032] Figure 2 Fig. 5 schematically shows a flow chart of obtaining a relationship classification model through classification learning according to an embodiment of the present application.

[0033] For details, please refer to Figure 2In the embodiments of the present application, the relationship classification model is obtained through classification learning, and the classification learning process specifically includes operations S210-S220.

[0034] In operation S210, a relationship category training set is obtained.

[0035] In operation S220, a decision tree model is constructed, and the decision tree model is learned based on the relationship category training set to obtain the relationship classification model.

[0036] For example, after obtaining the multi-source data, the relationship category data collection is designed to obtain a training set for machine classification learning. After obtaining the training set, the associated relationship features of the collected data are extracted, which can be numerical, categorical or text type. After extraction, standardization, normalization and other processing are required to facilitate subsequent model classification learning.

[0037] Further, the decision tree model is constructed, and the decision tree model is learned based on the relationship category training set to obtain the relationship classification model.

[0038] Specifically, in the embodiments of the present application, the xgboost implementation in the decision forest algorithm with the best classification effect for discrete type data is used for classification learning. The xgboost implementation is an ensemble learning algorithm, and the core is to construct a plurality of decision tree models, and to optimize the direction of the maximum gradient descent of the loss function of the previous decision tree model, so as to achieve the purpose of fast convergence and improved classification generalization performance.

[0039] Further, after obtaining the associated relationships between the entities, the associated relationships are checked for redundancy and aligned for deduplication.

[0040] In the embodiments of the present application, the redundancy check and alignment deduplication of the associated relationships include: checking the redundancy and alignment deduplication of the associated relationships through a semi-supervised learning entity alignment algorithm and an unsupervised scoring and sorting relationship disambiguation method.

[0041] Specifically, after constructing the associated relationships between the entities, a large number of repeated edges often appear, which can interfere with data analysis work, so it is necessary to check the redundancy of the associated relationships and to merge them. The relationship redundancy check is to determine whether there are some repeated relationships between two entities. First, the entity alignment algorithm based on semi-supervised learning is used to solve the entity alignment problem in the graph, and then a non-supervised relationship disambiguation method based on scoring and sorting is used to align and deduplicate the relationships between the entities.

[0042] Based on the above, after the alignment and deduplication of the association relationship is completed, a target entity among the multiple entities is determined according to the position and feature information of the entity. The target entity here is an entity used for subsequent monitoring and predicting the action trajectory.

[0043] In the embodiment of the application, determining the target entity among the multiple entities according to the position and feature information of the entity comprises: analyzing the association relationship between the multiple entities by using the calculation of the position association and the feature information membership degree, to identify the target entity among the multiple entities.

[0044] Specifically, the target association based on the radiation source electronic feature information is adopted, the potential relationship existing between the radiation source information and the target is analyzed by using the method of calculating the position association and the feature information membership degree, so as to identify the target, and the obtained result can be directly used for subsequent comprehensive processing.

[0045] In operation S130, after the target entity among the multiple entities is determined according to the position and feature information of the entity, an association tree of the association relationship between the target entities is dynamically constructed, to complete the construction of the time sequence knowledge graph.

[0046] Figure 3 A flowchart of dynamically constructing a relationship tree according to an embodiment of the application is schematically shown.

[0047] For details, please refer to Figure 3 In the embodiment of the application, the association tree of the association relationship between the target entities is dynamically constructed, comprising operations S310-S330.

[0048] In operation S310, a time association dataset and a space association dataset of the target entity are generated based on time matching and space association degree measurement.

[0049] In operation S320, a track matching between the target entities is performed based on the DTW algorithm, to generate a set of association relationships between the target entities.

[0050] In operation S330, a target entity having a target association relationship is searched in the set of association relationships based on the spanning tree algorithm matching, to determine a target association relationship between the multiple target entities, and generate an association tree.

[0051] In the embodiment of the application, after the target entity is determined, the related data of the target entity also includes the category and model of the target entity, the discovery and disappearance time, the coordinates, direction, height, distance and speed of the track coverage area.

[0052] For example, the data of the target entity is sorted, to obtain a track dataset S={N1, N2, …, N m}, N(C, M, t s, t f , t, X) represents the track information of a single target, where C represents the target category, M represents the target model, t s represents the target initial discovery time, t f represents the target disappearance time, t=t f -t s is the target duration, X{(x1, y1), ⋯, (x n , y n )} is the all point track coordinate data set of a single track.

[0053] Further, the data of the target entity is preprocessed, a given spatial range is taken, the motion duration is limited, the repeated data is deleted, and the error track point is corrected, etc. Data cleaning methods are used to obtain a data set sorted according to the target departure time. Secondly, the activity time correlation matching degree of multiple targets is designed, and the time correlation data set is generated by screening:

[0054]

[0055] In the formula: p i ' is the target set having time correlation with the i-th target entity, that is

[0056] .

[0057] Further, on this basis, the space correlation matching degree is designed, and the space correlation data set is generated by screening:

[0058]

[0059] In the formula: p i " is the target set having space correlation with the i-th target entity, that is

[0060] .

[0061] Finally, then, the DTW algorithm is used to match the space correlation data set obtained two by two, and the track overlap degree between two target entities is obtained, that is, the correlation relationship target set is

[0062]

[0063] In the formula: is the target set having correlation relationship with the target i, that is

[0064] .

[0065] Finally, the spanning tree method is used to find multiple target entities having correlation relationship, generate a correlation tree, and extract time related information from the data, integrate time and relationship, and form a time sequence knowledge graph of the target entity.

[0066] It can be understood that the above process of generating tree is also a multiple screening process, and the time correlation data set is generated through time correlation screening, the space correlation data set is generated through space correlation screening on the time correlation data set, the correlation relationship data set is generated through track coincidence degree screening on the space correlation data set by using the DTW algorithm, and finally the same branch target is found according to the generating tree method to generate the correlation tree.

[0067] In the embodiment of the application, the time sequence knowledge graph of the target entity is used to mine the correlation relationship rules between multiple target entities, which helps to determine the target scale. The massive target entity information such as typical target related attribute information, feature information, historical behavior data, time sequence graph knowledge data, etc. as input can sort, induce, analyze and mine the correlation relationship between typical target elements and the relationship between targets to realize the target state prediction analysis and target action auxiliary decision analysis of the typical target.

[0068] Through the embodiment of the application, firstly, the correlation relationship classification model is obtained through automatic classification learning of the correlation relationship based on the decision forest, then the redundant edges are eliminated by using the correlation relationship redundancy checking technology, and finally the time sequence knowledge graph is established by using the correlation tree generated by multiple screening, thereby solving the problem of false correlation between target entities.

[0069] Therefore, based on the time sequence knowledge graph established above, the spatio-temporal correlation and evolution prediction analysis of the target entity will be described in detail below.

[0070] In operation S140, based on the time sequence knowledge graph, the time sequence nonlinear correlation relationship between target entities is modeled and classified by using a deep neural network to generate a target space evolution model.

[0071] Figure 4 A flowchart for establishing a target space evolution model according to an embodiment of the application is schematically shown.

[0072] Please refer to Figure 4 In the embodiment of the application, based on the time sequence knowledge graph, the time sequence nonlinear correlation relationship between target entities is modeled and classified by using a deep neural network to generate a target space evolution model, including operations S410-S430.

[0073] In operation S410, the spatio-temporal attributes of the target entity are extracted from real-time information, and the time sequence nonlinear correlation relationship between the target entities is formed.

[0074] In operation S420, the time sequence nonlinear correlation relationship between the target entities is mapped to a spatial high-dimensional projection vector by using a recurrent neural network.

[0075] In the embodiments of the present application, the extracted target entity dynamic attributes related to the time series input are combined with the clustering or deep neural network feature space mapping method to extract the spatio-temporal attribute data of the target entity from real-time information, and then a new time series nonlinear correlation relationship between target entities is formed by a shallow or deep classifier. Meanwhile, based on the latest time series correlation relationship between each target entity and its adjacent correlation target entity, the adjacent correlation relationship with a non-fixed number and non-fixed length is mapped to a deep feature vector with a fixed length by a recurrent neural network, and this is used as a high-dimensional projection vector of the target entity in the state space.

[0076] In operation S430, a target space evolution model is established based on the spatial high-dimensional projection vector and the random process related model.

[0077] In the embodiments of the present application, for each type of target entity, the dynamic evolution process of the target entity is described and modeled by using a random process related model based on the previously obtained information and the high-dimensional projection vector of the target entity in the state space, and the target space evolution model is trained by using a pre-collected training set to estimate the possible future state of the target entity.

[0078] In operation S150, the behavior pattern and motion trajectory of the target entity are predicted based on the target space evolution model.

[0079] In the embodiments of the present application, the behavior pattern and motion trajectory of the target entity are predicted based on the target space evolution model, which includes determining the predicted state of the target entity on the evolution path based on the target space evolution model, and forming the decision information of the target entity.

[0080] Specifically, according to the target space evolution model, the evolution law of each target entity can be obtained, so that the state of the target entity at a future time can be predicted and analyzed from each possible state of the target entity on the evolution path, and finally the decision information with high confidence can be formed.

[0081] Further, in the embodiments of the present application, the behavior pattern and motion trajectory of the target entity are predicted based on the target space evolution model, which also includes obtaining historical data to form the historical action law cases of the target entity, and determining the current state of the target entity by comparing the historical action law cases of the target entity based on the predicted state of the target entity on the evolution path.

[0082] Specifically, historical data is retrieved from a historical case database, which includes historical cases. Each historical case includes a problem and the response of the target entity in solving that problem. This can be understood as follows: the problem in each historical case refers to a specific historical event that occurred at a particular moment; the response or action of the target entity related to solving that problem is also included; and the responses or actions of other target entities to the event are also included. This process forms a pattern of historical behavior patterns for target entities. Furthermore, based on the predicted state of the target entity obtained above, the current state of the target entity is determined by comparing it with these historical pattern patterns.

[0083] Furthermore, in embodiments of the present invention, predicting the behavior patterns and trajectories of target entities based on the target space evolution model further includes: determining the behavior patterns and trajectories of target entities by dynamically sampling historical trajectory data of target entities and performing nonlinear function fitting based on the spatiotemporal attributes, evolutionary state, and historical action patterns of target entities.

[0084] Specifically, based on the spatiotemporal attributes, evolutionary states, and historical behavior patterns of the target entity obtained above, a dynamic sampling method is used to acquire the historical trajectory data sequence of the target entity. Then, a complex nonlinear function containing polynomial and transcendental functions is used to fit the time-series data sequence of the target trajectory to determine the current behavior pattern and trajectory of the target entity. In the embodiments of this invention, the function fitting method has the characteristics of strong specificity, strong interpretability, high fit, and good generalization, and can accurately predict the behavior pattern and trajectory of the target entity.

[0085] Through the embodiments of the present invention, based on the constructed time-series knowledge graph data, the associations between target entities and the associations between target entities and the environment recorded therein are used to complete the analysis of information such as spatiotemporal distribution patterns, spatiotemporal association rules, spatiotemporal clustering rules, and spatiotemporal feature rules, effectively improving the accuracy of analysis and prediction of the behavioral patterns of target entities.

[0086] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A target action prediction method based on temporal knowledge graph, characterized in that, The method includes: Acquire multi-source data, which includes multiple entities; Based on the relationship classification model, the association relationships between the entities are obtained, and the redundancy check and alignment deduplication of the association relationships are completed. The relationship classification model is obtained by classification learning. After determining the target entity among the multiple entities based on the entity's location and feature information, a relationship tree of association between the target entities is dynamically constructed to complete the construction of the temporal knowledge graph. Based on the temporal knowledge graph, a target space evolution model is generated by using a deep neural network to model and classify the temporal nonlinear relationships between the target entities. Based on the target space evolution model, predict the behavior pattern and trajectory of the target entity; The step of generating a target spatial evolution model based on the temporal knowledge graph, by using a deep neural network to model and classify the temporal nonlinear relationships between the target entities, includes: Extract the spatiotemporal attributes of the target entities from real-time information and form a temporal nonlinear correlation between the target entities; The temporal nonlinear correlation between the target entities is mapped into a high-dimensional spatial projection vector through a recurrent neural network; Based on the aforementioned spatial high-dimensional projection vector and stochastic process correlation model, a target space evolution model is established; The dynamic construction of the association tree of relationships between the target entities includes: Based on time matching and spatial correlation metrics, a time correlation dataset and a spatial correlation dataset of the target entity are generated. Based on the DTW algorithm, track matching is performed between the target entities to generate a set of association relationships between the target entities; Based on the spanning tree algorithm, target entities with target relationships are matched and searched in the set of relationships to determine the target relationships between the multiple target entities and generate a relationship tree.

2. The target action prediction method based on time-series knowledge graph according to claim 1, characterized in that, The prediction of the target entity's behavior pattern and trajectory based on the target space evolution model includes: Based on the target space evolution model, the predicted state of the target entity on the evolution path is determined, and the decision information of the target entity is formed.

3. The target action prediction method based on time-series knowledge graph according to claim 1 or 2, characterized in that, The method of predicting the behavior pattern and trajectory of the target entity based on the target space evolution model further includes: Acquire historical data to form case studies of the target entity's historical behavior patterns; Based on the predicted state of the target entity on the evolution path, the current state of the target entity is determined by comparing historical behavior patterns of the target entity.

4. The target action prediction method based on time-series knowledge graph according to claim 3, characterized in that, The method of predicting the behavior pattern and trajectory of the target entity based on the target space evolution model further includes: Based on the spatiotemporal attributes, evolutionary state, and historical behavior patterns of the target entity, the behavior patterns and motion trajectories of the target entity are determined by dynamically sampling the historical trajectory data of the target entity and fitting nonlinear functions.

5. The target action prediction method based on time-series knowledge graph according to claim 1, characterized in that, The relationship classification model is obtained through classification learning, and the classification learning process includes: Obtain a training set of relation types; A decision tree model is constructed, and the decision tree model is learned based on the training set of the relationship types to obtain the relationship classification model.

6. The target action prediction method based on time-series knowledge graph according to claim 1, characterized in that, The redundancy check and deduplication of the association relationship includes: The redundancy check and deduplication of the association relationship are performed by using a semi-supervised learning entity alignment algorithm and an unsupervised scoring and ranking relationship disambiguation method.

7. The target action prediction method based on time-series knowledge graph according to claim 1, characterized in that, Determining the target entity among the plurality of entities based on the entity's location and feature information includes: By utilizing the location correlation of entities and the membership degree of the feature information, the association relationship between the multiple entities is analyzed to identify the target entity among the multiple entities.

8. The target action prediction method based on time-series knowledge graph according to claim 1, characterized in that, The multi-source data includes the entity's category, model, discovery and disappearance time, and the coordinates, orientation, altitude, distance, and speed of the area covered by the flight track.

Images