Method and device for associating text data with radar tracks

Abstract

The present application belongs to the technical field of multimodal data fusion, and specifically discloses a method and device for associating text data and radar tracks, wherein the method comprises: performing entity recognition and spatiotemporal change relationship extraction on the text data to be associated; converting the location entity into longitude and latitude coordinates, taking the longitude and latitude coordinates as nodes, and constructing a text data graph according to the chronological order of arrival time of each node, and the distance between nodes is the edge weight; intercepting the radar track within the departure time and arrival time range of the target, if the time period corresponding to the intercepted radar track is greater than a preset threshold, taking the longitude and latitude coordinates of each track point in the radar track as a node, and constructing a radar track graph according to the chronological order of arrival time of each node, and the distance between nodes is the edge weight; determining the embedding vectors of the text data graph and the radar track graph based on a graph neural network; splicing the embedding vectors of the text data graph and the radar track graph and inputting them into a binary classifier to obtain the association result of the text data.

Description

Technical Field

[0001] The present application belongs to the technical field of multimodal data fusion, and more specifically, to a method and device for associating text data with radar tracks. Background Art

[0002] Text data and radar tracks are widely used in many fields. For example, in air traffic management, text data includes flight number, aircraft model, flight time, etc., and radar tracks mainly refer to aircraft tracks. The association of text data and radar tracks allows air traffic controllers to effectively monitor and direct aircraft to ensure flight safety. For another example, in ocean monitoring, radar systems are used to track ships, and text data may include ship registration information, navigation plans, etc. Associating this information can help the Coast Guard or other maritime management agencies monitor maritime traffic.

[0003] In the related art, common methods for associating text data with radar tracks include rule-based association methods and probability-based association methods. Rule-based association methods rely on pre-defined rules, such as matching radar tracks and text data based on flight numbers, flight plans, take-off and landing airports, etc. Rules can be simple logical judgments or complex algorithms, depending on the needs and complexity of the system. Probability-based association methods use probability theory and statistical methods to evaluate the correlation between text data and radar tracks, such as using Bayesian filters to calculate the probability that a given radar track comes from specific text data.

[0004] However, rule-based association methods have poor flexibility and are difficult to handle complex and changeable scenarios. They are prone to false associations or missed associations for situations that are not covered by the rules. Probability-based association methods have high computational complexity, especially when dealing with large amounts of data and complex models. They require accurate prior probabilities and model parameters, which are often difficult to obtain accurately. Summary of the invention

[0005] In view of the defects existing in the related technologies, the embodiments of the present application provide a method and device for associating text data and radar tracks, aiming to solve the problems of low flexibility and accuracy of existing association methods.

[0006] In a first aspect, an embodiment of the present application provides a method for associating text data with radar tracks, including:

[0007] Perform entity recognition and spatiotemporal change relationship extraction on the associated text data to obtain the target entity, time entity and location entity described in the text data, as well as the departure time, arrival time, departure location and arrival location of the target;

[0008] The location entity is converted into longitude and latitude coordinates, and the longitude and latitude coordinates are used as nodes. The text data graph is constructed according to the chronological order of the arrival time of each node, and the distance between nodes is the edge weight;

[0009] Intercept the radar track within the departure and arrival time range of the target. If the time period corresponding to the intercepted radar track is greater than the preset threshold, take the longitude and latitude coordinates of each track point in the radar track as a node, and construct a radar track graph according to the chronological order of the arrival time of each node. The distance between nodes is the edge weight.

[0010] Determine the embedding vectors of the text data graph and the radar track graph based on the graph neural network;

[0011] The embedding vectors of the text data graph and the radar track graph are concatenated and input into the binary classifier to obtain the association result of the text data to be associated.

[0012] In a second aspect, an embodiment of the present application further provides a device for associating text data with radar tracks, including:

[0013] An acquisition module is used to perform entity recognition and spatiotemporal change relationship extraction on the text data to be associated, and acquire the target entity, time entity and location entity described in the text data, as well as the departure time, arrival time, departure location and arrival location of the target;

[0014] The first graph construction module is used to convert the location entity into longitude and latitude coordinates, take the longitude and latitude coordinates as nodes, and construct a text data graph according to the time sequence of the arrival time of each node, and the distance between the nodes is the edge weight;

[0015] The second graph construction module is used to intercept the radar track within the departure time and arrival time range of the target. If the time period corresponding to the intercepted radar track is greater than the preset threshold, the longitude and latitude coordinates of each track point in the radar track are used as nodes, and the radar track graph is constructed according to the time sequence of the arrival time of each node, and the distance between the nodes is the edge weight;

[0016] An embedding vector determination module is used to determine the embedding vectors of the text data graph and the radar track graph based on the graph neural network;

[0017] The association module is used to concatenate the embedding vectors of the text data graph and the radar track graph and input them into the binary classifier to obtain the association result of the text data to be associated.

[0018] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0019] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0020] In a fifth aspect, an embodiment of the present application further provides a computer program product, which, when executed on a processor, enables the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0021] The method and device for associating text data and radar tracks provided in the embodiments of the present application perform entity recognition and relationship extraction on text data to obtain the spatiotemporal variation content of the target described in the text data, then perform graph modeling on the text data and radar tracks, convert the spatiotemporal variation features described in the text data and radar tracks into graph structure data, and finally implement the association calculation of the text data and radar tracks through a graph neural network, thereby getting rid of the traditional method's reliance on association rules, reducing the complexity of calculation, and improving the flexibility and accuracy of association processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present application or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 It is one of the flowcharts of the method for associating text data and radar tracks provided in the embodiment of the present application;

[0024] Figure 2 This is the second flow chart of the method for associating text data and radar tracks provided in the embodiment of the present application;

[0025] Figure 3 It is a schematic diagram of the structure of a text data graph provided in an embodiment of the present application;

[0026] Figure 4 It is a schematic diagram of the structure of the radar track diagram provided in the embodiment of the present application;

[0027] Figure 5 This is the third flow chart of the method for associating text data and radar tracks provided in the embodiment of the present application;

[0028] Figure 6 It is a structural schematic diagram of a device for associating text data and radar tracks provided in an embodiment of the present application;

[0029] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] Figure 1 This is one of the flow charts of the method for associating text data and radar tracks provided in the embodiment of the present application, such as Figure 1 As shown, the method at least includes the following steps:

[0032] S101, performing entity recognition and spatiotemporal change relationship extraction on the text data to be associated, obtaining the target entity, time entity and location entity described in the text data, as well as the departure time, arrival time, departure location and arrival location of the target.

[0033] Specifically, the text data to be associated describes the spatiotemporal changes of a specific target.

[0034] Entity recognition and spatiotemporal variation relationship extraction are performed on the text data to be associated. Optionally, entity recognition and spatiotemporal variation relationship extraction are performed on the text data to be associated by a manual extraction method or a conventional natural language processing method.

[0035] Entity recognition obtains the target entity, time entity and location entity described in the text data, and spatiotemporal change relationship extraction obtains the content of the target spatiotemporal change relationship described in the text data, where the time change relationship is represented by the departure time and arrival time of the target, and the spatial change relationship is represented by the departure position and arrival position of the target. Optionally, the result of the spatial change relationship extraction also includes the path position of the target.

[0036] S102, converting the location entity into longitude and latitude coordinates, taking the longitude and latitude coordinates as nodes, constructing a text data graph according to the chronological order of the arrival time of each node, and the distance between the nodes as the edge weight.

[0037] Specifically, the basic components of a directed graph include nodes and edges. Each edge points from one node to another node, that is, each edge has directionality. The edge weight represents a quantitative measure of the edge connecting two nodes in the graph.

[0038] The identified location entities are converted into specific longitude and latitude coordinates through conventional text geocoding methods. The longitude and latitude coordinates are used as nodes, and a directed graph is constructed according to the chronological order of each node to obtain a text data graph. The distance between nodes is the edge weight.

[0039] S103, intercepting the radar track within the departure time and arrival time range of the target. If the time period corresponding to the intercepted radar track is greater than a preset threshold, the longitude and latitude coordinates of each track point in the radar track are used as nodes, and a radar track graph is constructed according to the time sequence between the nodes, and the distance between the nodes is the edge weight.

[0040] Specifically, after clarifying the departure time and arrival time of the target, the radar track within the target departure time and arrival time range is intercepted, the radar track data within the time period between the target departure time and arrival time is obtained, and it is determined whether the intercepted radar track is likely to be the radar track corresponding to the target.

[0041] The specific judgment method is: compare the time period corresponding to the intercepted radar track with the preset threshold. If it is less than the preset threshold, it is judged that the association between the radar track and the text data to be associated has failed; if it is greater than the preset threshold, it means that the radar track may be associated with the file data to be associated, and the association relationship needs to be further determined.

[0042] For radar tracks greater than the preset threshold, the longitude and latitude coordinates of each track point are used as nodes, and a directed graph is constructed according to the chronological order of the arrival time of each node to obtain the radar track graph, and the distance between nodes is the edge weight.

[0043] Optionally, the preset threshold is 1 / 2 of the duration between the departure time and the arrival time of the target identified by the text data to be associated.

[0044] S104. Determine the embedding vectors of the text data graph and the radar track graph based on the graph neural network.

[0045] Specifically, a graph neural network (GNN) is a deep learning model specifically designed to process graph structure data. It simulates the operation of neural networks on graph structures to learn the characteristics of nodes and edges and is used to solve various graph-related tasks. Graph neural networks include graph convolutional networks, graph attention networks, graph autoencoders, graph generation networks, and graph recurrent networks.

[0046] Graph-level Embedding aims to represent the entire graph (including all nodes and edges) as a single vector, and use graph neural networks to determine the full-graph embedding representation of the text data graph and radar track graph, obtain the embedding vector of the full graph of the text data graph and radar track graph, and use the full-graph embedding vector to represent all nodes and edges in the text data graph and radar track graph.

[0047] S105, concatenating the embedding vectors of the text data graph and the radar track graph and inputting them into a binary classifier to obtain an association result of the text data to be associated.

[0048] Specifically, the two embedding vectors of the text data graph and the radar track graph are concatenated and input into a binary classifier to calculate the association result of the text data to be associated, and the association result includes association success and association failure.

[0049] The binary classifier is used to classify the input data into two different categories, which are usually called positive and negative, or represented by labels 1 and 0, yes and no, etc. In the case of associating text data with radar tracks, the output is two categories: association success and association failure.

[0050] Optionally, the binary classifier uses a multi-layer perceptron (MLP), which is a feed-forward artificial neural network model, including an input layer, one or more hidden layers, and an output layer, which can be used for binary classification problems. In a binary classification task, the output layer generally contains a neuron, whose output is processed by an activation function and interpreted as the probability of a certain category.

[0051] Optionally, the input layer dimension of the MLP is consistent with the embedding vector, the activation function of the hidden layer is set to the ReLU function, the output layer is the Softmax classifier, and the classification results include two categories: association success and association recognition.

[0052] The method for associating text data and radar tracks provided in the embodiment of the present application obtains the spatiotemporal variation content of the target described in the text data by performing entity recognition and relationship extraction on the text data, then performs graph modeling on the text data and radar tracks, converts the spatiotemporal variation features described in the text data and radar tracks into graph structure data, and finally implements the association calculation of the text data and radar tracks through a graph neural network, which gets rid of the traditional method's reliance on association rules, reduces the complexity of calculation, and improves the flexibility and accuracy of association processing.

[0053] In some embodiments, the graph neural network is a graph attention network, and S104 specifically includes:

[0054] Determine the correlation coefficient between nodes in each layer of the graph attention network through the graph attention operation;

[0055] Determine the attention coefficient between nodes in each layer of the graph attention network based on the correlation coefficient;

[0056] Determine the vector representation of each node in each layer of the graph attention network based on the attention coefficient;

[0057] The embedding vector of the entire graph structure is determined based on the vector representation of each node, and the graph structure includes a text data graph and a radar track graph.

[0058] Specifically, the Graph Attention Network (GAT) is a type of graph neural network that introduces an attention mechanism to assign different weights to the neighbors of each node to achieve dynamic information aggregation. In the process of determining the full-graph embedding representation of the text data graph and the radar track graph, the graph attention network used can have multiple layers, for example, 2 layers.

[0059] First, the correlation coefficients between nodes in each layer of the graph attention network are calculated through the graph attention operation, specifically satisfying:

[0060]

[0061] in, is the correlation coefficient between nodes i and j in the l-th layer graph attention network, and are the vector representations of node i and node j in the l-th layer graph attention network, W (l) is the transformation matrix in the l-th layer graph attention network, CONCAT(·) is the concatenation function, a (1) is the transformed row vector in the l-th layer graph attention network, transforming the concatenated feature vector into a real number.

[0062] Then the correlation coefficient is used to calculate the attention coefficient between nodes, which specifically satisfies:

[0063]

[0064] in, is the attention coefficient (or attention weight) between nodes i and j in the l-th layer graph attention network, N i is the neighbor node index set of node i, and are the correlation coefficients between nodes i and j and between node i and its neighbor k node in the l-th layer graph attention network, respectively.

[0065] Finally, the vector representation of each node is calculated, which satisfies:

[0066]

[0067] in, is the vector representation of the output of node i in the graph structure in the lth layer of the graph attention network, that is, the vector representation in the l+1th layer of the graph attention network, W (l) is the transformation matrix in the l-th layer graph attention network, N i is the neighbor node index set of node i, is the attention coefficient between nodes i and j in layer l, and σ(·) is the activation function.

[0068] a (l) and W (l) is the training parameter of the l-th layer graph attention network.

[0069] The i-node vector representation of the last layer output of the graph attention network is recorded as Then the full image embedding vector h can be calculated G , specifically meet:

[0070]

[0071] Among them, N is the set of all node subscripts in the graph structure.

[0072] It should be noted that the text data graph and radar track graph use the same graph neural network during training and calculation.

[0073] In some embodiments, the method for associating text data with radar tracks further includes:

[0074] The scale of edge weights in the statistical text data graph is used to perform pooling on the radar track graph based on the scale.

[0075] Specifically, before calculating the embedding vectors of the entire text data graph and radar track graph, the radar track graph is pooled using the scale of the edge weights in the text data graph, some of the nodes are discarded and the corresponding edges are reconstructed, so that the scales of the features of the radar track graph and the text data graph are aligned.

[0076] In some embodiments, the scale of edge weights in the statistical text data graph includes:

[0077] The edge weights in the text data graph are traversed, and the minimum value of the edge weight is determined as the scale of the edge weights in the text data graph.

[0078] Specifically, the edge weight scale is used to quantify the degree of association or importance of the edges between nodes in the graph structure. Traverse the edge weights in the text data graph and select the minimum value d min is the scale of the text data graph, used for pooling of radar track graphs.

[0079] In some embodiments, the radar track image is pooled based on the scale, including:

[0080] Traverse all nodes in the radar track graph except the starting position in chronological order;

[0081] If the first edge weight between the current node and the previous node and the second edge weight between the current node and the next node meet the preset conditions, the current node is retained;

[0082] If the first edge weight and the second edge weight do not meet the preset conditions, the current node and the related edges are discarded, the edge between the previous node and the next node is rebuilt and the edge weight is calculated until the traversal is completed to obtain the pooled radar track map.

[0083] Specifically, a preset condition is set based on the scale of the edge weights in the text data graph to determine whether to perform discarding and edge reconstruction operations on the nodes in the radar track graph.

[0084] The set of all nodes in the radar track graph P = {p1, p2, ..., p i ,…,p n-1 ,p n}, where n is the number of nodes in the radar track map, p1 to p n Arrange the nodes in the order of the directed graph. Traverse p2 to p n-1 Each node of the current node is denoted by p c , the previous node is denoted as p pre , the next node is denoted as p post .

[0085] Determine p pre to p c The first edge weight d pre and p c to p post The second edge weight d post Whether the preset conditions are met, if so, the current node is retained, if not, the current node and related edges are discarded and p is rebuilt pre and p post The edges between them are traversed and the edge weights are calculated until the traversal is completed to obtain the pooled radar track map.

[0086] In some embodiments, the preset condition specifically satisfies:

[0087] d pre ≥0.6d min

[0088] d pre +d post ≥ 1.5 days min

[0089] It should be noted that the principle of setting the preset conditions is to make the distance between radar track nodes as close as possible to the text data, but not to make the edge weight too large after discarding the node. Therefore, the preset conditions can be set specifically based on experience, and the coefficients 0.6 and 1.5 can be adjusted.

[0090] The technical solution provided in the embodiment of the present application is further illustrated below by using a specific example.

[0091] Figure 2 This is a second flow chart of the method for associating text data and radar tracks provided in an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0092] Step 1: Entity recognition and relationship extraction of text data. Entity recognition obtains the target entity, time entity and location entity described in the text data, and spatiotemporal change relationship extraction obtains the content of the target spatiotemporal change relationship described in the text data, where the time change relationship is represented by the departure time and arrival time of the target, and the space change relationship is represented by the departure position, arrival position and path position of the target.

[0093] Table 1 is an example of the results of entity recognition and relation extraction from text data.

[0094] Table 1

[0095]

[0096] Step 2: Text data graph modeling.

[0097] For the location entity obtained in step 1, a dictionary-based text geocoding method is used to convert the entity text into specific latitude and longitude geographic coordinates. Table 2 is an example of a text geocoding dictionary.

[0098] Table 2

[0099] Serial number Text Data Latitude and longitude coordinates 1 Wuhan Tianhe Airport Longitude: 114.2081, Latitude: 30.7838 2 Yantai Penglai Airport Longitude: 120.9911, Latitude: 37.6556 3 Zhoushan Port Longitude: 122.2060, Latitude: 29.9931 4 Shenjiamen Port Longitude: 122.2437, Latitude: 29.9568 5 Liuheng Port Longitude: 122.1272, Latitude: 29.7433 6 Xiangshan Port Longitude: 121.8176, Latitude: 29.5727 … … …

[0100] Use the converted longitude and latitude coordinates as nodes and construct a directed graph according to the arrival order in step 1, which is the text data graph. Figure 3 is a schematic diagram of the structure of the text data graph provided in the embodiment of the present application, such as Figure 3 As shown, the edge weight is the distance between nodes.

[0101] Step 3: Radar track diagram modeling.

[0102] Intercept the radar track within the departure time and arrival time range of the target obtained in step 1. If the time period corresponding to the intercepted radar track is greater than 1 / 2 of the time range, it is determined that the radar track fails to be associated with the text data. Otherwise, the longitude and latitude coordinates of each track point in the intercepted radar track are used as nodes, and a directed graph is constructed according to the time sequence of the track points, which is the radar track graph. Figure 4 is a schematic diagram of the structure of the radar track diagram provided in the embodiment of the present application, such as Figure 4 As shown, the edge weight is the distance between nodes.

[0103] Step 4: Radar track map pooling.

[0104] Traverse the edge weights in the text data graph and select the minimum value d min is the scale of the text data graph, used for pooling of radar track graphs.

[0105] The set of all nodes in the radar track graph P = {p1, p2, ..., p i ,…,p n-1 ,p n}, where n is the number of nodes in the radar track map, p1 to p n Arrange the nodes in the order of the directed graph. Traverse p2 to p n-1 Each node of the current node is denoted by p c , the previous node is denoted as p pre , the next node is denoted as p post .

[0106] Determine p pre to p c The first edge weight d pre and p c to p post The second edge weight d post Whether it meets:

[0107] d pre ≥0.6d min

[0108] d pre +d post ≥ 1.5 days min

[0109] If it is satisfied, keep the current node; if it is not satisfied, discard the current node and related edges and rebuild p pre and p post The edges between them are traversed and the edge weights are calculated until the traversal is completed to obtain the pooled radar track map.

[0110] Step 5: Full-image representation learning.

[0111] Based on the graph attention network, the full-graph embedding representation of the text data graph and the radar track graph is realized, and the embedding vectors of the full-graph of the two graph structures are obtained.

[0112] First, the correlation coefficients between nodes in each layer of the graph attention network are calculated through the graph attention operation, specifically satisfying:

[0113]

[0114] in, is the correlation coefficient between nodes i and j in the l-th layer graph attention network, and are the vector representations of node i and node j in the l-th layer graph attention network, W (l) is the transformation matrix in the l-th layer graph attention network, CONCAT(·) is the concatenation function, a (1) is the transformed row vector in the l-th layer graph attention network, transforming the concatenated feature vector into a real number.

[0115] Then the correlation coefficient is used to calculate the attention coefficient between nodes, which specifically satisfies:

[0116]

[0117] in, is the attention coefficient (or attention weight) between nodes i and j in the l-th layer graph attention network, N i is the neighbor node index set of node i, and are the correlation coefficients between nodes i and j and between node i and its neighbor k node in the l-th layer graph attention network, respectively.

[0118] Finally, the vector representation of each node is calculated, which satisfies:

[0119]

[0120] in, is the vector representation of the output of node i in the graph structure in the lth layer of the graph attention network, that is, the vector representation in the l+1th layer of the graph attention network, W (l) is the transformation matrix in the l-th layer graph attention network, N i is the neighbor node index set of node i, is the attention coefficient between nodes i and j in layer l, and σ(·) is the activation function.

[0121] a (l) and W (l) is the training parameter of the l-th layer graph attention network.

[0122] The i-node vector representation of the last layer output of the graph attention network is recorded as Then the full image embedding vector h can be calculated G , specifically meet:

[0123]

[0124] Among them, N is the set of all node subscripts in the graph structure.

[0125] Specifically, the graph attention network is a two-layer network. The initialization codes of all nodes in the text data graph and radar track graph are calculated using the longitude and latitude coordinates of the nodes through the Earth Space Grid Coding Rules (GB / T40087-2021). According to the Earth Space Grid Coding Rules, 32-bit longitude and latitude codes are calculated for each node. After Morton crossover of the two sets of codes, the first 24 bits are selected as the initial code of the node. The network input layer is 24-dimensional, and the dimensions of the two-layer graph attention network are set to 64 and 32 respectively.

[0126] Step 6: Calculate the correlation results.

[0127] The two embedding vectors of the text data graph and the radar track graph are concatenated and input into the binary classifier to calculate the association result of the text data to be associated, which includes association success and association failure. The binary classifier uses a multi-layer perceptron. The multi-layer perceptron is set to 3 layers, the input layer dimension is 64 dimensions, and the hidden layer is set to 64 dimensions.

[0128] Figure 5 FIG. 3 is a flow chart of a method for associating text data with radar tracks provided in an embodiment of the present application. Figure 5 As shown in the figure, the overall network model is composed of a graph attention network and a multi-layer perceptron. The model loss function preferably uses a binary cross entropy loss function, which specifically satisfies:

[0129]

[0130] Among them, N is the number of training set samples, y i is the label of the i-th sample, 1 for successful association and 0 for failed association, y i ′ is the classification probability of the i-th sample output by the multilayer perceptron.

[0131] During the training of the model, the training data set is preferably manually annotated. The training data set is constructed as follows: select text data describing the change in the spatial position of a specific target and its corresponding radar track, process them according to steps 1 to 4, and annotate the obtained text data graph and radar track graph as a positive sample, that is, the label is successful association; select text data describing the change in the spatial position of a specific target and its non-corresponding radar track, process them according to steps 1 to 4, and annotate the obtained text data graph and radar track graph as a negative sample, that is, the label is failed association. The training data set consists of samples that have been annotated multiple times. The number of positive samples and negative samples in the training data set remains consistent.

[0132] The method for associating text data and radar tracks provided in the embodiment of the present application performs entity recognition and relationship extraction on the text data to obtain the spatiotemporal variation content of the target described in the text data, then performs graph modeling on the text data and the radar tracks, converts the spatiotemporal variation features described in the text data and the radar tracks into graph structure data, and performs pooling processing on the radar track graph using the scale of the edge weights in the text data graph to align the scales between the radar track graph and the text data graph features, and finally implements the association calculation of the text data and the pooled radar tracks through a graph neural network, thereby getting rid of the traditional method's reliance on association rules, reducing the complexity of calculation, and improving the flexibility and accuracy of association processing.

[0133] Figure 6 is a schematic diagram of the structure of the device for associating text data and radar tracks provided in an embodiment of the present application, such as Figure 6 As shown, the device at least includes:

[0134] The acquisition module 601 is used to perform entity recognition and spatiotemporal change relationship extraction on the text data to be associated, and acquire the target entity, time entity and location entity described in the text data, as well as the departure time, arrival time, departure location and arrival location of the target;

[0135] The first graph structure construction module 602 is used to convert the location entity into longitude and latitude coordinates, take the longitude and latitude coordinates as nodes, and construct a text data graph according to the time sequence of the arrival time of each node, and the distance between the nodes is the edge weight;

[0136] The second graph structure component module 603 is used to intercept the radar track within the range of the departure time and arrival time of the target. If the time period corresponding to the intercepted radar track is greater than the preset threshold, the longitude and latitude coordinates of each track point in the radar track are used as nodes, and the radar track graph is constructed according to the time sequence of the arrival time of each node, and the distance between the nodes is the edge weight;

[0137] An embedding vector determination module 604 is used to determine the embedding vectors of the text data graph and the radar track graph based on the graph neural network;

[0138] The association module 605 is used to concatenate the embedding vectors of the text data graph and the radar track graph and input them into a binary classifier to obtain an association result of the text data to be associated.

[0139] In some embodiments, the graph neural network is a graph attention network, and the embedding vector determination module 604 is specifically used to:

[0140] Determine the correlation coefficient between nodes in each layer of the graph attention network through the graph attention operation;

[0141] Determine the attention coefficient between nodes in each layer of the graph attention network based on the correlation coefficient;

[0142] Determine the vector representation of each node in each layer of the graph attention network based on the attention coefficient;

[0143] The embedding vector of the entire graph structure is determined based on the vector representation of each node, and the graph structure includes a text data graph and a radar track graph.

[0144] In some embodiments, the correlation coefficient satisfies:

[0145]

[0146] The attention coefficient satisfies:

[0147]

[0148] The vector representation of the node satisfies:

[0149]

[0150] The embedding vector of the entire graph structure satisfies:

[0151]

[0152] in, is the correlation coefficient between nodes i and j in the l-th layer graph attention network, and are the vector representations of node i and node j in the l-th layer graph attention network, W (l) is the transformation matrix in the l-th layer graph attention network, CONCAT(·) is the concatenation function, a (l) is the transformed row vector in the l-th layer graph attention network, transforming the concatenated feature vector into a real number; is the attention coefficient between nodes i and j in the l-th layer graph attention network, N i is the neighbor node index set of node i, is the correlation coefficient between node i and its neighbor node k in the l-th layer graph attention network; is the vector representation of node i in the l+1th layer of the graph attention network in the graph structure, σ(·) is the activation function; h G is the embedding vector of the entire graph structure, N is the set of all node subscripts in the graph structure, It is the i-node vector representation output by the last layer of the graph attention network.

[0153] In some embodiments, the apparatus further comprises:

[0154] The pooling module is used to count the scale of edge weights in the text data graph and perform pooling on the radar track graph based on the scale.

[0155] In some embodiments, the scale of edge weights in the statistical text data graph includes:

[0156] The edge weights in the text data graph are traversed, and the minimum value of the edge weight is determined as the scale of the edge weights in the text data graph.

[0157] In some embodiments, the radar track image is pooled based on the scale, including:

[0158] Traverse all nodes in the radar track graph except the starting position in chronological order;

[0159] If the first edge weight between the current node and the previous node and the second edge weight between the current node and the next node meet the preset conditions, the current node is retained;

[0160] If the first edge weight and the second edge weight do not meet the preset conditions, the current node and related edges are discarded, the edge between the previous node and the next node is rebuilt and the edge weight is calculated until the traversal is completed to obtain the pooled radar track map;

[0161] Among them, the preset conditions are based on the scale setting of edge weights in the text data graph.

[0162] In some embodiments, the preset condition is satisfied:

[0163] d pre ≥0.6d min

[0164] d pre +d post ≥ 1.5 days min

[0165] Among them, d pre is the first edge weight, d post is the second edge weight, d min is the scale of edge weights in the text data graph.

[0166] In some embodiments, the preset threshold is 1 / 2 of the duration between the departure time and the arrival time of the target.

[0167] In some embodiments, the binary classifier uses a multi-layer perceptron.

[0168] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0169] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0170] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device. The device may include: at least one memory for storing programs and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the above embodiment.

[0171] Figure 7 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 7 As shown, the electronic device may include: a processor (processor) 701, a communication interface (Communications Interface) 702, a memory (memory) 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704. The processor 701 can call the software instructions in the memory 703 to execute the method described in the above embodiment.

[0172] In addition, the logic instructions in the above-mentioned memory 703 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present application.

[0173] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0174] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0175] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0176] The method steps in the embodiments of the present application can be implemented by hardware, or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (Random Access Memory, RAM), flash memory, read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0177] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0178] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0179] It is easy for those skilled in the art to understand that the above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for associating text data with radar tracks, characterized in that: include: Performing entity recognition and spatiotemporal change relationship extraction on the text data to be associated, obtaining the target entity, time entity and location entity described in the text data, as well as the departure time, arrival time, departure location and arrival location of the target; The location entity is converted into longitude and latitude coordinates, and the longitude and latitude coordinates are used as nodes, and a text data graph is constructed according to the time sequence of the arrival time of each node, and the distance between nodes is the edge weight; Intercept the radar track within the departure time and arrival time range of the target. If the time period corresponding to the intercepted radar track is greater than a preset threshold, take the longitude and latitude coordinates of each track point in the radar track as a node, and construct a radar track graph according to the time sequence of the arrival time of each node, with the distance between nodes as the edge weight; Determine the embedding vectors of the text data graph and the full radar track graph based on a graph neural network; The embedded vectors of the text data graph and the radar track graph are spliced ​​and input into a binary classifier to obtain an association result of the text data to be associated; The method further comprises: Counting the scale of edge weights in the text data graph, and performing pooling processing on the radar track graph based on the scale; The performing pooling processing on the radar track image based on the scale includes: Traversing all nodes in the radar track graph except the starting position in chronological order; If the first edge weight between the current node and the previous node and the second edge weight between the current node and the next node meet the preset conditions, retain the current node; If the first edge weight and the second edge weight do not meet the preset condition, discard the current node and related edges, rebuild the edge between the previous node and the next node and calculate the edge weight until the traversal is completed to obtain a pooled radar track map; Wherein, the preset condition is set based on the scale of the edge weights in the text data graph.

2. The association method according to claim 1, characterized in that: The graph neural network is a graph attention network, and the method of determining the embedding vectors of the text data graph and the full radar track graph based on the graph neural network includes: Determine the correlation coefficient between nodes in each layer of the graph attention network through the graph attention operation; Determine the attention coefficient between nodes in each layer of the graph attention network based on the correlation coefficient; Determine a vector representation of each node in each layer of the graph attention network based on the attention coefficient; An embedding vector of the entire graph structure is determined based on the vector representation of each node, wherein the graph structure includes the text data graph and the radar track graph.

3. The association method according to claim 2, characterized in that: The correlation coefficient satisfies: The attention coefficient satisfies: The vector representation of the node satisfies: The embedding vector of the entire graph structure satisfies: in, For the Layer Graph Attention Network Node and The correlation coefficient between nodes, and They are Node and The node is Vector representation in layer graph attention network, For the Transformation matrix in layer graph attention network, is the concatenation function, For the The transformed row vector in the layer graph attention network transforms the concatenated feature vector into a real number; For the Layer Graph Attention Network Node and The attention coefficient between nodes, for The node's neighbor node index set, For the Layer Graph Attention Network Nodes and Neighbors Correlation coefficient between nodes; In the graph structure The node is Vector representation in layer graph attention network, is the activation function; is the embedding vector of the entire graph structure, is the set of all node subscripts in the graph structure, The output of the last layer of the graph attention network Node vector representation.

4. The association method according to claim 1, characterized in that: The scale for counting the edge weights in the text data graph includes: The edge weights in the text data graph are traversed, and the minimum value of the edge weight is determined as the scale of the edge weight in the text data graph.

5. The association method according to claim 1, characterized in that: The pre-conditions are met: in, is the first edge weight, is the second edge weight, is the scale of the edge weights in the text data graph.

6. The association method according to claim 1, characterized in that: The preset threshold is 1 / 2 of the duration between the departure time and the arrival time of the target.

7. The association method according to claim 1, characterized in that: The binary classifier adopts a multi-layer perceptron.

8. A device for associating text data with radar tracks, characterized in that: include: An acquisition module is used to perform entity recognition and spatiotemporal change relationship extraction on the text data to be associated, and acquire the target entity, time entity and location entity described in the text data, as well as the departure time, arrival time, departure location and arrival location of the target; A first graph construction module is used to convert the location entity into longitude and latitude coordinates, take the longitude and latitude coordinates as nodes, and construct a text data graph in the chronological order of the arrival time of each node, and the distance between the nodes is the edge weight; The second graph construction module is used to intercept the radar track within the departure time and arrival time range of the target. If the time period corresponding to the intercepted radar track is greater than a preset threshold, the longitude and latitude coordinates of each track point in the radar track are used as nodes, and a radar track graph is constructed according to the time sequence of the arrival time of each node, and the distance between the nodes is the edge weight; An embedding vector determination module, used to determine the embedding vectors of the text data graph and the full radar track graph based on a graph neural network; An association module, used for concatenating the embedding vectors of the text data graph and the radar track graph and inputting the concatenated vectors into a binary classifier to obtain an association result of the text data to be associated; The device also includes a pooling module, which is used to count the scale of the edge weights in the text data graph and perform pooling processing on the radar track graph based on the scale; The performing pooling processing on the radar track image based on the scale includes: Traversing all nodes in the radar track graph except the starting position in chronological order; If the first edge weight between the current node and the previous node and the second edge weight between the current node and the next node meet the preset conditions, retain the current node; If the first edge weight and the second edge weight do not meet the preset condition, discard the current node and related edges, rebuild the edge between the previous node and the next node and calculate the edge weight until the traversal is completed to obtain a pooled radar track map; Wherein, the preset condition is set based on the scale of the edge weights in the text data graph.

Images