A method for predicting spatiotemporal data, electronic device and storage medium

Abstract

The present invention provides a spatiotemporal data prediction method, electronic device and storage medium, which can be applied to the field of track prediction. The method includes: gridding and labeling the flight space and flight time series respectively; constructing a spatiotemporal knowledge graph of multi-dimensional relational data based on the labeled and encoded flight three-dimensional space grid and the labeled and encoded flight time grid; extracting a flight hotspot region sequence that changes with time using the spatiotemporal knowledge graph, and predicting the flight hotspot region based on the flight hotspot region sequence through a trained bidirectional long short-term memory network to obtain a first prediction result of the flight hotspot region; extracting a multivariate time series that changes with time using the spatiotemporal knowledge graph, and predicting the flight hotspot region based on the multivariate time series through a trained sequence model based on an attention mechanism to obtain a second prediction result of the flight hotspot region.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and big data technology, and in particular to a spatiotemporal data prediction method, an electronic device and a storage medium. Background Art

[0002] With the rapid development of drones, civil airliners, airships, special aircraft and other aircraft, the processing of track big data generated by aircraft has become increasingly urgent and important. The mining and utilization of track big data has great social value, economic value, research significance and safety significance for space management, resource allocation and safe dispatch. Compared with the trajectory data generated by ground movement of people, cars and other ground movements or sea movement of ships and other sea movements, in addition to the increase in dimension, the big data of aircraft track also has the characteristics of large noise, fragmented and incomplete data, and large interference from clouds, etc., strong temporal and spatial randomness without constraints such as ground road networks and buildings, many types of target aircraft, large performance span, and the track is greatly affected by human factors, which makes the mining and prediction of track big data rules extremely difficult. Although the existing technical solutions have applied artificial intelligence technology, there are still problems such as inaccurate prediction, difficulty in implementation and low prediction efficiency. Summary of the invention

[0003] In view of the above problems, the present invention provides a spatiotemporal data prediction method, an electronic device and a storage medium, in order to solve at least one of the above technical problems.

[0004] According to a first aspect of the present invention, there is provided a method for predicting spatiotemporal data, comprising:

[0005] Performing three-dimensional grid processing on the flight space where the aircraft is located, marking and coding the obtained three-dimensional space grid, and performing one-dimensional grid processing on the flight time series of the aircraft, marking and coding the obtained time grid;

[0006] Construct a spatiotemporal knowledge graph of multi-dimensional relational data based on the labeled and encoded flight 3D space grid and the labeled and encoded flight time grid;

[0007] The spatiotemporal knowledge graph is used to extract the flight hotspot region sequence that changes with time. Based on the flight hotspot region sequence, the flight hotspot region is predicted through the trained bidirectional long short-term memory network to obtain the first prediction result of the flight hotspot region.

[0008] The spatiotemporal knowledge graph is used to extract multivariate time series that change over time, and based on the multivariate time series, the flight hotspot area is predicted through the trained sequence model based on the attention mechanism to obtain the second prediction result of the flight hotspot area.

[0009] According to an embodiment of the present invention, the three-dimensional gridding of the flight space where the aircraft is located and the marking and coding of the obtained three-dimensional space grid include:

[0010] Convert the flight space where the aircraft is located into a two-dimensional plane composed of longitude and latitude and a one-dimensional space composed of altitude;

[0011] The two-dimensional plane is converted into a Cartesian coordinate system with east longitude as the X-axis, north latitude as the Y-axis and a predefined origin;

[0012] The Cartesian coordinate system is gridded based on a preset latitude and longitude resolution, and each obtained two-dimensional plane grid is marked and encoded;

[0013] The height is gridded based on a preset height resolution, and each height component grid obtained is marked and encoded.

[0014] According to an embodiment of the present invention, the one-dimensional gridding of the flight time series of the aircraft and the marking and encoding of the obtained time grids include:

[0015] Based on the preset time granularity, the flight time series of the aircraft is divided into multiple time grids, and each time grid is marked and encoded.

[0016] According to an embodiment of the present invention, the spatiotemporal knowledge graph of multi-dimensional relational data constructed based on the labeled coded flight three-dimensional space grid and the labeled coded flight time grid includes:

[0017] Based on the marked and coded flight three-dimensional space grid and the marked and coded flight time grid, multi-dimensional track data of the aircraft is obtained, and the multi-dimensional track data is numbered in batches;

[0018] Determine the entities used to construct the knowledge graph based on the numbered track data, determine the multi-dimensional association relationships between the entities, and use predefined statistical methods to determine the attribute values ​​of the multi-dimensional association relationships;

[0019] A spatiotemporal knowledge graph of multidimensional relational data is constructed based on entity-relationship-entity, entity attribute values, and attribute values ​​of multidimensional association relationships.

[0020] According to an embodiment of the present invention, the multi-dimensional track data includes flight date information, flight time domain information, flight area information, flight trajectory information, aircraft type information, flight speed information and flight altitude information of the aircraft.

[0021] According to an embodiment of the present invention, the above-mentioned multi-dimensional association relationships include the association relationship between flight date entities, the association relationship between flight date entities and flight trajectories, the association relationship between flight date entities and flight time domain entities, the association relationship between flight time domain entities and flight area entities, the association relationship between flight area entities and flight speed entities, the association relationship between flight area entities and flight altitude entities, the association relationship between flight area entities and aircraft type entities, the association relationship between aircraft type entities and flight speed entities, the association relationship between aircraft type entities and flight altitude entities, and the association relationship between aircraft type entities and flight trajectory entities.

[0022] According to an embodiment of the present invention, the above-mentioned use of the spatiotemporal knowledge graph to extract a sequence of flight hotspot areas that changes over time, and based on the sequence of flight hotspot areas, predicting the flight hotspot areas through a trained bidirectional long short-term memory network, obtains a first prediction result of the flight hotspot area, including:

[0023] The spatiotemporal knowledge graph is used to extract the flight hotspot region sequence that changes over time, and the flight hotspot region sequence is preprocessed through data cleaning and normalization operations;

[0024] The preprocessed flight hotspot region sequence is reconstructed using the trained bidirectional long short-term memory network, and the reconstructed flight hotspot region sequence is forward predicted and reversely predicted;

[0025] The forward prediction result and the reverse prediction result are concatenated, and the concatenated result is classified using the fully connected layer of the trained bidirectional long short-term memory network to obtain the first prediction result of the flight hotspot area.

[0026] According to an embodiment of the present invention, the above-mentioned multivariate time series that changes with time is extracted by using the spatiotemporal knowledge graph, and based on the multivariate time series, the flight hotspot area is predicted by the trained sequence model based on the attention mechanism, and the second prediction result of the flight hotspot area includes:

[0027] Extract entities and entity attribute values ​​that change over time from the spatiotemporal knowledge graph, and encode the extracted entities and entity attribute values ​​into graph structured data;

[0028] The graph-structured data are arranged into multivariate time series in chronological order, and the multivariate time series are processed using the trained attention-based sequence model to obtain the second prediction result of the flight hotspot area.

[0029] A second aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0030] The third aspect of the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0031] The above-mentioned spatiotemporal data prediction method provided by the present invention can quickly and accurately extract flight hotspots and their associated multivariate sequences based on the flight spatiotemporal knowledge graph; and use a bidirectional long short-term memory network and a sequence model neural network based on an attention mechanism to predict single-variable and multivariate flight hotspot areas for multivariate sequences. The above-mentioned spatiotemporal data prediction method provided by the present invention can well solve the problems of large-scale spatiotemporal data being difficult to represent, difficult to mine and process rules, and difficult to predict. The above-mentioned spatiotemporal data prediction method provided by the present invention has the characteristics of clear logic, simple structure, fast query and reasoning speed, high accuracy of running results, and easy engineering implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0033] Figure 1 is a flow chart of a spatiotemporal data prediction method according to an embodiment of the present invention;

[0034] Figure 2 It is an architecture diagram of spatiotemporal trajectory data prediction based on spatiotemporal knowledge graph, bidirectional long short-term memory network and Transformer according to an embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of two-dimensional gridding and label coding of the longitude and latitude plane of the track space according to an embodiment of the present invention;

[0036] Figure 4 is a schematic diagram of one-dimensional gridding and label coding of track space altitude components according to an embodiment of the present invention;

[0037] Figure 5 is a schematic diagram of a knowledge graph framework of spatiotemporal track big data according to an embodiment of the present invention;

[0038] Figure 6 is a schematic diagram of a flight hotspot prediction model architecture based on bidirectional LSTM according to an embodiment of the present invention;

[0039] Figure 7is a schematic diagram of a Transformer-based flight hotspot and associated multivariate prediction model architecture according to an embodiment of the present invention;

[0040] Figure 8 is a block diagram of an electronic device suitable for implementing a spatiotemporal data prediction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0042] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0043] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0044] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0045] In the technical solution of the present invention, the aircraft track data information involved (including but not limited to aircraft type information, altitude information, time domain information, area information, flight date information, etc., such as flight sensor information, etc.) and other data (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the relevant owners or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0046] Figure 1 is a flow chart of a spatiotemporal data prediction method according to an embodiment of the present invention.

[0047] like Figure 1 As shown, the above-mentioned spatiotemporal data prediction method includes operations S110 to S140.

[0048] In operation S110, the flight space where the aircraft is located is processed into three-dimensional grids, and the obtained three-dimensional space grids are marked and coded, and the flight time series of the aircraft is processed into one-dimensional grids, and the obtained time grids are marked and coded.

[0049] The flight space is processed into three-dimensional grids, and each three-dimensional grid is marked and encoded.

[0050] In operation S120, a spatiotemporal knowledge graph of multi-dimensional relational data is constructed based on the labeled and encoded flight three-dimensional space grid and the labeled and encoded flight time grid.

[0051] It should be noted that all aircraft-related data involved in the present invention have obtained the authorization of the data owner and are processed with the permission of the data owner. The relevant data processing process strictly complies with the provisions of relevant laws and regulations, and strict confidentiality measures are taken.

[0052] The multi-dimensional relationship data include but are not limited to aircraft type information, latitude and longitude information, speed information, altitude information, sensor information, etc. Those skilled in the art can reasonably expand the multi-dimensional relationship data according to actual needs.

[0053] In operation S130, a sequence of flight hotspot areas that changes with time is extracted using the spatiotemporal knowledge graph, and based on the sequence of flight hotspot areas, the flight hotspot areas are predicted using a trained bidirectional long short-term memory network to obtain a first prediction result of the flight hotspot areas.

[0054] In operation S140, the spatiotemporal knowledge graph is used to extract a multivariate time series that changes over time, and based on the multivariate time series, the flight hotspot area is predicted by a trained sequence model based on the attention mechanism to obtain a second prediction result of the flight hotspot area.

[0055] The above-mentioned multiple sequence variables include but are not limited to flight hotspot areas, speed, altitude, aircraft type, sensor parameters, etc. Those skilled in the art can reasonably expand the above-mentioned multiple sequence variables according to actual needs.

[0056] The above-mentioned trained sequence model based on the attention mechanism includes a Transformer model. Those skilled in the art may adopt other models according to actual needs.

[0057] The above-mentioned spatiotemporal data prediction method provided by the present invention can quickly and accurately extract flight hotspots and their associated multivariate sequences based on the flight spatiotemporal knowledge graph; and use a bidirectional long short-term memory network and a sequence model neural network based on an attention mechanism to predict single-variable and multivariate flight hotspot areas for multivariate sequences. The above-mentioned spatiotemporal data prediction method provided by the present invention can well solve the problems of large-scale spatiotemporal data being difficult to represent, difficult to mine and process rules, and difficult to predict. The above-mentioned spatiotemporal data prediction method provided by the present invention has the characteristics of clear logic, simple structure, fast query and reasoning speed, high accuracy of running results, and easy engineering implementation.

[0058] The above-mentioned spatiotemporal data prediction method provided by the present invention is further described in detail below through specific embodiments and drawings.

[0059] Figure 2 It is an architecture diagram of spatiotemporal trajectory data prediction based on spatiotemporal knowledge graph, bidirectional long short-term memory network and Transformer according to an embodiment of the present invention.

[0060] like Figure 2 As shown, the above-mentioned spatiotemporal trajectory data prediction mainly includes steps S10 to S70.

[0061] S10: The flight space is processed into three-dimensional grids, and each space grid is marked and encoded.

[0062] S20: Perform one-dimensional gridding on the flight time series and mark and encode each time grid.

[0063] Steps S10 and S20 correspond to the above-mentioned operation S110, and are mainly used to perform gridding processing on the space-time of the aircraft and mark and encode each grid.

[0064] S30: Construct a knowledge graph consisting of multi-dimensional relational data such as aircraft, longitude and latitude, speed, altitude, and sensors based on the label-coded spatiotemporal grid.

[0065] Step S30 corresponds to the above-mentioned operation S120, and its main purpose is to construct a spatiotemporal knowledge graph.

[0066] S40: Extract the flight hotspot region sequence that changes over time through the knowledge graph.

[0067] S50: Predict flight hotspots through bidirectional LSTM.

[0068] Steps S40 and S50 correspond to the above-mentioned operation S130, and mainly use a bidirectional long short-term memory network (LSTM: Long Short-Term Memory) to predict the flight hotspot area.

[0069] S60: Extract multivariate sequences such as flight hot spots, speed, altitude, aircraft type, sensor parameters, etc. that change over time through the knowledge graph.

[0070] S70: Use Transformer to predict flight hotspots and their associated multivariate sequences such as target type, flight speed, altitude, etc.

[0071] Steps S60 and S70 correspond to the above-mentioned operation S140, and mainly use Transformer to predict the flight hotspot area.

[0072] The present invention is based on the spatiotemporal knowledge graph and adopts two large artificial intelligence models to predict flight hotspots from different angles, greatly improving the accuracy of the prediction.

[0073] According to an embodiment of the present invention, the above-mentioned three-dimensional gridding of the flight space where the aircraft is located and marking and encoding the obtained three-dimensional space grid include: converting the flight space where the aircraft is located into a two-dimensional plane composed of longitude and latitude and a one-dimensional space composed of altitude; converting the two-dimensional plane into a Cartesian coordinate system with east longitude as the X-axis, north latitude as the Y-axis and a predefined origin; gridding the Cartesian coordinate system based on a preset longitude and latitude resolution, and marking and encoding each obtained two-dimensional plane grid; gridding the altitude based on a preset altitude resolution, and marking and encoding each obtained altitude component grid.

[0074] The above-mentioned spatiotemporal data gridding processing and marking encoding operation provided by the present invention are further described in detail below through specific embodiments and in combination with the accompanying drawings.

[0075] Figure 3 It is a schematic diagram of two-dimensional gridding and label coding of the longitude and latitude plane of the track space according to an embodiment of the present invention.

[0076] Assume that the entire aircraft flight space is W. Considering the habit of application, the present invention does not directly treat the entire W space as three-dimensional, but converts it into a two-dimensional plane S composed of longitude and latitude and a one-dimensional space H composed of altitude, that is, W=S+H. For the S plane space composed of longitude and latitude, the present invention proposes an efficient marking and encoding and decoding method, which is easy to implement by computer and fast calculation. The method is introduced in detail below.

[0077] like Figure 3As shown. A Cartesian coordinate system is established with east longitude as the X-axis and north latitude as the Y-axis, and the origin of the coordinate is set as , the longitude resolution is idx, and the dimensional resolution is idy. The Cartesian coordinate system is gridded according to (idx, idy). Each grid is marked with a general mark "Sxy", where "x" is represented by a two- or three-digit decimal number, representing the number of the grid in the longitude direction; "y" is represented by a two- or three-digit decimal number, representing the number of the grid in the latitude direction. The actual longitude and latitude represented by the grid marked by Sxy can be represented by the upper left corner coordinates and the lower right corner coordinates of the grid (left-top, right-down). The relationship between the mark Sxy and (left-top, right-down) can be calculated using the following formula (1):

[0078] (1),

[0079] For any track point , if the following inequality (2) is satisfied at the same time,

[0080] (2),

[0081] Then the point is Located in the marked within the grid.

[0082] for There is a fast algorithm for mapping to longitude and latitude as shown in (3):

[0083] (3),

[0084] In particular, when idx=idy=1, we have formula (4):

[0085] (4),

[0086] in is the starting longitude and latitude of the coordinate origin, and "x" and "y" are the values ​​of the marker number. For example, if , , then S0102 represents area.

[0087] In order to facilitate numerical calculations in deep learning, "Sxy" is directly encoded as the value "xy" (removing the leading padded "0" if any), such as "S0102" is directly encoded as the value 102, "S1503" is directly encoded as 1503, "S1234" is directly encoded as 1234, and so on.

[0088] Figure 4 Schematic diagram of one-dimensional gridding and label coding of track space altitude components according to an embodiment of the present invention.

[0089] like Figure 4 As shown, the marking and encoding of the highly structured one-dimensional space H is relatively simple. , n is a positive integer, let idh represent the height resolution of equal division, then we have formula (5):

[0090] (5),

[0091] in, Indicates the interval where the i-th one-dimensional grid height is located.

[0092] Similarly, for any height component h of a track point, if the following inequality (6) is satisfied,

[0093] (6),

[0094] The track point is located at the within the grid.

[0095] In order to facilitate numerical calculations in deep learning, the " is directly encoded as the value "i", such as " " is directly encoded as the value 0," " is directly encoded as 1," ” is directly encoded as 100, and so on.

[0096] According to an embodiment of the present invention, the above-mentioned one-dimensional gridding of the flight time series of the aircraft and marking and encoding the obtained time grids include: dividing the flight time series of the aircraft based on a preset time granularity to obtain multiple time grids, and marking and encoding each time grid.

[0097] The operations of gridding and marking encoding the height dimension provided by the present invention are further described in detail below through specific embodiments.

[0098] Time series prediction based on knowledge graph can be based on years, days, hours, minutes, etc. Therefore, the processing of time is also critical, and the granularity should not be too fine or too coarse. In the present invention, time is divided into four stages per day: early morning, morning, afternoon and evening, and the corresponding time periods are shown in Table (1).

[0099] Table 1 Time period table

[0100]

[0101] In order to further refine the entire time series, the present invention divides 24 hours into 24 intervals by hour and encodes them in sequence as , corresponding to 0:00-1:00, 1:00-2:00, 2:00-3:00, ┄, 23:00-24:00 respectively.

[0102] According to an embodiment of the present invention, the above-mentioned spatiotemporal knowledge graph for constructing multidimensional relational data based on the labeled encoded three-dimensional flight space grid and the labeled encoded flight time grid includes: obtaining multidimensional track data of the aircraft based on the labeled encoded three-dimensional flight space grid and the labeled encoded flight time grid, and numbering the multidimensional track data in batches; determining the entities used to construct the knowledge graph based on the numbered track data, and determining the multidimensional association relationships between the entities, and determining the attribute values ​​of the multidimensional association relationships using a predefined statistical method; constructing a spatiotemporal knowledge graph of multidimensional relational data based on entity-relationship-entity, entity attribute values, and attribute values ​​of the multidimensional association relationships.

[0103] According to an embodiment of the present invention, the multi-dimensional track data includes flight date information, flight time domain information, flight area information, flight trajectory information, aircraft type information, flight speed information and flight altitude information of the aircraft.

[0104] According to an embodiment of the present invention, the above-mentioned multi-dimensional association relationships include the association relationship between flight date entities, the association relationship between flight date entities and flight trajectories, the association relationship between flight date entities and flight time domain entities, the association relationship between flight time domain entities and flight area entities, the association relationship between flight area entities and flight speed entities, the association relationship between flight area entities and flight altitude entities, the association relationship between flight area entities and aircraft type entities, the association relationship between aircraft type entities and flight speed entities, the association relationship between aircraft type entities and flight altitude entities, and the association relationship between aircraft type entities and flight trajectory entities.

[0105] The construction process of the spatiotemporal knowledge picture provided by the present invention is further described in detail below through specific embodiments and in combination with the accompanying drawings.

[0106] Figure 5 It is a schematic diagram of the spatiotemporal trajectory big data knowledge graph framework according to an embodiment of the present invention.

[0107] After gridding the space and time, the flight track can be processed. The basic structure of the flight track data includes: longitude and latitude, altitude, speed, category, sensor parameters, etc. Each track is numbered in batches.

[0108] The knowledge graph is a directed graph data structure composed of entities, relationships and attributes. It organizes and represents information at the semantic level, reveals the laws of development and evolution of things, and is widely used in semantic search, machine question answering, information retrieval, e-commerce, intelligent recommendation and other fields. The present invention uses the powerful information representation ability of the knowledge graph to construct a knowledge graph based on a large number of different batches of historical track data, which can mine, analyze and predict the associations between data and hidden behavioral laws.

[0109] like Figure 5 As shown in the figure, it is a schematic diagram of a basic spatiotemporal track big data knowledge graph framework constructed by the present invention for common application scenarios. In this basic framework, entities are represented by E, relationships are represented by R, and parameters in curly brackets {} represent attributes. In this framework, entities include nine categories: year, month, day, time domain, region, track, aircraft, speed, and altitude, and their corresponding attributes are introduced as follows.

[0110] Year label: year{name, id}, where name is a string, such as "2025", and id is an integer, such as 2025; similar to year label: month label and day label; region label: region{name, leftTop, rightDown}, where name is a string, which is a gridded mark, leftTop is the latitude and longitude of the upper left corner of the grid, and rightDown is the latitude and longitude of the lower right corner of the grid. For example, a region entity is region{name: "S0133", leftTop: (127E, 39N), rightDown: (129E, 38N)}; time domain label: time domain{name, id}, where name is the time gridded label, which can be one of T0, T1, ┄, T23, id is an integer, such as 2025010100, indicating year, month, day and hour; track label: track{batch, number of points}, where batch indicates the number of a complete track, and number of points indicates how many points the track consists of in total; aircraft label: aircraft{name,id}, where name indicates the aircraft name, and id indicates the aircraft number; altitude label: altitude{name,id}, where name is the altitude encoding value, such as "H10", and id is a numerical value, such as 10; similar labels include speed labels.

[0111] The relationships include twelve basic relationships: year-month, month-day, track-day, day-area, day-time domain, time domain-area, area-speed, area-altitude, aircraft-area, flight-speed, aircraft-altitude, aircraft-track, etc. Their corresponding attributes are introduced as follows.

[0112] Year-month relationship: month{name, id}, where name represents a certain year and month, and the type is a string, such as "202501", and the id type is an integer, such as 202501; similar relationships include month-day relationship, from month to day; track-day relationship: time{aircraft}, where the aircraft can be represented by the aircraft name or number; day-area relationship: occurrence{batch, point, time}, where batch represents the track number passing through the area on that day, point represents the number of track points belonging to the area on that day, and time represents the date of that day; day-time domain relationship: time domain{batch, point, aircraft}; time domain-area relationship: occurrence{batch, point, time, aircraft}; area-speed relationship: speed{batch, point, time, aircraft}; area-altitude relationship: altitude{batch, point, time, aircraft}; aircraft-track relationship: track {time}; aircraft-area relationship: through {batch, point, duration}; aircraft-speed relationship: speed {batch, point, time}; aircraft-altitude relationship: altitude {batch, point, time}; in these relationships, batch, point, aircraft, time, and duration have the same meanings as in the other relationships mentioned above, except that the calculation of their attribute values ​​must be carried out under the specific constraints of the entities connected at both ends, and the specific values ​​of their attributes are determined through data statistical calculations.

[0113] According to an embodiment of the present invention, the above-mentioned use of the spatiotemporal knowledge graph to extract a sequence of flight hotspot areas that changes with time, and based on the flight hotspot area sequence, predicting the flight hotspot area through a trained bidirectional long short-term memory network to obtain a first prediction result of the flight hotspot area includes: using the spatiotemporal knowledge graph to extract a sequence of flight hotspot areas that changes with time, and preprocessing the flight hotspot area sequence through data cleaning operations and normalization operations; using the trained bidirectional long short-term memory network to reconstruct the preprocessed flight hotspot area sequence, and forward predicting and reverse predicting the reconstructed flight hotspot area sequence; splicing the forward prediction result with the reverse prediction result, and using the fully connected layer of the trained bidirectional long short-term memory network to classify the splicing result, to obtain a first prediction result of the flight hotspot area.

[0114] The process of using LSTM to predict flight hotspot areas provided by the present invention is further described in detail below through specific embodiments and in combination with the accompanying drawings.

[0115] Figure 6 4 is a schematic diagram of the architecture of a flight hotspot prediction model based on bidirectional LSTM according to an embodiment of the present invention.

[0116] The construction of the knowledge graph lays the foundation for knowledge graph-based applications, the two most core applications of which are the prediction of flight hotspots and the prediction of related associated parameters around the hotspots.

[0117] First, the knowledge graph can quickly generate hot spots generated on a daily or hourly basis. The so-called "hot spot" refers to the area with the largest number of points within a specific time period. There can be one or more hot spots, such as the top ten areas. By sorting the data over the years according to the hot spots generated daily or hourly, a single-variable or multi-variable (multiple hot spots) time series of hot spots is formed. Let this sequence be:

[0118] , where m is the number of sequences, and x is the value after encoding the hot spot area; if it is a multi-variable sequence, then , where X is a matrix composed of multiple hot spots, where k is the number of multi-variables, is composed of m-dimensional vectors. To obtain good training results, it is usually necessary to clean and normalize the input data X. If necessary, the law and trend of the time series can be further strengthened by resampling to further improve the prediction accuracy. Thus, the generation of flight hot spots is completed, corresponding to step S40.

[0119] Secondly, design a bidirectional long short-term memory network LSTM (Bi-LSTM) model and train the model. As Figure 6 shown, Bi-LSTM is formed by combining forward and reverse LSTMs. The forward LSTM can obtain the past information of the input sequence, and the reverse LSTM can obtain the future information of the input sequence. The bidirectional LSTM further improves the globality and integrity of feature extraction. Let the prediction step (or time window) be n (n < m), and the input sequence is reconstructed into m - n n-dimensional vectors: , , ……, ; at the same time, reverse these vectors to obtain the corresponding reverse order arrangement, such as: , , ……, ; input the input sequence into the forward LSTM and reverse LSTM respectively, splice or add the output results, and finally output the results to the fully connected layer to obtain the final prediction result. In the training stage, the corresponding output Y, that is, the value to be predicted, is respectively: . The training loss function can choose mean squared error (MSE) or cross entropy, and the number of hidden layers can be reasonably selected according to the actual data volume.

[0120] Finally, in the model prediction and inference stage, it can be divided into single-step prediction and multi-step prediction, that is, infer the hot spot area tomorrow or the hot spot areas in the future from the hot spot area data of today and the previous n - 1 days. The same applies to the hot spots generated hourly. In single-step prediction, assuming that it is necessary to predict the flight hot spot area on the i-th day, the hot spot area data for the previous n days is , in reverse order , input into Bi-LSTM, and obtain the flight hotspot area on the i-th day In multi-step prediction, first use single-step prediction to get the prediction value of the first day, then use the prediction value of the first day together with other true values ​​as input, and continue to use single-step prediction to get the prediction value of the second day, and so on, to get the prediction of hot spots for multiple days in the future. This method is simple and feasible, but the prediction error will accumulate, resulting in greater prediction errors in the later prediction days. Usually, a reasonable number of multi-step prediction steps is selected according to the model operation effect.

[0121] According to an embodiment of the present invention, the above-mentioned use of the spatiotemporal knowledge graph to extract a multivariate time series that changes over time, and based on the multivariate time series, predict the flight hotspot area through a trained sequence model based on the attention mechanism to obtain a second prediction result of the flight hotspot area, including: extracting entities and entity attribute values ​​that change over time from the spatiotemporal knowledge graph, encoding the extracted entities and entity attribute values ​​into graph structured data; arranging the graph structured data into a multivariate time series in chronological order, and using the trained sequence model based on the attention mechanism to process the multivariate time series to obtain a second prediction result of the flight hotspot area.

[0122] The process of using the Transformer model to predict flight hotspot areas provided by the present invention is further described in detail below through specific embodiments and in conjunction with the accompanying drawings.

[0123] Figure 7 2 is a schematic diagram of a Transformer-based flight hotspot and associated multivariate prediction model architecture according to an embodiment of the present invention.

[0124] Through the knowledge graph, the prediction of related parameters of the hot core, such as speed, altitude, aircraft type, sensor detection parameters and other multivariate predictions are also the focus of attention. For example, one of the usage scenarios: one day in the future, over a certain area, it is very likely that a certain aircraft will pass by at a certain altitude and speed, and it can be detected using certain sensor parameters.

[0125] There is no essential difference between aircraft multivariate sequence prediction and univariate prediction. The core problem lies in the structured processing and management of multi-source heterogeneous sequence data. Based on the knowledge graph, the present invention represents the longitude and latitude, altitude, speed, aircraft category, sensor parameters and time, etc., through encoding and decoding, into an efficient and computable graph structured data structure, which is easy to apply. , where X is a multidimensional matrix composed of time, hotspot, altitude, speed, aircraft type, sensor parameters, etc., and k is the number of multivariate variables.

[0126] , where m is the dimension of each component, and this sequence is arranged in chronological order to form a multivariate time series, corresponding to step S60.

[0127] As Figure 7 shown. By introducing the attention mechanism, Transformer can efficiently calculate and accurately learn long-range correlation relationships. Its core lies in its encoder and decoder structures, where the encoder is responsible for converting the input sequence into an intermediate representation, and the decoder uses these representations to generate the output sequence. In time series analysis, the Transformer model also shows strong advantages, being able to effectively handle long-range dependencies in time series data and achieving good performance in tasks such as prediction and classification. The present invention applies the Transformer model to the processing of spatio-temporal data, especially the prediction of multivariate relationships around flight hotspots. In this model, the parameters of the encoder, decoder, linear layer, and position encoder are all learned from the data, and the difficulty lies in the organization of the input and output data. The present invention solves this well through the knowledge graph. In the training stage, assuming the prediction step size is n (n < m), for the j-th (j < k) component of the input of k multivariate sequences, this input sequence can be reconstructed into m - n n-dimensional vectors: , , ……, , where the i-th vector is: , and k such vectors are formed into a batch and input into the input end of the encoder; for simplicity, the decoder is kept equal to the encoder input; there is a position offset at the output end of the decoder relative to the input end of the decoder. For the j-th component, its output is , then it can be seen that the last value of the output sequence is the required predicted value. Similar to the Bi-LSTM prediction of flight hotspots, the time series prediction based on Transformer can also be single-step or multi-step, and the prediction process based on bidirectional LSTM can be referred to.

[0128] In summary, this embodiment introduces in detail a method for predicting spatiotemporal trajectory data based on spatiotemporal knowledge graph, bidirectional LSTM and Transformer, especially for the prediction of flight hotspots and speed, altitude, aircraft type, sensor parameters, etc. related to hotspots. The main steps of the method include: three-dimensional grid processing of flight space, one-dimensional grid processing of flight time, construction of knowledge graph, extraction of flight hotspot area sequence based on knowledge graph and prediction of flight hotspot area using bidirectional LSTM, extraction of flight hotspot area, speed, altitude, aircraft type, sensor parameters and other multivariate sequences based on knowledge graph and prediction of multivariate sequences using Transformer, etc. The core of the method is to propose a relatively complete spatiotemporal data encoding and decoding method, based on which the spatiotemporal knowledge graph can be conveniently constructed, so that flight hotspots and their associated multivariate sequences can be quickly and accurately extracted; based on the sequence, Bi-LSTM or Transformer is used for prediction, and the difficulty lies in the management and arrangement of time series input and output data. In the training and reasoning process, the present invention proposes a simple and feasible method. Practice has proved that the spatiotemporal trajectory data prediction method based on spatiotemporal knowledge graph, bidirectional LSTM and Transformer proposed in the present invention can effectively solve the problems of difficult representation, regularity mining, processing and prediction of large-scale spatiotemporal data. It has the characteristics of clear logic, simple structure, fast query and reasoning speed, high operating result accuracy and easy engineering implementation.

[0129] Figure 8 A block diagram of an electronic device suitable for implementing a spatiotemporal data prediction method according to an embodiment of the present invention is schematically shown.

[0130] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0131] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0132] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.

[0133] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0134] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0135] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0137] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for predicting spatiotemporal data, characterized in that: The method comprises: Performing three-dimensional grid processing on the flight space where the aircraft is located, marking and coding the obtained three-dimensional space grid, and performing one-dimensional grid processing on the flight time series of the aircraft, marking and coding the obtained time grid; Constructing a spatiotemporal knowledge graph of multi-dimensional relational data based on the labeled and encoded flight three-dimensional space grid and the labeled and encoded flight time grid, wherein the entities of the spatiotemporal knowledge graph include year, month, day, time domain, region, track, aircraft, speed and altitude; The flight hotspot region sequence that changes with time is extracted by using the spatiotemporal knowledge graph, the flight hotspot region sequence is resampled, and based on the resampled flight hotspot region sequence, the flight hotspot region is predicted by a trained bidirectional long short-term memory network to obtain a first prediction result of the flight hotspot region, wherein the flight hotspot region is the region with the most track points in a specific time period; The flight hotspot area is predicted by the trained bidirectional long short-term memory network, and the first prediction result of the flight hotspot area is divided into single-step prediction and multi-step prediction; The spatiotemporal knowledge graph is used to extract a multivariate time series that changes over time, and based on the multivariate time series, the flight hotspot area is predicted by a trained sequence model based on an attention mechanism to obtain a second prediction result of the flight hotspot area, wherein the trained sequence model based on the attention mechanism includes a Transformer model, and the second prediction result includes the speed, altitude, type, and sensor detection parameters of the aircraft; The flight hotspot area is predicted by the trained sequence model based on the attention mechanism, and the second prediction result of the flight hotspot area is divided into single-step prediction and multi-step prediction; The three-dimensional gridding of the flight space where the aircraft is located and the marking and coding of the obtained three-dimensional space grid include: Converting the flight space where the aircraft is located into a two-dimensional plane consisting of longitude and latitude and a one-dimensional space consisting of altitude; The two-dimensional plane is converted into a Cartesian coordinate system with east longitude as the X-axis, north latitude as the Y-axis and a predefined origin; Gridding the Cartesian coordinate system based on a preset latitude and longitude resolution, and marking and encoding each obtained two-dimensional plane grid; Gridding the height based on a preset height resolution, and marking and encoding each height component grid obtained; Wherein, each of the two-dimensional plane grids is labeled and encoded as in formula (1): （1）， in, represents the coordinate origin of the Cartesian coordinate system, idx represents the longitude resolution, idy represents the latitude resolution, x represents the number of each of the two-dimensional plane grids in the longitude direction, and y represents the number of each of the two-dimensional plane grids in the latitude direction; Wherein, each height component grid is marked and encoded as in formula (5): （5）， in, It represents the interval where the i-th one-dimensional grid height is located, and idh represents the equally divided height resolution.

2. The method according to claim 1, characterized in that The flight time series of the aircraft is processed into a one-dimensional grid, and the obtained time grid is marked and encoded, including: Based on a preset time granularity, the flight time sequence of the aircraft is divided to obtain a plurality of time grids, and each of the time grids is marked and coded.

3. The method according to claim 1, characterized in that The spatiotemporal knowledge graph of multi-dimensional relational data based on the labeled coded flight 3D space grid and the labeled coded flight time grid includes: Based on the marked and coded flight three-dimensional space grid and the marked and coded flight time grid, multi-dimensional track data of the aircraft is acquired, and the multi-dimensional track data is numbered in batches; Determine the entities used to construct the knowledge graph according to the numbered track data, determine the multi-dimensional association relationships between the entities, and use a predefined statistical method to determine the attribute values ​​of the multi-dimensional association relationships; A spatiotemporal knowledge graph of the multi-dimensional relational data is constructed based on entity-relationship-entity, the entity attribute values, and the attribute values ​​of the multi-dimensional association relationships.

4. The method according to claim 3, characterized in that The multi-dimensional track data includes the flight date information, flight time domain information, flight area information, flight trajectory information, aircraft type information, flight speed information and flight altitude information of the aircraft.

5. The method according to claim 3, characterized in that: The multi-dimensional association relationship includes the association relationship between flight date entities, the association relationship between the flight date entity and the flight trajectory, the association relationship between the flight date entity and the flight time domain entity, the association relationship between the flight time domain entity and the flight area entity, the association relationship between the flight area entity and the flight speed entity, the association relationship between the flight area entity and the flight altitude entity, the association relationship between the flight area entity and the aircraft type entity, the association relationship between the aircraft type entity and the flight speed entity, the association relationship between the aircraft type entity and the flight altitude entity, and the association relationship between the aircraft type entity and the flight trajectory entity.

6. The method according to claim 1, characterized in that The flight hotspot region sequence that changes with time is extracted by using the spatiotemporal knowledge graph, and based on the flight hotspot region sequence, the flight hotspot region is predicted by a trained bidirectional long short-term memory network, and the first prediction result of the flight hotspot region is obtained, including: Extracting a sequence of flight hotspot regions that changes over time using the spatiotemporal knowledge graph, and preprocessing the sequence of flight hotspot regions through data cleaning and normalization operations; Reconstructing the preprocessed flight hotspot region sequence using the trained bidirectional long short-term memory network, and performing forward prediction and reverse prediction on the reconstructed flight hotspot region sequence; The forward prediction result and the reverse prediction result are spliced ​​together, and the spliced ​​result is classified using the fully connected layer of the trained bidirectional long short-term memory network to obtain a first prediction result of the flight hotspot area.

7. The method according to claim 1, characterized in that The multivariate time series that changes with time is extracted by using the spatiotemporal knowledge graph, and based on the multivariate time series, the flight hotspot area is predicted by using a trained sequence model based on the attention mechanism, and the second prediction result of the flight hotspot area is obtained, including: Extracting entities and entity attribute values ​​that change over time from the spatiotemporal knowledge graph, and encoding the extracted entities and entity attribute values ​​into graph structured data; The graph structured data is arranged in chronological order into the multivariate time series, and the multivariate time series is processed using the trained sequence model based on the attention mechanism to obtain a second prediction result of the flight hotspot area.

8. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program or instruction is stored thereon, wherein when the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images