Airport operation situation dynamic prediction method, device, equipment, storage medium and program product

Abstract

The application discloses an airport operation situation dynamic prediction method and device, equipment, a storage medium and a program product, and relates to the technical field of airport management. The airport operation situation dynamic prediction method comprises the following steps: acquiring multi-source heterogeneous data of airport operation; performing multi-modal fusion based on the multi-source heterogeneous data to obtain multi-modal space-time data; inputting the multi-modal space-time data into a pre-trained airport operation situation space-time model to perform situation dynamic prediction, and obtaining a situation prediction result. Since the multi-modal fusion is performed based on the multi-source heterogeneous data, the complementarity and synergy between the data can be mined, so that all-round dynamic prediction of the airport operation situation can be realized. The situation dynamic prediction is performed through the pre-trained airport operation situation space-time model, the instantaneity of the processing of the abnormal time is improved, and the application scenarios are increased.

Description

Technical Field

[0001] This application relates to the field of airport management technology, and in particular to methods, devices, equipment, storage media and program products for dynamic prediction of airport operation status. Background Technology

[0002] As a core spatial node in air transport, the operational efficiency, safety, and service level of airports directly impact regional economic development and passenger travel experience. Predicting airport operational trends can provide decision-making support for airport operations.

[0003] Currently, most airport situation forecasting is based on traffic flow. This approach may lead to imbalances in resource allocation and insufficient capacity to respond to unusual events. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment, storage medium and program product for dynamic prediction of airport operation status, which aims to solve the technical problems that dynamic prediction of airport operation status can easily lead to imbalance in resource allocation and insufficient ability to respond to abnormal events.

[0005] To achieve the above objectives, this application proposes a dynamic prediction method for airport operational status, which includes:

[0006] Acquire multi-source heterogeneous data on airport operations;

[0007] Multimodal fusion is performed based on the aforementioned multi-source heterogeneous data to obtain multimodal spatiotemporal data;

[0008] The multimodal spatiotemporal data is input into a pre-trained spatiotemporal model of airport operation status for dynamic situation prediction, and the situation prediction results are obtained.

[0009] In one embodiment, the step of performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal data features includes:

[0010] Obtain the temporal and spatial information of the multi-source heterogeneous data;

[0011] Based on the time information and the spatial information, the multi-source heterogeneous data is spatiotemporally aligned to obtain the aligned multi-source heterogeneous data.

[0012] Multimodal feature extraction is performed on the aligned multi-source heterogeneous data to obtain multimodal data features;

[0013] An adaptive spatiotemporal hotspot map is obtained by fusing and constructing the multimodal data features.

[0014] The multimodal spatiotemporal data of airport operations are determined based on the adaptive spatiotemporal heat map.

[0015] In one embodiment, the step of fusing and constructing an adaptive spatiotemporal heatmap based on the multimodal data features includes:

[0016] Obtain airport layout data;

[0017] An initial association diagram of the airport is constructed based on the layout data;

[0018] Construct an association adjacency matrix based on the relationships between spatial nodes in the initial association graph;

[0019] The multimodal features are fused in the time dimension based on the association adjacency matrix to obtain an adaptive spatiotemporal hotspot map.

[0020] In one embodiment, before the step of inputting the multimodal spatiotemporal data into a pre-trained airport operational situation spatiotemporal model for dynamic situation prediction and obtaining the prediction result, the method further includes:

[0021] Acquire training multimodal spatiotemporal data and training spatiotemporal heatmap;

[0022] The labeled training dataset is obtained by labeling the training spatiotemporal heatmap and the training multimodal spatiotemporal data.

[0023] The labeled training dataset is input into the initial airport operation status spatiotemporal model for training to obtain the airport operation status spatiotemporal model.

[0024] In one embodiment, the initial airport operational status spatiotemporal model includes: a feature processing layer, a sharing layer, a private layer, and an output layer;

[0025] The feature processing layer is used to obtain the labeled training data in the labeled training dataset and determine the private channel corresponding to the labeled training data. The labeled training data includes the labeled training spatiotemporal heatmap and the labeled training multimodal spatiotemporal data.

[0026] The shared layer is used to extract shared features based on the training spatiotemporal heatmap;

[0027] The private layer includes several private channels;

[0028] The private layer is used to process the corresponding labeled training data based on the private channel to obtain private channel features;

[0029] The output layer is used to output the prediction result of the labeled training data based on the shared features and the private channel features.

[0030] In one embodiment, the step of inputting the labeled training dataset into the initial airport operational status spatiotemporal model for training to obtain the airport operational status spatiotemporal model includes:

[0031] The labeled training data in the labeled training dataset is input into the initial airport operational status spatiotemporal model for prediction, and the prediction result is obtained.

[0032] The prediction results are evaluated for loss based on a preset multi-task loss function to obtain loss parameters.

[0033] Based on the loss parameters, the initial airport operational status spatiotemporal model is iteratively optimized to obtain the iteratively optimized initial airport status spatiotemporal model.

[0034] When the iteratively optimized initial airport situation spatiotemporal model meets the preset iteration conditions, the iteratively optimized initial airport situation spatiotemporal model is used as the airport operation situation spatiotemporal model.

[0035] Furthermore, to achieve the above objectives, this application also proposes an airport operation status dynamic prediction device, which includes:

[0036] The data acquisition module is used to acquire multi-source heterogeneous data on airport operations;

[0037] The data fusion module is used to perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data;

[0038] The situation prediction module is used to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction and to obtain the situation prediction result.

[0039] In addition, to achieve the above objectives, this application also proposes an airport operation status dynamic prediction device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the airport operation status dynamic prediction method as described above.

[0040] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the airport operation status dynamic prediction method described above.

[0041] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the airport operation status dynamic prediction method described above.

[0042] One or more technical solutions proposed in this application have at least the following technical effects:

[0043] This application acquires multi-source heterogeneous data on airport operations; performs multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; and inputs the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction, obtaining the situation prediction results. Because it involves multimodal fusion based on multi-source heterogeneous data, it can uncover the complementarity and synergy between data points, thereby achieving comprehensive dynamic prediction of airport operation situations. Furthermore, using a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction improves the immediacy of handling anomalies and expands the application scenarios. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating the first embodiment of the airport operation status dynamic prediction method of this application;

[0047] Figure 2 A schematic diagram of the adaptive spatiotemporal hotspot map generation model provided in Embodiment 2 of the airport operation dynamic prediction method of this application;

[0048] Figure 3 This is a schematic diagram of the spatiotemporal model of airport operation status provided in Embodiment 3 of the airport operation status dynamic prediction method of this application;

[0049] Figure 4 This is a schematic diagram of the module structure of the airport operation status dynamic prediction device according to an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the dynamic prediction method of airport operation status in this application embodiment.

[0051] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0053] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The main solution of this application embodiment is: to acquire multi-source heterogeneous data of airport operation; to perform multimodal fusion based on multi-source heterogeneous data to obtain multimodal spatiotemporal data; to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction and obtain situation prediction results.

[0055] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a computer or server, or an electronic device or virtual device capable of performing the above functions. The following description uses an airport operation status dynamic prediction device (hereinafter referred to as the prediction device) as an example to illustrate this embodiment and the subsequent embodiments.

[0056] Based on this, the embodiments of this application provide a method for dynamic prediction of airport operational status, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the airport operation status dynamic prediction method of this application.

[0057] In this embodiment, the airport operation status dynamic prediction method includes steps S10~S30:

[0058] Step S10: Obtain multi-source heterogeneous data on airport operations;

[0059] It should be noted that the aforementioned multi-source heterogeneous data may include data sets from different sources and may have different structures, formats, or semantics. In the embodiments of this application, the aforementioned multi-source heterogeneous data may include airspace usage data, satellite meteorological data, flight schedule data, radar data, traffic flow data, video surveillance data, and data from IoT sensors, which can be used to form a real-time digital twin model of the overall airport operational status.

[0060] In this embodiment of the application, no specific method for acquiring multi-source heterogeneous data is restricted; the method can be selected according to the actual application.

[0061] Step S20: Perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data.

[0062] Understandably, by fusing multi-modal data from different sources, the complementarity and synergy between the data can be uncovered, thereby enabling comprehensive and dynamic prediction of airport operational status.

[0063] It should be noted that the multi-source heterogeneous data in this application embodiment may include relevant temporal and spatial information. The temporal information allows for the determination of the generation time of the multi-source heterogeneous data on a timeline, while the spatial information allows for the determination of the generation location of the multi-source heterogeneous data spatially. By fusing multi-source heterogeneous data in multiple modes, a unified representation of the data in terms of time, space, and multi-modal characteristics can be achieved, thereby improving the accuracy and robustness of airport operational status prediction.

[0064] In some embodiments of this application, the above-mentioned multimodal fusion method may be weighted fusion or fusion using a neural network, or other fusion methods. This application does not limit this method.

[0065] In some embodiments of this application, the multimodal fusion method may be: extracting spatial features from multi-source heterogeneous data using a convolutional neural network, extracting temporal features from multi-source heterogeneous data using a long short-term memory network, and extracting semantic vectors from multi-source heterogeneous data using a Transformer encoder. By setting independent encoders (such as GCN or CNN) for each source of multi-source heterogeneous data to encode the extracted features, corresponding spatial feature codes, temporal feature codes, and semantic vector codes are obtained, and then multimodal fusion is achieved through weighted summation of the feature codes.

[0066] It should be noted that the multimodal spatiotemporal data in this application embodiment is data that reflects airport flight dynamics, meteorological dynamics, and other information more comprehensively and accurately by fusing multi-source heterogeneous data in the time and spatial dimensions. By fusing multi-source data such as flight dynamics, meteorological data, command data, and traffic flow data, the bias of a single data source can be avoided, and the prediction accuracy and stability of the model can be improved.

[0067] Step S30: Input the multimodal spatiotemporal data into the pre-trained airport operation situation spatiotemporal model for dynamic situation prediction and obtain the situation prediction result.

[0068] It should be noted that the aforementioned airport operation status spatiotemporal model is a predictive model that can be used to dynamically predict based on shared and private features in input multimodal spatiotemporal data. Through this model, the airport operation status can be predicted from multiple dimensions, thereby achieving a more even distribution of airport scheduling resources. Furthermore, since this application is based on dynamic prediction using multi-source heterogeneous data, it improves the timeliness of handling abnormal events and expands its application scenarios.

[0069] This application embodiment acquires multi-source heterogeneous data on airport operations; performs multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; and inputs the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction, obtaining the situation prediction result. Because multimodal fusion is performed based on multi-source heterogeneous data, the complementarity and synergy between data can be explored, thereby achieving comprehensive dynamic prediction of airport operation situation; and using a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction improves the immediacy of handling anomalies and expands application scenarios.

[0070] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This application provides a schematic diagram of the adaptive spatiotemporal hotspot map generation model structure for Embodiment 2 of the airport operation status dynamic prediction method.

[0071] In this embodiment of the application, the step of performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal data features includes:

[0072] Step S21: Obtain the temporal and spatial information of the multi-source heterogeneous data;

[0073] Step S22: Based on the time information and the spatial information, perform spatiotemporal alignment on the multi-source heterogeneous data to obtain the aligned multi-source heterogeneous data.

[0074] It should be noted that when an event occurs at a certain spatial node (such as a flight about to arrive, a conveyor belt being used, etc.), the time and spatial information of the data generation can be recorded simultaneously. In this embodiment of the application, by unifying the data structure of the time and location information of multi-source heterogeneous data from different data sources, and aligning them on the time axis and spatial location, aligned multi-source heterogeneous data is obtained.

[0075] In some embodiments of this application, the time and location information of data collected from different sources may be expressed inconsistently. When these multi-source heterogeneous data are obtained, they can be standardized to unify them in the same time and spatial representation, such as the year-month-day representation, the XYZ coordinate axis representation, the latitude and longitude representation, etc. This application does not impose any limitations on this.

[0076] It is understandable that aligning multi-source heterogeneous data can unify the structure of this data in both time and space dimensions, thereby enabling the construction of adaptive spatiotemporal heatmaps to represent the changing states of each spatial node along the time axis. Based on these changing states, multi-dimensional airport situation prediction can be achieved, improving prediction accuracy.

[0077] It should be understood that the aforementioned spatial nodes are also entities within the airport that require situational prediction, such as conveyor belts, boarding gates, flights, shuttle buses, roads, etc., and this application embodiment does not impose any limitations on them. By using the spatial information of multi-source heterogeneous data, the spatial nodes corresponding to the multi-source heterogeneous data can be determined; by using the temporal information of the multi-source heterogeneous data, the time when the multi-source heterogeneous data was generated can be determined. Using both spatial and temporal information, the changing state of this type of multi-source heterogeneous data within that spatial node can be determined.

[0078] In a specific implementation, the prediction device of this application embodiment can acquire the temporal and spatial information of multi-source heterogeneous data, and perform spatiotemporal alignment of the multi-source heterogeneous data based on the temporal and spatial information, so that the temporal and spatial information can be represented in the same spatiotemporal dimension, and thus obtain the aligned multi-source heterogeneous data.

[0079] Step S23: Based on the aligned multi-source heterogeneous data, perform multimodal feature extraction to obtain multimodal data features;

[0080] Step S24: Based on the multimodal data features, an adaptive spatiotemporal hotspot map is obtained by fusing and constructing the data.

[0081] Step S25: Determine the multimodal spatiotemporal data of airport operation based on the adaptive spatiotemporal heat map.

[0082] It should be noted that the above-mentioned multimodal feature extraction process can include temporal feature extraction, spatial feature extraction, and semantic vector extraction. That is, the extracted multimodal data features can include event features, spatial features, and semantic vector features. Through temporal feature extraction, the periodicity and trend characteristics of multi-source heterogeneous data within a spatial node can be determined. Through spatial feature extraction, the regional relationships and adjacency matrices of each spatial node can be determined. Through semantic vector extraction, the implicit associations and event descriptions within each spatial node can be determined.

[0083] It should be explained that by fusing the above multimodal data features, an adaptive spatiotemporal hotspot map can be obtained to represent the data changes in each spatial node, thereby determining the multimodal spatiotemporal data of airport operations.

[0084] In some embodiments of this application, the step of fusing and constructing an adaptive spatiotemporal heatmap based on the multimodal data features includes: acquiring airport layout data; constructing an initial association graph of the airport based on the layout data; constructing an association adjacency matrix based on the relationships between spatial nodes in the initial association graph; and fusing the multimodal features in the time dimension based on the association adjacency matrix to obtain the adaptive spatiotemporal heatmap.

[0085] It should be noted that the aforementioned airport layout data may include the airport's physical layout data, specifically including airport runways and their lengths, apron size and location, terminal location and structure, etc. By processing the layout data using the static initial association graph generation module, an initial association graph of the airport can be constructed. This initial association graph allows for the display of the airport's static physical layout, thereby enabling dynamic spatiotemporal association of multimodal data features. In this embodiment, the specific method of constructing the initial association graph based on the layout data is not limited; it may include digital twin model construction, building information modeling, etc. This embodiment does not impose any limitations on this method.

[0086] Specifically, in the embodiments of this application, reference is made to Figure 2 The adaptive spatiotemporal hotspot graph generation model of this application includes an input layer, a static initial correlation graph generation module, a dynamic fusion correlation graph generation module, and an output layer. The dynamic fusion correlation graph generation module has a similar structure to the static initial correlation graph generation module.

[0087] Understandably, the static initial correlation graph generation module can include attention units, max pooling units, and several convolutional units. Each convolutional unit includes a convolutional sub-unit (capable of both graph convolution and temporal convolution) and a pruning sub-unit. Through the operations of multiple convolutional units, the output of each convolutional sub-unit is pruned by the pruning sub-unit to ensure consistent output. Then, the attention unit captures the cross-temporal and spatial correlations between features, improving the model's multimodal fusion capability. Finally, the max pooling unit obtains the most representative features.

[0088] It should be explained that the initial association graph may include several spatial nodes, each of which may correspond to a time axis. By processing the initial association graph using the dynamic fusion association graph generation module, an adaptive spatiotemporal heatmap can be obtained. Specifically, based on the attention mechanism and through semantic vectors in the multimodal data features, implicit associations between the spatial nodes and events occurring at each time point in the initial association graph can be realized. Simultaneously, by using the spatial and temporal features in the multimodal data features, data fusion with the spatial nodes and time axis in the initial association graph can be achieved, thereby constructing a dynamically associated adaptive spatiotemporal heatmap.

[0089] It should be noted that the matrix dimension of the adjacency matrix can be the number of spatial nodes multiplied by the number of spatial nodes, and the matrix elements in the adjacency matrix can be used to represent the strength of the relationship between spatial nodes. The weights in the adjacency matrix can be calculated based on the distance between spatial nodes in the matrix, or they can be calculated comprehensively based on factors such as flight frequency and event priority; this application does not impose any limitations on this.

[0090] It is understandable that the adaptive spatiotemporal heat map can be a two-dimensional or three-dimensional map, which can be used to represent the events that occur at different times in each spatial node of the airport and the implicit connections between the events. By making predictions based on the adaptive spatiotemporal heat map, the accuracy and comprehensiveness of situation prediction are improved.

[0091] This application embodiment acquires temporal and spatial information from multi-source heterogeneous data; performs spatiotemporal alignment of the multi-source heterogeneous data based on the temporal and spatial information to obtain aligned multi-source heterogeneous data; extracts multimodal features from the aligned multi-source heterogeneous data to obtain multimodal data features; fuses and constructs an adaptive spatiotemporal heatmap based on the multimodal data features; and determines the multimodal spatiotemporal data of airport operations based on the adaptive spatiotemporal heatmap. Because the alignment is based on the temporal and spatial information of multi-source acquired data, and the adaptive spatiotemporal heatmap is constructed based on the fusion of the aligned multi-source heterogeneous data, it can more comprehensively reflect the changes of each spatial node in the airport, improving the accuracy of prediction.

[0092] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and / or second embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a schematic diagram of the spatiotemporal model of airport operation status provided in Embodiment 3 of the airport operation status dynamic prediction method of this application.

[0093] like Figure 3 As shown in the embodiment of this application, before the step of inputting the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction and obtaining the prediction result, the method further includes:

[0094] Step S100: Obtain training multimodal spatiotemporal data and training spatiotemporal heatmap;

[0095] Step S200: Label the training spatiotemporal heatmap and the training multimodal spatiotemporal data to obtain a labeled training dataset;

[0096] Step S300: Input the labeled training dataset into the initial airport operation status spatiotemporal model for training to obtain the airport operation status spatiotemporal model.

[0097] It should be noted that the aforementioned training multimodal spatiotemporal data is the same as the multimodal spatiotemporal data used for training, and the aforementioned training spatiotemporal heatmap is the same as the spatiotemporal heatmap corresponding to the aforementioned training multimodal spatiotemporal data. By labeling the training spatiotemporal heatmap and the training multimodal spatiotemporal data, the model can learn the dynamic evolution of the spatiotemporal heatmap more quickly during model training, improve the model's ability to capture cross-modal causal relationships, and thus improve the model's prediction accuracy.

[0098] In some embodiments of this application, the initial airport operational status spatiotemporal model includes: a feature processing layer, a shared layer, a private layer, and an output layer; wherein, the feature processing layer is used to acquire labeled training data in the labeled training dataset and determine the private channels corresponding to the labeled training data, the labeled training data including the labeled training spatiotemporal heatmap and the labeled training multimodal spatiotemporal data; the shared layer is used to extract shared features based on the training spatiotemporal heatmap; the private layer includes several private channels; the private layer is used to process the corresponding labeled training data based on the private channels to obtain private channel features; the output layer is used to output the prediction results of the labeled training data based on the shared features and the private channel features.

[0099] It should be noted that the feature processing layer can identify the private channels corresponding to the labeled training data. For the private layer, each private channel can correspond to a prediction task for a spatial node. For the shared layer, the common low-level feature representations of all private channels can be extracted to promote feature sharing and reuse. In the output layer, activation functions can be set; different activation functions can be set for different private channels, thereby achieving multi-dimensional prediction results output.

[0100] It should be explained that the shared layer can include a spatiotemporal graph convolution module and a temporal attention module. The spatiotemporal graph convolution module can acquire the spatial node features and adjacency matrix of the training spatiotemporal heatmap, and aggregate the neighbor information of each spatial node through graph convolution to capture the dependencies between spatial nodes, thereby enhancing the training spatiotemporal heatmap and outputting an enhanced training spatiotemporal heatmap. The temporal attention module can capture the temporal series features in the spatiotemporal heatmap and capture long-term temporal dependencies based on a multi-head attention mechanism to achieve automatic learning of temporal weights, thereby enhancing the temporal series features of the training spatiotemporal heatmap. By weighted fusion of the spatiotemporal heatmaps output by the spatiotemporal graph convolution module and the temporal attention module, a shared spatiotemporal feature representation can be obtained, that is, a spatiotemporal heatmap that can be input to each private channel.

[0101] It should be noted that each private channel can set its own prediction target for each spatial node. The specific prediction target can be set according to the needs of actual application, such as flight punctuality prediction branch, delay risk prediction branch, traffic flow prediction branch, etc. This application embodiment does not limit this.

[0102] In some embodiments of this application, in order to train the initial airport situation spatiotemporal model, the step of inputting the labeled training dataset into the initial airport operational situation spatiotemporal model for training to obtain the airport operational situation spatiotemporal model includes: inputting the labeled training data in the labeled training dataset into the initial airport operational situation spatiotemporal model for prediction to obtain a prediction result; performing loss evaluation on the prediction result based on a preset multi-task loss function to obtain loss parameters; iteratively optimizing the initial airport operational situation spatiotemporal model based on the loss parameters to obtain the iteratively optimized initial airport situation spatiotemporal model; and when the iteratively optimized initial airport situation spatiotemporal model meets preset iteration conditions, using the iteratively optimized initial airport situation spatiotemporal model as the airport operational situation spatiotemporal model.

[0103] It should be noted that the value of the multi-task loss function mentioned above can be a weighted sum of the loss parameters corresponding to each private channel. The loss function and the weight of the loss function corresponding to each private channel can be dynamically adjusted based on the actual application situation. This application embodiment does not limit this.

[0104] It is understandable that the aforementioned loss parameters are values ​​calculated based on the loss function. The loss parameters of each private channel can be used to update the parameters of each private channel in the initial airport situation spatiotemporal model; the loss parameters corresponding to the multi-task loss function can be used to update the parameters of the shared layer, promoting the learning of lower-level features. By continuously iterating and optimizing the initial airport operational situation spatiotemporal model using the loss parameters, the final airport operational situation spatiotemporal model can be obtained.

[0105] It should be noted that the above-mentioned preset iteration conditions can be that the number of iterations is greater than the preset number of iterations, or that the accuracy of the prediction result is greater than the preset accuracy. This application embodiment does not limit this.

[0106] This application embodiment acquires training multimodal spatiotemporal data and training spatiotemporal heatmaps; labels the training spatiotemporal heatmaps and training multimodal spatiotemporal data to obtain a labeled training dataset; and inputs the labeled training dataset into an initial airport operation status spatiotemporal model for training to obtain the airport operation status spatiotemporal model. Because the airport operation status spatiotemporal model is trained based on multimodal spatiotemporal data and spatiotemporal heatmaps, the robustness and prediction accuracy of the model are improved.

[0107] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the dynamic prediction method for airport operation status of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0108] This application also provides an airport operation status dynamic prediction device, please refer to... Figure 4 , Figure 4 This is a schematic diagram of the module structure of the airport operation status dynamic prediction device according to an embodiment of this application. The airport operation status dynamic prediction device includes:

[0109] Data acquisition module 10 is used to acquire multi-source heterogeneous data on airport operations;

[0110] Data fusion module 20 is used to perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data;

[0111] The situation prediction module 30 is used to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction and to obtain the situation prediction result.

[0112] The airport operation status dynamic prediction device provided in this application, employing the airport operation status dynamic prediction method in the above embodiments, can solve the technical problems that airport operation status dynamic prediction easily leads to resource allocation imbalance and insufficient response capability to abnormal events. Compared with the prior art, the beneficial effects of the airport operation status dynamic prediction device provided in this application are the same as the beneficial effects of the airport operation status dynamic prediction method provided in the above embodiments, and other technical features in the airport operation status dynamic prediction device are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0113] This application provides an airport operation status dynamic prediction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the airport operation status dynamic prediction method in the above embodiment 1.

[0114] The following is for reference. Figure 5The diagram illustrates a structural schematic suitable for implementing the airport operation status dynamic prediction device in the embodiments of this application. The airport operation status dynamic prediction device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), vehicle terminals (e.g., vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The airport operation status dynamic prediction device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0115] like Figure 5 As shown, the airport operation status dynamic prediction device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the airport operation status dynamic prediction device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the airport operations dynamic prediction equipment to communicate wirelessly or wiredly with other equipment to exchange data. Although the figure shows airport operations dynamic prediction equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0116] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0117] The airport operation status dynamic prediction device provided in this application, employing the airport operation status dynamic prediction method in the above embodiments, can solve the technical problems that airport operation status dynamic prediction easily leads to resource allocation imbalance and insufficient response capability to abnormal events. Compared with the prior art, the beneficial effects of the airport operation status dynamic prediction device provided in this application are the same as the beneficial effects of the airport operation status dynamic prediction method provided in the above embodiments, and other technical features in this airport operation status dynamic prediction device are the same as the features disclosed in the method of the previous embodiment, and will not be repeated here.

[0118] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0119] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0120] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the airport operation status dynamic prediction method in the above embodiments.

[0121] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0122] The aforementioned computer-readable storage medium may be included in the airport operation status dynamic prediction device; or it may exist independently and not be installed in the airport operation status dynamic prediction device.

[0123] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the airport operational status dynamic prediction device, cause the airport operational status dynamic prediction device to:

[0124] Acquire multi-source heterogeneous data on airport operations;

[0125] Multimodal fusion is performed based on the aforementioned multi-source heterogeneous data to obtain multimodal spatiotemporal data;

[0126] The multimodal spatiotemporal data is input into a pre-trained spatiotemporal model of airport operation status for dynamic situation prediction, and the situation prediction results are obtained.

[0127] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0130] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described dynamic prediction method for airport operational status. This addresses the technical problems of resource allocation imbalance and insufficient response to abnormal events caused by dynamic prediction of airport operational status. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the dynamic prediction method for airport operational status provided in the above embodiments, and will not be elaborated upon here.

[0131] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for dynamic prediction of airport operational status.

[0132] The computer program product provided in this application can solve the technical problems that dynamic prediction of airport operation status easily leads to imbalance in resource allocation and insufficient response to abnormal events. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the dynamic prediction method for airport operation status provided in the above embodiments, and will not be repeated here.

[0133] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for dynamic prediction of airport operational status, characterized in that, The method includes: Acquire multi-source heterogeneous data on airport operations; Multimodal fusion is performed based on the aforementioned multi-source heterogeneous data to obtain multimodal spatiotemporal data; The multimodal spatiotemporal data is input into a pre-trained airport operation situation spatiotemporal model for dynamic situation prediction, and the situation prediction result is obtained. The step of performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data includes: Obtain the temporal and spatial information of the multi-source heterogeneous data; Based on the time information and the spatial information, the multi-source heterogeneous data is spatiotemporally aligned to obtain the aligned multi-source heterogeneous data. Multimodal feature extraction is performed on the aligned multi-source heterogeneous data to obtain multimodal data features; An adaptive spatiotemporal hotspot map is obtained by fusing and constructing the multimodal data features. Based on the adaptive spatiotemporal heat map, multimodal spatiotemporal data of airport operations are determined; The step of fusing and constructing an adaptive spatiotemporal heatmap based on the multimodal data features includes: Obtain airport layout data; An initial association diagram of the airport is constructed based on the layout data; Construct an association adjacency matrix based on the relationships between spatial nodes in the initial association graph; The multimodal features are fused in the time dimension based on the association adjacency matrix to obtain an adaptive spatiotemporal hotspot map; Before the step of inputting the multimodal spatiotemporal data into a pre-trained airport operational situation spatiotemporal model for dynamic situation prediction and obtaining the situation prediction result, the method further includes: Acquire training multimodal spatiotemporal data and training spatiotemporal heatmap; The labeled training dataset is obtained by labeling the training spatiotemporal heatmap and the training multimodal spatiotemporal data. The labeled training dataset is input into the initial airport operation status spatiotemporal model for training to obtain the airport operation status spatiotemporal model. The initial airport operational status spatiotemporal model includes: a feature processing layer, a shared layer, a private layer, and an output layer; The feature processing layer is used to obtain the labeled training data in the labeled training dataset and determine the private channel corresponding to the labeled training data. The labeled training data includes the labeled training spatiotemporal heatmap and the labeled training multimodal spatiotemporal data. The shared layer is used to extract shared features based on the training spatiotemporal heatmap; The private layer includes several private channels; The private layer is used to process the corresponding labeled training data based on the private channel to obtain private channel features; The output layer is used to output the prediction result of the labeled training data based on the shared features and the private channel features.

2. The airport operation status dynamic prediction method as described in claim 1, characterized in that, The step of inputting the labeled training dataset into the initial airport operational status spatiotemporal model for training to obtain the airport operational status spatiotemporal model includes: The labeled training data in the labeled training dataset is input into the initial airport operational status spatiotemporal model for prediction, and the prediction result is obtained. The prediction results are evaluated for loss based on a preset multi-task loss function to obtain loss parameters. Based on the loss parameters, the initial airport operational status spatiotemporal model is iteratively optimized to obtain the iteratively optimized initial airport status spatiotemporal model. When the iteratively optimized initial airport situation spatiotemporal model meets the preset iteration conditions, the iteratively optimized initial airport situation spatiotemporal model is used as the airport operation situation spatiotemporal model.

3. An airport operation status dynamic prediction device, characterized in that, The airport operation status dynamic prediction device includes: The data acquisition module is used to acquire multi-source heterogeneous data on airport operations; The data fusion module is used to perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; The situation prediction module is used to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform dynamic situation prediction and obtain situation prediction results. The data fusion module is further configured to acquire the temporal and spatial information of the multi-source heterogeneous data; perform spatiotemporal alignment of the multi-source heterogeneous data based on the temporal and spatial information to obtain aligned multi-source heterogeneous data; extract multimodal features based on the aligned multi-source heterogeneous data to obtain multimodal data features; fuse and construct an adaptive spatiotemporal heatmap based on the multimodal data features; and determine the multimodal spatiotemporal data of airport operations based on the adaptive spatiotemporal heatmap. The data fusion module is also used to acquire airport layout data; construct an initial association graph of the airport based on the layout data; construct an association adjacency matrix based on the relationship between spatial nodes in the initial association graph; and fuse the multimodal features in the time dimension according to the association adjacency matrix to obtain an adaptive spatiotemporal hotspot map. The situation prediction module is also used to acquire training multimodal spatiotemporal data and training spatiotemporal heatmaps; to label the training spatiotemporal heatmaps and training multimodal spatiotemporal data to obtain a labeled training dataset; and to input the labeled training dataset into the initial airport operation situation spatiotemporal model for training to obtain the airport operation situation spatiotemporal model. The initial airport operational status spatiotemporal model includes: a feature processing layer, a shared layer, a private layer, and an output layer; The feature processing layer is used to obtain the labeled training data in the labeled training dataset and determine the private channel corresponding to the labeled training data. The labeled training data includes the labeled training spatiotemporal heatmap and the labeled training multimodal spatiotemporal data. The shared layer is used to extract shared features based on the training spatiotemporal heatmap; The private layer includes several private channels; The private layer is used to process the corresponding labeled training data based on the private channel to obtain private channel features; The output layer is used to output the prediction result of the labeled training data based on the shared features and the private channel features.

4. An airport operation status dynamic prediction device, characterized in that, The device includes: The system includes a memory, a processor, and an airport operations dynamic prediction program stored in the memory and executable on the processor, the airport operations dynamic prediction program being configured to implement the steps of the airport operations dynamic prediction method as described in claim 1 or 2.

5. A storage medium, characterized in that, The storage medium stores an airport operation status dynamic prediction program, which, when executed by a processor, implements the steps of the airport operation status dynamic prediction method as described in claim 1 or 2.

6. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the airport operation status dynamic prediction method as described in any one of claims 1 or 2.

Images