Airport operation situation dynamic prediction method and device, equipment, storage medium and program product

Abstract

The invention discloses an airport operation situation dynamic prediction method and device, equipment, a storage medium and a program product, and relates to the technical field of airport management.The airport operation situation dynamic prediction method comprises the steps that multi-source heterogeneous data of airport operation is acquired; performing multi-modal fusion based on the multi-source heterogeneous data to obtain multi-modal spatio-temporal data; and inputting the multi-modal spatio-temporal data into a pre-trained airport operation situation spatio-temporal model for situation dynamic prediction to obtain a situation prediction result. Due to the fact that multi-modal fusion is carried out based on the multi-source heterogeneous data, complementarity and collaboration among the data can be mined, and therefore all-dimensional dynamic prediction of the airport operation situation is achieved. The situation dynamic prediction is carried out through the pre-trained airport operation situation spatial-temporal model, so that the instantaneity of processing the abnormal time is improved, and the application scene is enlarged.

Description

Technical Field

[0001] The present application relates to the field of airport management technology, and in particular to a method, device, equipment, storage medium and program product for dynamic prediction of airport operation status. Background Art

[0002] As core spatial nodes of air transportation, airports' operational efficiency, safety, and service levels directly impact regional economic development and passenger travel experience. Predicting airport operational trends can provide decision support for airport operations.

[0003] Currently, most airport situation forecasting methods focus on traffic flow, which can lead to unbalanced resource allocation and insufficient response capabilities to abnormal events. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment, storage medium and program product for dynamic prediction of airport operation status, aiming to solve the technical problems that dynamic prediction of airport operation status easily leads to imbalance in resource allocation and insufficient ability to respond to abnormal events.

[0005] To achieve the above objectives, the present application proposes a method for dynamic prediction of airport operation status, which includes: Acquire multi-source heterogeneous data on airport operations; Performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; The multimodal spatiotemporal data is input into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a situation prediction result.

[0006] In one embodiment, the step of performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal data features includes: Acquiring time information and spatial information of the multi-source heterogeneous data; Performing spatiotemporal alignment on the multi-source heterogeneous data based on the time information and the spatial information to obtain aligned multi-source heterogeneous data; Performing multimodal feature extraction based on the aligned multi-source heterogeneous data to obtain multimodal data features; A self-adaptive spatiotemporal heat map is obtained by fusing and constructing the multimodal data features; Multimodal spatiotemporal data of airport operations are determined based on the adaptive spatiotemporal heat map.

[0007] In one embodiment, the step of fusing and constructing the multimodal data features to obtain an adaptive spatiotemporal heat map includes: Get the layout data of the airport; constructing an initial association map of the airport based on the layout data; Constructing an association adjacency matrix based on the relationships between spatial nodes in the initial association graph; The multimodal features are fused in the time dimension according to the correlation adjacency matrix to obtain an adaptive spatiotemporal heat map.

[0008] In one embodiment, before the step of inputting the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model for situation dynamic prediction and obtaining a prediction result, the step further includes: Obtain training multimodal spatiotemporal data and training spatiotemporal heat maps; Marking is performed based on the training spatiotemporal heat map and the training multimodal spatiotemporal data to obtain a marked training data set; The labeled training data set is input into the initial airport operation situation spatiotemporal model for training to obtain the airport operation situation spatiotemporal model.

[0009] In one embodiment, the initial airport operation situation spatiotemporal model includes: a feature processing layer, a shared layer, a private layer, and an output layer; The feature processing layer is used to obtain the labeled training data in the labeled training data set and determine the private channel corresponding to the labeled training data, wherein the labeled training data includes the labeled training spatiotemporal heat map and the labeled training multimodal spatiotemporal data; The shared layer is used to extract shared features based on the training spatiotemporal heat map; The private layer includes several private channels; The private layer is configured to process the corresponding labeled training data based on the private channel to obtain private channel features; The output layer is used to output a prediction result of the labeled training data based on the shared features and the private channel features.

[0010] In one embodiment, the step of inputting the labeled training data set into the initial airport operation situation spatiotemporal model for training to obtain the airport operation situation spatiotemporal model includes: Inputting the labeled training data in the labeled training data set into the initial airport operation situation spatiotemporal model for prediction to obtain a prediction result; Performing a loss evaluation on the prediction result based on a preset multi-task loss function to obtain a loss parameter; Iteratively optimizing the initial airport operation situation spatiotemporal model based on the loss parameter to obtain the iteratively optimized initial airport situation spatiotemporal model; When the iteratively optimized initial airport situation spatiotemporal model meets the preset iteration conditions, the iteratively optimized initial airport situation spatiotemporal model is used as the airport operation situation spatiotemporal model.

[0011] In addition, to achieve the above-mentioned purpose, the present application also proposes a device for dynamically predicting airport operation status, which includes: Data acquisition module, used to obtain multi-source heterogeneous data of airport operations; A data fusion module, configured to perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; The situation prediction module is used to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a situation prediction result.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a device for dynamic prediction of airport operation status, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the method for dynamic prediction of airport operation status as described above.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by the processor, the steps of the airport operation status dynamic prediction method as described above are implemented.

[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the airport operation status dynamic prediction method as described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This application obtains multi-source heterogeneous data on airport operations; performs multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; and inputs the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain situation prediction results. Because multimodal fusion is performed based on multi-source heterogeneous data, the complementarity and synergy between the data can be explored, thereby achieving a full range of dynamic predictions of the airport operation situation; using the pre-trained airport operation situation spatiotemporal model for situation dynamic prediction improves the immediacy of abnormal event processing and expands the application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A flowchart of the first embodiment of the method for dynamic prediction of airport operation status provided in this application; Figure 2 Schematic diagram of the structure of the adaptive spatiotemporal heat map generation model provided in Example 2 of the method for dynamic prediction of airport operation status of this application; Figure 3 A schematic diagram of the structure of the airport operation situation spatiotemporal model provided in Example 3 of the method for dynamic prediction of airport operation situation of this application; Figure 4 This is a schematic diagram of the module structure of the device for dynamically predicting airport operation status according to an embodiment of the present application; Figure 5 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the method for dynamically predicting airport operating conditions in an embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of the embodiment of the present application is: to obtain multi-source heterogeneous data of airport operations; to perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain situation prediction results.

[0023] It should be noted that the execution entity of this embodiment may be a computing service device with data processing, network communication, and program execution capabilities, such as a computer or server, or an electronic device or virtual device capable of implementing the aforementioned functions. This embodiment and the following embodiments will be described below using an airport operational status dynamic prediction device (hereinafter referred to as the prediction device) as an example.

[0024] Based on this, the embodiment of the present application provides a method for dynamic prediction of airport operation status, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for dynamically predicting airport operation status in this application.

[0025] In this embodiment, the airport operation status dynamic prediction method includes steps S10 to S30: Step S10, acquiring multi-source heterogeneous data of airport operations; It should be noted that the multi-source heterogeneous data may include data sets from different sources and may have different structures, formats, or semantics. In the embodiment of the present application, the multi-source heterogeneous data may include airspace usage, satellite weather data, flight schedule data, radar data, traffic flow data, video surveillance data, and IoT sensor data, which can be used to form a real-time digital twin model of the airport's overall operational status.

[0026] In the embodiments of the present application, the specific method of acquiring multi-source heterogeneous data is not limited and can be selected according to the actual application situation.

[0027] Step S20: performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data.

[0028] It can be understood that by multimodally fusing multi-source heterogeneous data from different sources, the complementarity and synergy between the data can be explored, thereby achieving a full range of dynamic predictions of airport operation status.

[0029] It should be noted that the multi-source heterogeneous data in the embodiments of this application may be accompanied by relevant time and space information. The time information can be used to determine the time when the multi-source heterogeneous data was generated from the time axis, and the space information can be used to determine the location where the multi-source heterogeneous data was generated from the space axis. By performing multimodal fusion on the multi-source heterogeneous data, a unified representation of the data's temporal, spatial, and multimodal characteristics can be achieved, thereby improving the accuracy and robustness of airport operational status predictions.

[0030] In some implementations of the embodiments of the present application, the multimodal fusion method may be weighted fusion or fusion using a neural network, or other fusion methods, which are not limited in the embodiments of the present application.

[0031] In some implementations of the embodiments of the present application, the multimodal fusion method in the embodiments of the present application may be: extracting spatial features from multi-source heterogeneous data using a convolutional neural network, extracting temporal features from multi-source heterogeneous data using a long short-term memory network, and extracting semantic vectors from multi-source heterogeneous data using a Transformer encoder. By setting up independent encoders (such as GCN and CNN) for each source of multi-source heterogeneous data to encode the extracted features, corresponding spatial feature encoding, temporal feature encoding, and semantic vector encoding are obtained, and multimodal fusion is achieved by weighting the feature encodings.

[0032] It should be noted that the multimodal spatiotemporal data in the embodiment of the present application is data that can more comprehensively and accurately reflect the airport's flight dynamics, weather dynamics and other information by fusing multi-source heterogeneous data in the time dimension and the space dimension. By fusing multi-source data such as flight dynamics, weather data, instruction data, and traffic flow data, the deviation of a single data source can be avoided, and the prediction accuracy and stability of the model can be improved.

[0033] Step S30: input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a situation prediction result.

[0034] It should be noted that the aforementioned airport operational status spatiotemporal model is a predictive model that can be used for dynamic prediction based on the shared and private features of the input multimodal spatiotemporal data. This model enables multi-dimensional prediction of airport operational status, thereby achieving a uniform distribution of airport dispatch resources. Furthermore, because this application performs dynamic predictions based on multi-source heterogeneous data, it improves the immediacy of abnormal event processing and expands its application scenarios.

[0035] The embodiments of the present application obtain multi-source heterogeneous data on airport operations; perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; and input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain situation prediction results. Because multimodal fusion is performed based on multi-source heterogeneous data, the complementarity and synergy between the data can be explored, thereby achieving a full range of dynamic predictions of airport operation status. Using the pre-trained airport operation situation spatiotemporal model for situation dynamic prediction improves the immediacy of abnormal event processing and expands the application scenarios.

[0036] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 , Figure 2Schematic diagram of the structure of the adaptive spatiotemporal heat map generation model provided in Example 2 of the method for dynamic prediction of airport operation status of the present application.

[0037] In the embodiment of the present application, the step of performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal data features includes: Step S21, obtaining time information and spatial information of the multi-source heterogeneous data; Step S22 : performing spatiotemporal alignment on the multi-source heterogeneous data based on the time information and the spatial information to obtain the aligned multi-source heterogeneous data.

[0038] It should be noted that when an event occurs at a spatial node (such as a flight approaching arrival, a conveyor belt being used, etc.), the time information and spatial information generated by the data can be recorded synchronously. In the embodiment of the present application, the time information and location information of multi-source heterogeneous data from different data sources are unified in data structure and aligned on the time axis and spatial position to obtain aligned multi-source heterogeneous data.

[0039] In some implementations of the present application, the time and location information corresponding to data collected from different sources may be inconsistent. When obtaining these multi-source heterogeneous data, they can be standardized to unify them into the same time and space representations, such as year-month-day representation, XYZ coordinate axis representation, longitude and latitude representation, etc., which are not limited in the present application.

[0040] It's easy to understand that by aligning multi-source heterogeneous data, we can unify the structure of this data across both the temporal and spatial dimensions. This allows us to construct adaptive spatiotemporal heat maps to represent the changing states of each spatial node along the time axis. Predictions based on these changing states enable multi-dimensional airport situation forecasting, improving prediction accuracy.

[0041] It should be understood that the aforementioned spatial nodes are entities in the airport that require situation prediction, such as conveyor belts, boarding gates, flights, shuttle buses, roads, etc., and this embodiment of the present application does not limit this. The spatial information of multi-source heterogeneous data can be used to determine the spatial nodes corresponding to the multi-source heterogeneous data; the time information of the multi-source heterogeneous data can be used to determine the time when the multi-source heterogeneous data was generated. The spatial information and time information can be used to determine the changing state of this type of multi-source heterogeneous data in the spatial node.

[0042] In a specific implementation, the prediction device of an embodiment of the present application can obtain the time information and spatial information of multi-source heterogeneous data, and perform spatiotemporal alignment on the multi-source heterogeneous data based on the time information and spatial information, so that these time information and spatial information can be represented in the same spatiotemporal dimension, thereby obtaining aligned multi-source heterogeneous data.

[0043] Step S23, performing multimodal feature extraction based on the aligned multi-source heterogeneous data to obtain multimodal data features; Step S24, performing fusion construction based on the multimodal data features to obtain an adaptive spatiotemporal heat map; Step S25: determining multimodal spatiotemporal data of airport operations based on the adaptive spatiotemporal heat map.

[0044] It should be noted that the multimodal feature extraction process described above can include temporal feature extraction, spatial feature extraction, and semantic vector extraction. That is, the extracted multimodal data features can include event features, spatial features, and semantic vector features. Temporal feature extraction can determine temporal features such as periodicity and trend characteristics of multi-source heterogeneous data within a spatial node. Spatial feature extraction can determine spatial features such as regional associations and adjacency matrices for each spatial node. Semantic vector extraction can determine semantic vector features such as implicit associations and event descriptions within each spatial node.

[0045] It should be explained that by fusing and constructing the above-mentioned multimodal data features, an adaptive spatiotemporal heat map can be obtained to represent the data changes in each spatial node, and then the multimodal spatiotemporal data of airport operations can be determined.

[0046] In some implementations of the embodiments of the present application, the step of fusing and constructing the multimodal data features to obtain an adaptive spatiotemporal heat map includes: obtaining the layout data of the airport; constructing an initial association graph of the airport based on the layout data; constructing an association adjacency matrix based on the relationship between spatial nodes in the initial association graph; and fusing the multimodal features in the time dimension according to the association adjacency matrix to obtain an adaptive spatiotemporal heat map.

[0047] It should be noted that the layout data of the above-mentioned airport may include the physical layout data of the airport, which may specifically include the airport runway and runway length, apron size and location, terminal location and structure, etc. Based on the static initial association graph generation module, the layout data is processed to construct an initial association graph of the airport. Through the initial association graph, the static physical layout of the airport can be displayed, and then the dynamic spatiotemporal association of multimodal data features can be realized. In the embodiment of the present application, the specific method of constructing the initial association graph based on the layout data is not limited, and it can be digital twin model construction, building information modeling, etc., and the embodiment of the present application does not limit this.

[0048] Specifically, in the embodiments of this application, refer to Figure 2 The adaptive spatiotemporal heat map generation model of the present application includes an input layer, a static initial correlation map generation module, a dynamic fusion correlation map generation module, and an output layer. The dynamic fusion correlation map generation module has a similar structure to the static initial correlation map generation module.

[0049] It is understood that the static initial correlation map generation module can include attention units, maximum pooling units, and several convolution units. Each convolution unit includes a convolution subunit (which performs both graph convolution and temporal convolution) and a cropping subunit. Through the operation of multiple convolution units, the output of each convolution subunit is cropped by the cropping subunit at each step to ensure output consistency. The attention unit then captures the cross-temporal and spatial correlations between features, improving the model's multimodal fusion capabilities. Finally, the maximum pooling unit is used to obtain the most representative features.

[0050] It should be explained that the initial association graph obtained may include several spatial nodes, and the spatial nodes may correspond to a time axis with a time dimension. By processing the initial association graph with a dynamic fusion association graph generation module, an adaptive spatiotemporal heat map can be obtained. Specifically, based on the attention mechanism and through the semantic vectors in the multimodal data features, the implicit association between each spatial node in the initial association graph and the events occurring at each time point can be realized; at the same time, based on the spatial features and time features in the multimodal data features, data fusion with the spatial nodes and time axis in the initial association graph can be achieved, thereby constructing a dynamically associated adaptive spatiotemporal heat map.

[0051] It should be noted that the matrix dimension of the association adjacency matrix can be the number of spatial nodes × the number of spatial nodes, and the matrix elements in the association adjacency matrix can be used to represent the strength of the relationship between spatial nodes. The weights in the adjacency matrix can be calculated based on the distance between spatial nodes in the matrix, or can be a comprehensive calculation based on flight frequency, event priority, etc., which is not limited in this embodiment of the application.

[0052] It can be understood that the adaptive spatiotemporal heat map can be a two-dimensional map or a three-dimensional map, which can be used to represent events occurring at different times at various spatial nodes in the airport and the implicit connections between the events. By making predictions based on the adaptive spatiotemporal heat map, the accuracy and comprehensiveness of situation predictions are improved.

[0053] The embodiments of the present application obtain temporal and spatial information from multi-source heterogeneous data; perform spatiotemporal alignment of the multi-source heterogeneous data based on the temporal and spatial information to obtain aligned multi-source heterogeneous data; extract multimodal features from the aligned multi-source heterogeneous data to obtain multimodal data features; fuse and construct an adaptive spatiotemporal heat map based on the multimodal data features; and determine multimodal spatiotemporal data of airport operations based on the adaptive spatiotemporal heat map. Because the alignment is based on the temporal and spatial information of the multi-source purchased data, and the adaptive spatiotemporal heat map is constructed based on the fusion of the aligned multi-source heterogeneous data, it can more comprehensively reflect the changes in various spatial nodes in the airport, thereby improving the accuracy of the prediction.

[0054] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment and / or the second embodiment can be referred to the above introduction and will not be described in detail later. Figure 3 , Figure 3 This is a structural diagram of the airport operation situation spatiotemporal model provided in Example 3 of the airport operation situation dynamic prediction method of this application.

[0055] like Figure 3 As shown, in the embodiment of the present application, before the step of inputting the multimodal spatiotemporal data into the pre-trained airport operation situation spatiotemporal model for situation dynamic prediction and obtaining the prediction result, the method further includes: Step S100, obtaining training multimodal spatiotemporal data and training spatiotemporal heat map; Step S200, labeling is performed based on the training spatiotemporal heat map and the training multimodal spatiotemporal data to obtain a labeled training data set; Step S300: input the labeled training data set into the initial airport operation situation spatiotemporal model for training to obtain the airport operation situation spatiotemporal model.

[0056] It should be noted that the aforementioned training multimodal spatiotemporal data is also the multimodal spatiotemporal data used for training, and the aforementioned training spatiotemporal heat map is also the spatiotemporal heat map corresponding to the aforementioned training multimodal spatiotemporal data. By labeling the training spatiotemporal heat map and the training multimodal spatiotemporal data, the dynamic evolution of the spatiotemporal heat map can be learned more quickly during model training, improving the model's ability to capture cross-modal causal relationships and, consequently, enhancing the model's predictive accuracy.

[0057] In some implementations of the embodiments of the present application, the initial airport operation situation spatiotemporal model includes: a feature processing layer, a shared layer, a private layer and an output layer; wherein the feature processing layer is used to obtain the labeled training data in the labeled training data set and determine the private channel corresponding to the labeled training data, and the labeled training data includes the labeled training spatiotemporal heat map and the labeled training multimodal spatiotemporal data; the shared layer is used to extract shared features based on the training spatiotemporal heat map; the private layer includes several private channels; the private layer is used to process the corresponding labeled training data based on the private channel to obtain private channel features; the output layer is used to output the prediction results of the labeled training data based on the shared features and the private channel features.

[0058] It should be noted that the feature processing layer can determine the private channels corresponding to the labeled training data. In the private layer, each private channel can correspond to a prediction task for a spatial node. In the shared layer, underlying feature representations common to all private channels can be extracted to facilitate feature sharing and reuse. The output layer can be configured with different activation functions corresponding to different private channels, thereby achieving multi-dimensional prediction output.

[0059] It should be explained that the shared layer may include a spatiotemporal graph convolution module and a temporal attention module. The spatiotemporal graph convolution module can obtain the spatial node features and adjacency matrix in the training spatiotemporal heat map, and aggregate the neighbor information of each spatial node through graph convolution to capture the dependency relationship between spatial nodes to enhance the training spatiotemporal heat map, and then output the enhanced training spatiotemporal heat map. The temporal attention module can capture the time series features in the spatiotemporal heat map, and capture long-term temporal dependencies based on the multi-head attention mechanism to achieve automatic learning of time weights, thereby enhancing the time series features of the training spatiotemporal heat map. By weighted fusion of the spatiotemporal heat maps output by the spatiotemporal graph convolution module and the temporal attention module, a shared spatiotemporal feature representation can be obtained, that is, a spatiotemporal heat map that can be input into each private channel.

[0060] It should be noted that each private channel can set an exclusive prediction target for each spatial node. The corresponding specific prediction target can be set according to the needs of actual applications, such as flight punctuality prediction branch, detention risk prediction branch, traffic flow prediction branch, etc. The embodiment of this application does not limit this.

[0061] In some implementations of the embodiments of the present application, in order to realize the training of the initial airport situation spatiotemporal model, the step of inputting the labeled training data set into the initial airport operation situation spatiotemporal model for training to obtain the airport operation situation spatiotemporal model includes: inputting the labeled training data in the labeled training data set into the initial airport operation situation spatiotemporal model for prediction to obtain a prediction result; performing a loss evaluation on the prediction result based on a preset multi-task loss function to obtain a loss parameter; iteratively optimizing the initial airport operation situation spatiotemporal model based on the loss parameter to obtain the iteratively optimized initial airport situation spatiotemporal model; when the iteratively optimized initial airport situation spatiotemporal model meets the preset iteration conditions, the iteratively optimized initial airport situation spatiotemporal model is used as the airport operation situation spatiotemporal model.

[0062] It should be noted that the value of the above-mentioned multi-task loss function can be the weighted sum of the loss parameters corresponding to each private channel. The loss function corresponding to each private channel and the loss function weight can be dynamically adjusted based on the situation in the actual application. The embodiment of the present application does not limit this.

[0063] It's understandable that the aforementioned loss parameters are calculated based on the loss function. The loss parameters of each private channel can be used to update the parameters of each private channel in the initial airport situation spatiotemporal model. The loss parameters corresponding to the multi-task loss function can be used to update the parameters of the shared layer, promoting the learning of underlying features. By continuously iteratively optimizing the initial airport operation situation spatiotemporal model using the loss parameters, the final airport operation situation spatiotemporal model can be obtained.

[0064] It should be noted that the above-mentioned preset iteration condition can be that the number of iterations is greater than the preset number of iterations, or that the accuracy of the prediction result is greater than the preset accuracy. The embodiment of the present application does not limit this.

[0065] This embodiment of the present application obtains training multimodal spatiotemporal data and a training spatiotemporal heat map; labels the training spatiotemporal heat map and the training multimodal spatiotemporal data to obtain a labeled training dataset; and then inputs the labeled training dataset into an initial airport operational status spatiotemporal model for training, thereby obtaining an airport operational status spatiotemporal model. Because the airport operational status spatiotemporal model is trained based on multimodal spatiotemporal data and a spatiotemporal heat map, the robustness and prediction accuracy of the model are improved.

[0066] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the airport operation status dynamic prediction method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0067] This application also provides a device for dynamic prediction of airport operation status, please refer to Figure 4 , Figure 4 This is a schematic diagram of the module structure of the airport operation situation dynamic prediction device according to an embodiment of the present application, which includes: The data acquisition module 10 is used to acquire multi-source heterogeneous data of airport operations; A data fusion module 20 is configured to perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; The situation prediction module 30 is used to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a situation prediction result.

[0068] The airport operation status dynamic prediction device provided in this application, utilizing the airport operation status dynamic prediction method described in the aforementioned embodiments, can address the technical issues that dynamic prediction of airport operation status can easily lead to imbalanced resource allocation and insufficient response capabilities to abnormal events. Compared to the prior art, the airport operation status dynamic prediction device provided in this application achieves the same beneficial effects as the airport operation status dynamic prediction method described in the aforementioned embodiments. Other technical features of the airport operation status dynamic prediction device are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0069] The present application provides a device for dynamically predicting the operation status of an airport, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for dynamically predicting the operation status of an airport in the above-mentioned embodiment one.

[0070] Reference below Figure 5 , which shows a schematic diagram of the structure of a device for dynamically predicting airport operational status suitable for implementing embodiments of the present application. The device for dynamically predicting airport operational status in embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The airport operation status dynamic prediction device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0071] like Figure 5As shown, the airport operational situation dynamic prediction device may include a processing device 1001 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the airport operational situation dynamic prediction device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. Communication device 1009 allows the airport operational situation dynamic prediction device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an airport operational situation dynamic prediction device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0072] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0073] The airport operation status dynamic prediction device provided in this application utilizes the airport operation status dynamic prediction method described in the aforementioned embodiment, resolving the technical issues that dynamic prediction of airport operation status can easily lead to imbalanced resource allocation and insufficient response capabilities to abnormal events. Compared to the prior art, the airport operation status dynamic prediction device provided in this application achieves the same beneficial effects as the airport operation status dynamic prediction method described in the aforementioned embodiment. Other technical features of this device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0074] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0075] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0076] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer program) stored thereon, and the computer-readable program instructions are used to execute the airport operation status dynamic prediction method in the above-mentioned embodiment.

[0077] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0078] The computer-readable storage medium may be included in the airport operation situation dynamic prediction device; or it may exist independently without being assembled into the airport operation situation dynamic prediction device.

[0079] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the airport operation situation dynamic prediction device, the airport operation situation dynamic prediction device: Acquire multi-source heterogeneous data on airport operations; Performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; The multimodal spatiotemporal data is input into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a situation prediction result.

[0080] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0081] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0082] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0083] The computer-readable storage medium provided in this application is a computer-readable storage medium storing computer-readable program instructions (i.e., a computer program) for executing the aforementioned method for dynamically predicting airport operational status. This computer-readable storage medium can address the technical issues that dynamic prediction of airport operational status can easily lead to imbalanced resource allocation and insufficient response capabilities to abnormal events. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the method for dynamically predicting airport operational status provided in the aforementioned embodiments and are not further elaborated here.

[0084] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for dynamic prediction of airport operation status.

[0085] The computer program product provided in this application can address the technical issues that dynamic prediction of airport operational status can easily lead to imbalanced resource allocation and insufficient response capabilities to abnormal events. Compared to existing technologies, the beneficial effects of the computer program product provided in this application are the same as those of the dynamic prediction method for airport operational status provided in the aforementioned embodiments, and are not further elaborated here.

[0086] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for dynamic prediction of airport operation status, characterized in that: The method comprises: Acquire multi-source heterogeneous data on airport operations; Performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; The multimodal spatiotemporal data is input into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a situation prediction result.

2. The method for dynamic prediction of airport operation status according to claim 1, characterized in that: The step of performing multimodal fusion based on the multi-source heterogeneous data to obtain multimodal data features includes: Acquiring time information and spatial information of the multi-source heterogeneous data; Performing spatiotemporal alignment on the multi-source heterogeneous data based on the time information and the spatial information to obtain aligned multi-source heterogeneous data; Performing multimodal feature extraction based on the aligned multi-source heterogeneous data to obtain multimodal data features; A self-adaptive spatiotemporal heat map is obtained by fusing and constructing the multimodal data features; Multimodal spatiotemporal data of airport operations are determined based on the adaptive spatiotemporal heat map.

3. The method for dynamic prediction of airport operation status according to claim 2, characterized in that: The step of fusing and constructing the multimodal data features to obtain an adaptive spatiotemporal heat map includes: Get the layout data of the airport; constructing an initial association map of the airport based on the layout data; Constructing an association adjacency matrix based on the relationships between spatial nodes in the initial association graph; The multimodal features are fused in the time dimension according to the correlation adjacency matrix to obtain an adaptive spatiotemporal heat map.

4. The method for dynamic prediction of airport operation status according to claim 1, characterized in that: Before the step of inputting the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a prediction result, the method further includes: Obtain training multimodal spatiotemporal data and training spatiotemporal heat maps; Marking is performed based on the training spatiotemporal heat map and the training multimodal spatiotemporal data to obtain a marked training data set; The labeled training data set is input into the initial airport operation situation spatiotemporal model for training to obtain the airport operation situation spatiotemporal model.

5. The method for dynamic prediction of airport operation status according to claim 4, characterized in that: The initial airport operation situation spatiotemporal model includes: a feature processing layer, a sharing layer, a private layer and an output layer; The feature processing layer is used to obtain the labeled training data in the labeled training data set and determine the private channel corresponding to the labeled training data, wherein the labeled training data includes the labeled training spatiotemporal heat map and the labeled training multimodal spatiotemporal data; The shared layer is used to extract shared features based on the training spatiotemporal heat map; The private layer includes several private channels; The private layer is configured to process the corresponding labeled training data based on the private channel to obtain private channel features; The output layer is used to output a prediction result of the labeled training data based on the shared features and the private channel features.

6. The method for dynamic prediction of airport operation status according to claim 5, characterized in that: The step of inputting the labeled training data set into the initial airport operation situation spatiotemporal model for training to obtain the airport operation situation spatiotemporal model includes: Inputting the labeled training data in the labeled training data set into the initial airport operation situation spatiotemporal model for prediction to obtain a prediction result; Performing a loss evaluation on the prediction result based on a preset multi-task loss function to obtain a loss parameter; Iteratively optimizing the initial airport operation situation spatiotemporal model based on the loss parameter to obtain the iteratively optimized initial airport situation spatiotemporal model; When the iteratively optimized initial airport situation spatiotemporal model meets the preset iteration conditions, the iteratively optimized initial airport situation spatiotemporal model is used as the airport operation situation spatiotemporal model.

7. A device for dynamic prediction of airport operation status, characterized in that: The airport operation situation dynamic prediction device comprises: Data acquisition module, used to obtain multi-source heterogeneous data of airport operations; A data fusion module, configured to perform multimodal fusion based on the multi-source heterogeneous data to obtain multimodal spatiotemporal data; The situation prediction module is used to input the multimodal spatiotemporal data into a pre-trained airport operation situation spatiotemporal model to perform situation dynamic prediction and obtain a situation prediction result.

8. A dynamic prediction device for airport operation status, characterized in that: The device includes: a memory, a processor, and an airport operation situation dynamic prediction program stored in the memory and executable on the processor, wherein the airport operation situation dynamic prediction program is configured to implement the steps of the airport operation situation dynamic prediction method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores an airport operation situation dynamic prediction program, which, when executed by a processor, implements the steps of the airport operation situation dynamic prediction method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for dynamic prediction of airport operation status according to any one of claims 1 to 6 are implemented.

Images