Flight data real-time effective method, device, equipment and medium
By separating the full and incremental flight databases, combined with binary format and linked list storage structure, the problem of low efficiency in traditional flight data storage is solved, and efficient real-time query and update of flight data is achieved.
Patent Information
- Application Number
- CN202510746061.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-05
- Publication Date
- 2025-10-14
AI Technical Summary
Traditional flight schedule data storage methods are inefficient and cannot meet the real-time requirements of the fare search system. In addition, flight schedule data is updated slowly, resulting in incomplete query results.
It adopts a design that separates the full and incremental flight databases. By parsing the flight plan data and writing it into a memory-mapped file in binary format, it establishes primary and secondary indexes, and uses a linked list storage structure to store new flight information, enabling fast insertion and query.
This improves the efficiency of flight searches, ensures real-time queries and data integrity, and avoids query delays and data inconsistencies during full database updates.
Smart Images
Figure CN120780733A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of data processing, and particularly relates to a flight data real-time validation method, device, equipment and medium. BACKGROUND
[0002] In a fare search system, corresponding flight results and prices need to be calculated according to the user's itinerary. As is known to all, the fare search system is a very complex business system, and the real-time requirement of the search is relatively high, which requires that both the fare search and the flight search need to be highly efficient. If the flight plan data is stored in the traditional way, the flight data is calculated by real-time query during the fare search, and the efficiency is relatively low, which cannot meet the system requirements. Moreover, the flight plan data is relatively slow to update, cannot be updated in real time, and may lead to incomplete query results. Therefore, a new flight data real-time validation method needs to be developed. SUMMARY
[0003] In view of the above problems, the present application discloses a flight data real-time validation method, comprising the following steps:
[0004] Obtaining flight plan data;
[0005] Analyzing the flight plan data;
[0006] Indexing the analyzed flight plan data to generate a full flight database;
[0007] After the full flight database is generated, when new flight information is received, the new flight information is updated to an incremental flight database;
[0008] Based on the full flight database and the incremental flight database, flight data query is performed.
[0009] Further, the analyzing the flight plan data comprises the following steps:
[0010] Converting the analyzed flight plan data into a binary format;
[0011] Writing the converted flight plan data into an in-memory mapping file.
[0012] Further, the index comprises a primary index and a secondary index;
[0013] The primary index key is the departure place and the departure date;
[0014] The secondary index key is the destination and the arrival date.
[0015] Further, after the full flight database is generated, when new flight information is received, the new flight information is updated to an incremental flight database, which comprises the following steps:
[0016] After the full flight database is generated, when new flight information is pushed to the search engine through the queue, the search engine updates the new flight information to the incremental flight database.
[0017] Further, the incremental flight database adopts a linked list storage structure.
[0018] The linked list storage structure includes a linked list header and a linked list intermediate node.
[0019] The linked list header includes a flight departure place, a flight destination and a flight date.
[0020] The linked list intermediate node includes a flight number, a departure time, an arrival time, an airline code and an aircraft type.
[0021] Further, after the full flight database is generated, when new flight information is received, the following steps are included after updating to the incremental flight database:
[0022] When the full flight database is updated, the new flight information in the incremental flight database is merged into the full flight database, and the incremental flight database is emptied.
[0023] Further, the flight data query based on the full flight database and the incremental flight database includes the following steps:
[0024] An iterator is designed to integrate the full flight database and the incremental flight database.
[0025] Flight data query is performed based on the integrated full flight database and incremental flight database.
[0026] The application also discloses a flight data real-time effective device, which comprises:
[0027] An acquisition unit is configured to acquire flight plan data.
[0028] An analysis unit is configured to analyze the flight plan data.
[0029] A generation unit is configured to index the analyzed flight plan data and generate a full flight database.
[0030] An update unit is configured to update the full flight database after the full flight database is generated, and update the new flight information to the incremental flight database.
[0031] A query unit is configured to perform flight data query based on the full flight database and the incremental flight database.
[0032] The application further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the flight data real-time validation method when executing the computer program.
[0033] The application further discloses a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the flight data real-time validation method.
[0034] Compared with the prior art, the embodiment of the application has at least the following advantages:
[0035] The application analyzes flight plan data, then writes the flight plan data in a binary format into a memory mapping file, and indexes the analyzed flight plan data, and new flight information is stored in a linked list storage structure, so that the search efficiency of the flight search involved in the fare search system is greatly improved.
[0036] The linked list storage structure allows new flight information to be quickly inserted into the incremental flight database without reordering or index construction of the existing full flight data; each time new flight information arrives, only the linked list table header corresponding to the departure-destination and date needs to be found, and the new flight information can be efficiently inserted into the linked list.
[0037] Since the linked list table header is indexed according to the departure-destination and date, the system can quickly locate the incremental flight data of a certain specific departure-destination and date when querying, greatly improving the query efficiency when the full flight database has not been updated.
[0038] The separation design of the incremental flight database and the full flight database enables the system to continue processing and querying the latest flight information while the full flight database is being updated, and this incremental mechanism can effectively avoid the query delay or data inconsistency problem that may occur during the update of the full flight database.
[0039] Other features and advantages of the application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims. BRIEF DESCRIPTION OF DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and the other drawings can be obtained by those skilled in the art without creative effort based on these drawings.
[0041] Figure 1 A flow chart of a method for real-time validation of flight data according to an embodiment of the present application is shown;
[0042] Figure 2 A schematic diagram of a single-day flight data storage format according to an embodiment of the present application is shown;
[0043] Figure 3 A schematic diagram of a single-day flight data index format according to an embodiment of the present application is shown;
[0044] Figure 4 A schematic diagram of an incremental flight information storage format according to an embodiment of the present application is shown;
[0045] Figure 5 A schematic diagram of a device for real-time validation of flight data according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0046] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely explain the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort fall within the scope of protection of the present application.
[0047] Figure 1 A flow chart of a method for real-time validation of flight data according to an embodiment of the present application is shown. As shown in Figure 1 , the method for real-time validation of flight data proposed in the embodiment of the present application includes the following steps:
[0048] Obtaining flight plan data;
[0049] Analyzing the flight plan data;
[0050] Establishing an index for the analyzed flight plan data and generating a full flight database;
[0051] After the full flight database is generated, when new flight information is received, the new flight information is updated to the incremental flight database;
[0052] Flight data query is carried out based on full flight database and incremental flight database.
[0053] The application discloses a flight data real-time validation method, and provides a flight data validation method, especially a newly-built flight validation method.
[0054] The application analyzes flight plan data, writes the analyzed flight plan data into a memory mapping file in a binary format, indexes the analyzed flight plan data, and stores new flight information in a linked list storage structure, so that the flight search efficiency in a fare search system is greatly improved.
[0055] Flight plan data obtained from an SSIM message (a text message queue conforming to an SSIM (Standard Schedules Information Manual) format) channel is exported as a text file at a daily frequency and in a specific standard format.
[0056] The flight plan data exported at the daily frequency is used to generate a full flight database, and the full flight database is preprocessed with a large amount of flight-related information, such as whether the flight is a domestic or international flight, whether the flight is a stopover flight, and the stopover time calculation.
[0057] After the full flight database is generated, the data of the full flight database is copied multiple times and distributed to each search engine node.
[0058] As shown in FIG. Figure 2 In some embodiments, the step of analyzing the flight plan data comprises the following steps:
[0059] The analyzed flight plan data is converted into a binary format.
[0060] The converted flight plan data is written into a memory mapping file.
[0061] In order to improve the data reading efficiency and save the system memory, the parsed flight plan data is not directly stored in the traditional database, but after splitting, all specific date flight instances in the period are generated, which contain all original information related to the flight, such as flight number, departure date, arrival date, departure place and destination information, etc. in the form of binary, and are written into a memory-mapped file. The memory mapping technology can directly map the file to the system memory, so that when the data needs to be read, the flight records can be quickly accessed like operating memory. In the specific implementation, after the flight plan data is serialized, it is converted into a compact binary format. This binary format can greatly reduce the space occupied by the flight plan data and ensure the fast reading of the search service.
[0062] As shown in Figure 3 In some embodiments, the index includes a primary index and a secondary index;
[0063] The primary index primary key is the departure place and the departure date;
[0064] The secondary index primary key is the destination and the arrival date.
[0065] As shown in Figure 3 In order to realize efficient OD (origin-destination) query, the application designs two key indexes: one based on the departure place and the departure date, used for quickly locating the flight from a certain airport; the other based on the destination and the arrival date, used for querying the destination and arrival time of the flight. The value is a set of address pointers of flight data conforming to the primary index and the secondary index primary key. This double index mechanism ensures that users can quickly query the flight information from a certain place and arrive at the destination within a certain time range.
[0066] In some embodiments, after the full flight database is generated, when new flight information is received, the incremental flight database is updated, including the following steps:
[0067] After the full flight database is generated, when new flight information is pushed to the search engine through the queue, the search engine updates the new flight information to the incremental flight database.
[0068] The real-time new flight information is sent to the update unit of each search engine node in chronological order by the parsing unit, and the update unit stores the time sequence data to the incremental flight database. From the incremental flight database, the update data of all flights from the time when the full flight database is generated to the current time can be obtained.
[0069] Due to the extremely large amount of flight plan data in the past two years, parsing the original flight plan data and building the two indexes requires processing a large amount of data and takes a long time. When facing millions of flight records, the computing cost of frequent full flight database creation and index construction will increase significantly, and there is also a large resource consumption when the data push and search engine switches to the latest full flight database. Therefore, the present application adopts a regular update strategy, such as performing full flight database construction and update once a day.
[0070] However, the update of flight information is a continuous and dynamic process. During the period when the full flight database has not been updated, new flight information is still pushed through the queue. These new flight information cannot wait for the next execution of full flight database update, so the incremental flight database is introduced to process and store these new flight information during the period when the full flight database is not updated.
[0071] As shown in Figure 4 When new flight information is pushed to the search engine through the queue, the search engine updates the new flight information to the incremental flight database. The incremental flight database is specially used to quickly process newly added flight information, and adopts a linked list storage structure, which includes a linked list header and a linked list intermediate node. In the incremental flight database, each flight record is stored in the form of a linked list node, the linked list header contains the departure or destination information and the flight date of the flight as an index identifier. The linked list intermediate node stores detailed flight information, including flight number, departure time, arrival time, airline code, aircraft type and other data.
[0072] The linked list storage structure allows new flight information to be quickly inserted into the incremental flight database without reordering or index construction of the existing full flight data; each time new flight information arrives, only the linked list header corresponding to the departure-destination and date needs to be found, and the new flight information can be efficiently inserted into the linked list.
[0073] Since the linked list header is indexed according to the departure-destination and date, the system can quickly locate the incremental flight data of a specific departure and destination when querying, greatly improving the query efficiency when the full flight database has not been updated.
[0074] In some embodiments, after the full flight database is generated, when new flight information is received, the following steps are included after updating to the incremental flight database:
[0075] When the full flight database is updated, the new flight information in the incremental flight database is merged into the full flight database, and the incremental flight database is emptied to process the next round of incremental data push.
[0076] Through the combination of the full flight database and the incremental flight database, the integrity and timeliness of the full market flight data are ensured, and the real-time and reliability of the query service are ensured by quickly responding to changes in new flight information.
[0077] In some embodiments, the flight data query based on the full flight database and the incremental flight database includes the following steps:
[0078] An iterator is designed to integrate the full flight database and the incremental flight database;
[0079] Flight data query is performed based on the integrated full flight database and incremental flight database.
[0080] The application also provides an interface for obtaining all the latest full flight data and incremental flight data. The interface is transparent to the outside and implements the process of superimposing the full flight data and the incremental flight data. By designing a special iterator, the data in the full flight database and the new flight information in the incremental flight database are integrated. In this way, when the full flight database has not been updated, the latest flight information in the incremental flight database can also be queried.
[0081] Sequential access to elements: The iterator provides a method to access each element in the collection in order. It usually has a method such as next() to get the next element.
[0082] Maintain state: The iterator usually maintains the current access position of the flight inside. When next() is called, the iterator returns the next flight and updates its internal state.
[0083] Lazy loading: Generate the next flight on demand.
[0084] The separation design of the incremental flight database and the full flight database enables the system to continue processing and querying the latest flight information while the full flight database is being updated. This incremental mechanism can effectively avoid query delays or data inconsistency problems that may occur during the full flight database update process.
[0085] The method for real-time effect of flight data is described below.
[0086] The flight schedule data in the SSIM message usually contains periodic information, i.e. some flights only operate on specific days of the week, such as once a week on Monday, Wednesday, and Friday. This structure is convenient for description, but it is not suitable for direct storage and query. Therefore, when parsing the message, the periodic flights need to be expanded into specific daily flight records according to the start date and end date of the flight plan and combined with the periodic field, as shown in Table 1.
[0087] Table 1 Daily flight record
[0088]
[0089] Wherein, CA is China International Airlines; HU is Hainan Airlines; PEK is Beijing Capital International Airport; HKG is Hong Kong International Airport; BKK is Bangkok Suvarnabhumi International Airport; LHR is London Heathrow Airport.
[0090] After obtaining the flight plan data, the background system parses the data into the data structure described in the technical solution for storage. According to the departure airport, destination, departure date, and arrival date, an index based on the departure airport and departure date is established for quickly locating flights departing from a certain airport; another index based on the destination and arrival date is established for querying the destination and arrival time of the flight, as shown in Table 2.
[0091] Table 2 Index
[0092]
[0093] Wherein, SHA is Shanghai Hongqiao International Airport. The secondary index value is the corresponding flight number.
[0094] The queue pushes new flight information to the search engine, which updates the new flight information to the incremental flight database. For example, in Table 3, CX777 and CA111 have the same key, and are calculated to the bucket index by the hash function. The flight data is in the form of a linked list in the same hash bucket.
[0095] Table 3 New incremental flight database
[0096] Key Value PEK_20240928 CX777 (HKG) -> CA111 (BKK) PEK_20240929 MU123 (TPE) ... ...
[0097] Wherein, TPE is Taiwan Taoyuan International Airport.
[0098] Data usage:
[0099] a. Non-stop:
[0100] For example, a user searches for non-stop flights from Beijing to Hong Kong on September 28, 2024. First, find out which destinations have flights departing from Beijing on September 28 from the full flight database through the index. If there is Hong Kong among the destinations, all non-stop flights to Hong Kong can be found. From the incremental flight database, find flights departing from Beijing on September 28, and find flights with Hong Kong as the destination among the new flights. The superposition of the two parts of the flight data is the latest complete flight from Beijing to Hong Kong.
[0101] b. Connecting:
[0102] For example, a user searches for Beijing-Hong Kong flights on September 28. Take the departure date + departure place as an example:
[0103] 1) Find all new flights from Beijing on September 28 from the incremental flight database. Because the flight data in the incremental flight database is small, a secondary index can be established in real time according to the index structure of the full flight database and stored in the service memory. Use the iterator that integrates the data in the full flight database and the new flight data in the incremental flight database. When the iterator is used externally, the effect shown in Table 4 can be achieved:
[0104] Table 4 Query results with departure place + departure date as index
[0105]
[0106] 2) Take the destination as the starting point, and the flight arrival date or the arrival date + 1 day as the departure date for the next search, and continue to find all subsequent flights, as shown in Table 5.
[0107] Table 5 Query results with destination as starting point
[0108]
[0109] In the first subsequent flight search, Beijing-Hong Kong transfer connections are found: PEK-SHA-HKG: MU5126-CX567 or CA1563-CX567.
[0110] 3) Continue to use breadth-first search or bidirectional breadth-first search to find out if there are flights to Hong Kong. If so, you can get Beijing departure, Hong Kong transfer 2 times, 3 times, etc. Connection flights.
[0111] As the search depth increases, other optimization methods can be used to implement real-time flight connection building.
[0112] As shown in Figure 5 Based on the above flight data real-time effect method, the embodiment proposes a flight data real-time effect device, which includes:
[0113] An acquisition unit is configured to acquire flight plan data.
[0114] An analysis unit is configured to receive flight plan data, analyze the flight plan data, and distribute the analyzed flight plan data to each search engine node.
[0115] A generation unit is configured to index the analyzed flight plan data and generate a full flight database.
[0116] An updating unit is configured to update the incremental flight database with new flight information after the full flight database is generated, and the updating unit is deployed on each search engine node.
[0117] A querying unit is configured to query flight data based on the full flight database and the incremental flight database.
[0118] The application further provides an electronic device, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for making flight data effective in real time when executing the computer program.
[0119] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the method for making flight data effective in real time.
[0120] The application further provides a computer program product, comprising computer instructions, wherein the computer instructions are executable on a processor to implement the method for making flight data effective in real time.
[0121] Although the application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced equivalently, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.
Claims
1. A method for real-time validation of flight data, characterized in that: The following steps are involved: Get flight schedule data; Parsing flight schedule data; Create an index for the parsed flight plan data and generate a full flight database; After the full flight database is generated, when new flight information is received, it is updated to the incremental flight database; Flight data query is performed based on the full flight database and incremental flight database.
2. The method for real-time validation of flight data according to claim 1, characterized in that: The analysis of the flight plan data includes the following steps: Convert the parsed flight schedule data into binary format; Write the converted flight plan data into a memory-mapped file.
3. The method for real-time validation of flight data according to claim 1, characterized in that: The index includes a primary index and a secondary index; The primary key of the primary index is the departure place and departure date; The secondary index primary keys are destination and arrival date.
4. The method for real-time validation of flight data according to claim 1, characterized in that: After the full flight database is generated, when new flight information is received, updating it to the incremental flight database includes the following steps: After the full flight database is generated, when new flight information is pushed to the search engine through the queue, the search engine updates the new flight information to the incremental flight database.
5. The method for real-time validation of flight data according to claim 4, characterized in that: The incremental flight database adopts a linked list storage structure; The linked list storage structure includes a linked list header and a linked list intermediate node; The linked list header includes the flight departure place, flight destination and flight date; The intermediate nodes of the linked list include flight number, departure time, arrival time, airline code and aircraft model.
6. The method for real-time validation of flight data according to claim 1 or 4, characterized in that: After the full flight database is generated, when new flight information is received, the incremental flight database is updated and the following steps are included: When the full flight database is updated, the new flight information in the incremental flight database is merged into the full flight database, and the incremental flight database is cleared.
7. The method for real-time validation of flight data according to claim 1, characterized in that: The flight data query based on the full flight database and the incremental flight database includes the following steps: Design an iterator to integrate the full flight database and the incremental flight database; Flight data query is performed based on the integrated full flight database and incremental flight database.
8. A device for real-time validation of flight data, characterized in that: include: An acquisition unit, used to acquire flight plan data; A parsing unit, used to parse flight plan data; A generation unit, used to create an index for the parsed flight plan data and generate a full flight database; An update unit, which is used to update the incremental flight database when new flight information is received after the full flight database is generated; The query unit is used to query flight data based on the full flight database and the incremental flight database.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for real-time validation of flight data according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for real-time validation of flight data according to any one of claims 1 to 7 is implemented.