Data processing system, method, medium and equipment

Abstract

The invention provides a data processing system and method, a medium and equipment, and relates to the technical field of data processing.The data processing system comprises a data processing module, a first data storage module and a second data storage module, and the first data storage module is used for storing original flight data and key flight data obtained through updating according to the second data storage module; the second data storage module is used for storing the key flight data obtained according to the data processing module; compared with the prior art, the second data storage module is arranged, that is, the temporarily-arranged memory space is utilized, the number of times of interaction between the data processing module and the database is reduced, and due to the fact that the memory is extremely high in data reading and writing speed compared with the database, the overall data processing time is shortened, and the data processing speed is increased.

Description

Background Art

[0002] With the rapid development of information technology, the amount of data generated and accumulated in various industries has shown an explosive growth. In the calculation of civil aviation flight execution rate, it is necessary to modify the actual flight data according to the adjustment document and then calculate the flight execution rate to obtain a more accurate flight execution rate. However, since the original actual flight data is stored in the corresponding database, in the data processing process, whether it is obtaining data from the database or storing the processed data back to the database, a record-by-record processing method is often adopted. When the amount of data is small, the impact on the system performance is not obvious. But when the volume of data to be processed is large, the number of interactions between the data processing module and the database will increase sharply. For example, when processing sales data containing millions of records, if the record-by-record processing method is adopted, the data processing module frequently interacts with the database, and each interaction requires a series of operations such as network communication and database query parsing, which will undoubtedly consume a lot of time. Moreover, frequent interactions may also cause the database to be overloaded, affecting the normal access of other services to the database.

[0003] This inefficient data processing method not only prolongs the time of the entire data processing process, reduces the system response speed, but also may limit the scale of business development. For example, in the scenario of real-time data analysis, due to the low data processing efficiency, accurate data support cannot be provided in time for decision-making, thus affecting the timeliness and accuracy of enterprise decision-making. Therefore, how to optimize the interaction method between the data processing module and the database, reduce the number of interactions, and improve the data processing efficiency has become an urgent problem to be solved in the current data processing field. Summary of the Invention

[0004] In view of the above technical problems, the present application provides a data processing system, method, medium, and device, which at least partially solve the problems existing in the prior art.

[0005] In the first aspect of the present application, a data processing system is provided. The system includes: a data processing module, a first data storage module, and a second data storage module; wherein:

[0006] The first data storage module is used to store the original flight data and the key flight data updated according to the second data storage module; the key flight data is obtained by the data processing module after modifying the original flight data or the key flight data.

[0007] The second data storage module is used to store the key flight data obtained according to the data processing module; wherein, the storage space of the second data storage module is smaller than that of the first data storage module, and the read / write speed of the second data storage module is greater than that of the first data storage module; the second data storage module has a corresponding preset data quantity threshold; each key flight data in the second data storage module has a corresponding original flight data or key flight data in the first data storage module; if the quantity of the key flight data stored in the second data storage module is equal to the preset data quantity threshold, then update the original flight data corresponding to each key flight data stored in the first data storage module according to each key flight data stored in the second data storage module.

[0008] In a second aspect of the present application, a data processing method is provided, and the method includes:

[0009] S001, obtain a target query instruction; wherein, the target query instruction has a corresponding target unique identification sequence, a target departure location identification, a target destination location identification, and a target query time period;

[0010] S002, obtain a number of original flight data and / or key flight data from the first data storage module according to the target query instruction, and obtain a first data list Y = (Y1, Y2,..., Y i ,..., Y n ); i = 1, 2,..., n; where n is the number of original flight data and / or key flight data in the first data storage module whose corresponding executed time is within the target query time period, and the unique identification sequence, the departure location identification, and the destination location identification are all correspondingly the same as the target unique identification sequence, the target departure location identification, and the target destination location identification; Y i is the i-th original flight data or key flight data corresponding to the target query instruction in the first data storage module;

[0011] S003, obtain a number of key flight data from the second data storage module according to the target query instruction, and obtain a second data list E = (E1, E2,..., E j ,..., E m ); j = 1, 2,..., m; m is the number of key flight data in the second data storage module whose corresponding executed time is within the target query time period, and the unique identification sequence, the departure location identification, and the destination location identification are all correspondingly the same as the target unique identification sequence, the target departure location identification, and the target destination location identification; E j is the j-th key flight data corresponding to the target query instruction in the second data storage module; m ≤ n;

[0012] S004, update Y according to E to obtain a third data list Y’ = (Y1’, Y2’, …, Y i ’, …, Y n ’); where Y i ’ is the i-th original flight data and / or critical flight data; if the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to Y i ’ are the same as those corresponding to E j , then determine Y i ’ = E j ; if the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to Y i ’ are different from each critical flight data in E, then Y i ’ = Y i ; the third data list is used to process the target query instruction.

[0013] In the third aspect of the present application, a non-transitory computer-readable storage medium is provided, and at least one instruction or at least one program segment is stored in the storage medium. The at least one instruction or at least one program segment is loaded and executed by a processor to implement the foregoing data processing method.

[0014] In the fourth aspect of the present application, an electronic device is provided, including a processor and the above non-transitory computer-readable storage medium.

[0015] The present application has at least the following beneficial effects:

[0016] The data processing system provided by this application includes a data processing module, a first data storage module, and a second data storage module. Among them, the first data storage module can be a database, which is usually used for persistent storage of a large amount of data. The data is stored on non-volatile storage media such as hard disks. Even if the computer is powered off, the data will not be lost. It has a large storage capacity and can easily accommodate a large amount of data, but the data read and write speed is relatively slow. The second data storage module can be memory. As a temporary storage area of the computer, memory is used to temporarily store the data and program instructions that the CPU is currently processing. Its data read and write speed is extremely fast and can interact with the CPU at a high speed, but the capacity is relatively small. Since the storage space of the second data storage module is smaller than that of the first data storage module, after the second data storage module accumulates a preset number threshold of key flight data, each of the key flight data is written back to the first data storage module. That is, the original flight data corresponding to each key flight data is updated. Compared with the prior art, this application example sets up a second data storage module, that is, uses a temporarily set memory space to reduce the number of interactions between the data processing module and the database. Since the data read and write speed of memory is extremely fast compared with that of the database, the overall data processing time is reduced and the data processing speed is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a structural block diagram of the data processing system provided by the embodiment of this application;

[0019] Figure 2 It is a flowchart of the data processing method provided by the embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0021] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] It should be noted that the following describes various aspects of embodiments within the scope of the appended claims. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on this application, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement a device and / or practice a method. In addition, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.

[0023] Please refer to Figure 1 As shown, an embodiment of this application provides a data processing system 100, which is characterized in that the system includes: a data processing module 110, a first data storage module 120, and a second data storage module 130; wherein:

[0024] The first data storage module 120 is used to store the original flight data and the key flight data updated according to the second data storage module 130; the key flight data is obtained by the data processing module 110 after modifying the original flight data or the key flight data.

[0025] The second data storage module 130 is configured to store the critical flight data obtained according to the data processing module 110. Among them, the storage space of the second data storage module 130 is smaller than that of the first data storage module 120, and the read / write speed of the second data storage module 130 is greater than that of the first data storage module 120. The second data storage module 130 has a corresponding preset data quantity threshold. Each critical flight data in the second data storage module 130 has corresponding original flight data or critical flight data in the first data storage module 120. If the quantity of the critical flight data stored in the second data storage module 130 is equal to the preset data quantity threshold, the original flight data corresponding to each critical flight data stored in the first data storage module 120 is updated according to each critical flight data stored in the second data storage module 130.

[0026] Specifically, the data processing system provided in this application includes a data processing module, a first data storage module, and a second data storage module. The data processing module can calculate the flight execution rate according to the user's requirements. As an example, the flight execution rate of a certain flight in a certain flight season can be calculated. The flight execution rate of a certain flight in a certain flight season is the ratio of the actual number of flights of this flight in this flight season to the planned number of flights of this flight in this flight season. Among them, the actual number of flights of this flight in this flight season refers to the number of times this flight actually takes off and completes the flight according to the flight plan within this specific flight season. The planned number of flights of this flight in this flight season refers to the number of flights planned to be executed by the airline for this flight in this flight season before the start of this specific flight season, based on factors such as market demand, route planning, and resource allocation. However, during the actual execution process, the flight may be rescheduled or the number of flights may increase due to various factors. Therefore, for this flight in this flight season, there is a corresponding adjustment file, which records the time periods during which this flight has been rescheduled or the number of flights has increased in this flight season.

[0027] In addition, to improve the accuracy of the flight execution rate calculation, it is necessary to modify some relevant original flight data according to the adjustment file. As an example: Flight A is scheduled to fly on Mondays, Wednesdays, and Fridays every week during the flight season from March 2024 to September 2024. However, due to certain reasons, during a certain week between March 2024 and September 2024, Flight A was adjusted to fly on Tuesdays, Thursdays, and Saturdays (as recorded in the adjustment file), and Flight A also flew normally on Tuesdays, Thursdays, and Saturdays during that week. At this time, when calculating the flight execution rate of Flight A during the flight season from March 2024 to September 2024, if the actual flight data for the adjusted week is not modified to Mondays, Wednesdays, and Fridays, these three flight execution data will be omitted. Therefore, it is necessary to adjust the actual flight data for the adjusted week back to Tuesdays, Thursdays, and Saturdays. Thus, the modification of some relevant original flight data according to the adjustment file in this embodiment is as described in the above example.

[0028] When generating the target query instruction, the data processing module can, according to each original flight data corresponding to the target query instruction, if there is matching adjustment data in the adjustment file, obtain the corresponding original flight data in the first data storage module and generate the corresponding target modification instruction. That is, each target query instruction has a corresponding target modification instruction. After the corresponding target modification instruction is executed, the data correction is completed. At this time, the data query result obtained based on the corrected data is more accurate, and the final flight execution rate calculation result is also more accurate.

[0029] Furthermore, the first data storage module in this embodiment can be a database. A database is usually used to persistently store a large amount of data, and the data is stored on non-volatile storage media such as hard disks. Even if the computer is powered off, the data will not be lost. It has a large storage capacity and can easily accommodate a large amount of data, but the data reading and writing speed is relatively slow. The first data storage module is used to store the original flight data, planned flight data, and adjustment files corresponding to multiple flights. Among them, the original flight data is the actual flight data. And each flight data can include multiple fields. As an example: it includes information such as flight number, airport identification of the departure airport, airport identification of the arrival airport, actual flight date, and corresponding actual flight time.

[0030] The second data storage module is the memory. As the temporary storage area of the computer, the memory is used to temporarily store the data and program instructions that the CPU is currently processing. It has an extremely fast data reading and writing speed and can interact with the CPU at a high speed, but its capacity is relatively small. In this embodiment, the second data storage module is used to store the key flight data obtained according to the data processing module. Here, the key flight data is the flight data obtained by modifying the original flight data according to the adjustment file. Therefore, each piece of key flight data in the second data storage module has a corresponding original flight data in the first data storage module. Moreover, the storage space of the memory is smaller than that of the database. The second data storage module also has a corresponding preset data quantity threshold. As an example, the preset data quantity threshold is 1000. That is, whenever the number of key flight data stored in the second data storage module is equal to 1000, the key flight data stored in the second data storage module is written back to the first data storage module. That is, the original flight data corresponding to each piece of key flight data is updated.

[0031] It should be noted that, as can be seen from the above, the first data storage module stores not only the original flight data but also the key flight data, and the key flight data here comes from the second data storage module.

[0032] In summary, compared with the prior art, this embodiment sets up a second data storage module, that is, by using the temporarily set memory space, the number of interactions between the data processing module and the database is reduced. Since the data reading and writing speed of the memory is extremely fast compared with the database, the overall data processing time is reduced and the data processing speed is improved.

[0033] In an exemplary embodiment of the present application, each piece of original flight data and each piece of key flight data have corresponding unique identification sequences, departure location identifiers, destination location identifiers, and execution times; the data processing module is used to perform the following steps:

[0034] S110, obtain a target query instruction; wherein, the target query instruction has a corresponding target unique identification sequence, target departure location identifier, target destination location identifier, and target query time period.

[0035] Among them, each piece of original flight data and each piece of key flight data have corresponding unique identification sequences, departure location identifiers, destination location identifiers, and execution times. The unique identification sequence can be the flight number, the departure location identifier can be the airport identifier of the departure airport, the destination location identifier can be the airport identifier of the landing airport, and the execution time can be the execution time of the flight.

[0036] Here, the target query instruction is the query instruction determined by the data processing module according to the user's needs. As an example: the target query instruction can be to obtain the actual flight data of flight A from airport A to airport B from March 2024 to September 2024. Here, the target query instruction has a corresponding target unique identification sequence, target departure location identification, target destination location identification, and target query time period.

[0037] S120, obtain a number of original flight data and / or key flight data from the first data storage module according to the target query instruction, and obtain the first data list Y = (Y1, Y2,..., Y i ,..., Y n ); i = 1, 2,..., n; where n is the number of original flight data and / or key flight data in the first data storage module whose corresponding execution time is within the target query time period, and the unique identification sequence, departure location identification, and destination location identification are all the same as the target unique identification sequence, target departure location identification, and target destination location identification; Y i is the i-th original flight data or key flight data corresponding to the target query instruction in the first data storage module.

[0038] Specifically, each original flight data and / or key flight data in the first data list is the original flight data and / or key flight data in the first data list whose corresponding execution time is within the target query time period, and the unique identification sequence, departure location identification, and destination location identification are all the same as the target unique identification sequence, target departure location identification, and target destination location identification.

[0039] S130, obtain a number of key flight data from the second data storage module according to the target query instruction, and obtain the second data list E = (E1, E2,..., E j ,..., E m ); j = 1, 2,..., m; m is the number of key flight data in the second data storage module whose corresponding execution time is within the target query time period, and the unique identification sequence, departure location identification, and destination location identification are all the same as the target unique identification sequence, target departure location identification, and target destination location identification; E j is the j-th key flight data corresponding to the target query instruction in the second data storage module; m ≤ n.

[0040] S140, if E is not empty, then update Y according to E to obtain the third data list Y' = (Y1', Y2',..., Y i ',..., Y n '); where Y i ' is the i-th original flight data and / or key flight data; if Y i'The corresponding unique identification sequence, departure location identification, destination identification and elapsed time are the same as E j If the corresponding unique identification sequence, departure location identification, destination identification and elapsed time are the same, then determine Y i '=E j If Y i 'The corresponding unique identification sequence, departure location identification, destination identification and execution time are different from each key flight data in E, then Y i '=Y i ; The third data list is used to process the target query instruction.

[0041] Specifically, since each time the data processing module modifies the data, the key flight data obtained is first stored in the second data storage module, the data in the second data storage module is more accurate than the data in the second data storage module. For example, the original flight data of flight A from Airport A to Airport B on May 18, 2024 is stored in the first data storage module, but due to the record of the adjustment file, May 18, 2024 should be modified to May 17, 2024. The key flight data obtained after the modification of this original flight data may be stored in the second data storage module. And the ID corresponding to this data has not been changed (unique identification sequence, departure place identification, destination identification and execution time), therefore, if there is data with the same ID in the second data storage module and the first data storage module at the same time, the winning data in the obtained third data list adopts the corresponding data in the second data storage module. Thus, an updated third data list is obtained, wherein the third data list may contain some original flight data from the first data storage module, which has not been modified; and there may also be some key flight data from the first data storage module, which has been modified by the data processing module, and falls back to the first data storage module after the storage of the second data storage module reaches the upper limit. At this time, the data does not exist in the second data storage module; in addition, there may also be some key flight data from the second data storage module, which has been modified by the data processing module and temporarily stored in the second data storage module, and has not yet fallen back to the first data storage module, but in the first data storage module, there are corresponding original flight data or key flight data (modified once and again). Therefore, for this part of the data, the data in the second data storage module should be trusted.

[0042] The method provided in this embodiment makes the query result, that is, the third data list, more accurate and can accurately identify the modified data, so that the query result finally returned is more accurate and the subsequent calculation results are also more accurate.

[0043] In an exemplary embodiment of the present application, the second data storage module is further configured to store the calculation result data obtained by the data processing module through the flight execution rate calculation process.

[0044] In an exemplary embodiment of the present application, each piece of original flight data or critical flight data includes several fields; the second data storage module is further configured to perform the following steps:

[0045] S210, in response to detecting that the number of critical flight data stored in the second data storage module is equal to the preset data quantity threshold, obtain each piece of critical flight data in the second data storage module to obtain a critical flight data list G=(G1, G2,..., G x ,..., G y ); x = 1, 2,..., y; where y is the preset data quantity threshold corresponding to the second data storage module; G x is the x-th piece of critical flight data stored in the second data storage module.

[0046] Specifically, each piece of original flight data or critical flight data includes several fields; if it is detected that the number of critical flight data stored in the second data storage module is equal to the preset data quantity threshold, it indicates that the space in the second data storage space has been fully occupied at this time. At this time, it is necessary to move the critical flight data stored in the second data storage module back to the first data storage module with a larger storage space.

[0047] S220, based on G, obtain several first critical clustering clusters to obtain a first clustering cluster list YJ=(YJ1, YJ2,..., YJ a ,..., YJ b ); a = 1, 2,..., b; where b is the number of first critical clustering clusters; YJ a is the cluster identifier of the a-th first critical clustering cluster; all the field identifiers of the modified fields of the critical flight data included in YJ a are the same.

[0048] Specifically, the data stored in the second data storage module is the updated data obtained after being modified according to the target modification instruction. Here, the data can be full-scale data, that is, the entire data is replaced, or only the modified fields are stored. Furthermore, clustering is performed according to the field identifiers of the modified fields, that is, first, the critical flight data that has modified the same field is clustered into one category. Those that have modified the departure airport are grouped into one category, and those that have modified the execution date are grouped into one category.

[0049] S230, based on YJ, obtain a second critical clustering cluster list set EJ=(EJ1, EJ2,..., EJ a ,..., EJ b ); where EJ aThe second critical clustering cluster list obtained by clustering all the critical flight data within the a-th first critical clustering cluster; EJ a =(EJ a，1 , EJ a，2 , …, EJ a，c , …, EJ a，f(a) ); c = 1, 2, …, f(a); f(a) is the number of the second critical clustering clusters corresponding to EJ a ; EJ a，c is the cluster identifier of the c-th second critical clustering cluster obtained by clustering all the critical flight data within the a-th first critical clustering cluster; all the modified field values of all the critical flight data included in EJ a，c are the same.

[0050] Furthermore, after the first clustering, secondary clustering is performed within each first critical clustering cluster. The secondary clustering clusters those critical flight data that have modified the same field and have the same modified field value into one class.

[0051] S240. According to EJ, obtain the data update instruction list set ZG = (ZG1, ZG2, …, ZG a , …, ZG b ); where ZG a is the data update instruction list corresponding to EJ a ; ZG a =(ZG a，1 , ZG a，2 , …, ZG a，c , …, ZG a，f(a) ); ZG a，c is the data update instruction corresponding to EJ a，c ; the data update instructions corresponding to any two second critical clustering clusters are different.

[0052] Here, each second critical clustering cluster has a corresponding data update instruction, and this data update instruction is used to control the modification of each data corresponding to the above critical flight data in the first data storage module.

[0053] S250. According to ZG, update the original flight data and / or critical flight data stored in the first data storage module.

[0054] In this embodiment, for a class of critical flight data that modifies the same field and has the same modified field value, a modification instruction is generated, which reduces the occupation of computing resources and improves the efficiency of data update.

[0055] In an exemplary embodiment of the present application, the second data storage module is used to store the target modification instructions sent by the data processing module; the target modification instructions are used to modify the original flight data and / or critical flight data corresponding to the target query instruction in the first data storage module when the number of target modification instructions stored in the second data storage module is less than the preset instruction number threshold and the data processing module executes the target query instruction; and when the number of target modification instructions stored in the second data storage module is equal to the preset instruction number threshold, update the original flight data corresponding to each target modification instruction stored in the first data storage module according to each target modification instruction stored in the second data storage module.

[0056] Specifically, after step S120, the data processing module is used to perform the following steps:

[0057] S150, if the second data storage module is not empty, obtain the target modification instruction list X = (X1, X2,..., X d ,..., X e ); d = 1, 2,..., e; where e is the number of target modification instructions included in the second data storage module; X d is the d-th target modification instruction included in the second data storage module.

[0058] S160, update Y according to the target modification instruction to obtain the third data list Y' = (Y1', Y2',..., Y i ',..., Y n '); where Y i ' is the updated original flight data or critical flight data of Y i ; if the unique identification sequence, departure place identification, and destination identification corresponding to Y i ' are the same as those corresponding to X d , and the executed time corresponding to Y i ' is within the modification time period corresponding to X d ; then update Y d according to X i to obtain Y i '; if the unique identification sequence, departure place identification, destination identification, and executed time corresponding to Y i ' are all different from those corresponding to X d , then determine Y i ' = Y i .

[0059] S170, execute the target query instruction according to Y'.

[0060] In this embodiment, the second data storage module stores the target modification instruction corresponding to the target query instruction. At this time, when the data processing module executes the corresponding target query instruction, it first filters and matches the original flight data or key flight data in the first data storage module. (That is, the original flight data and / or key flight data whose execution time is within the target query time period and whose unique identification sequence, departure location identification, and destination location identification are all correspondingly the same as the target unique identification sequence, target departure location identification, and target destination location identification). At this time, if the second data storage module is not empty, it indicates that some data in Y may need to be modified. At this time, the partial data in the first data storage module is modified according to the target modification instruction in the second data storage module to obtain the third data list. Finally, the target query instruction is executed according to the third data list, and the data corresponding to the target query instruction obtained is more accurate. Therefore, the result of the subsequent flight execution rate calculation is also more accurate.

[0061] In an exemplary embodiment of the present application, the second data storage module is further configured to perform the following steps:

[0062] S260, if the number of key flight data stored in the second data storage module is less than the preset data quantity threshold, and the second data storage module receives the target modification instruction from the data processing module; wherein, the target modification instruction has a corresponding unique identification sequence, departure location identification, destination location identification, and modification time period; and the target modification instruction has a corresponding field to be modified and a modification value corresponding to the field to be modified; each key flight data has a corresponding unique identification sequence, departure location identification, destination location identification, and execution time.

[0063] S270, obtain the target key flight data list MG=(MG1, MG2,..., MG x ,..., MG y ); x = 1, 2,..., y; where y is the number of target key flight data corresponding to the target modification instruction in the second data module; MG x is the xth target key flight data corresponding to the target modification instruction in the second data module; the unique identification sequence, departure location identification, and destination location identification of the target key flight data are all correspondingly the same as the unique identification sequence, departure location identification, and destination location identification corresponding to the target modification instruction; and the execution time of the target key flight data is within the modification time period corresponding to the target modification instruction, and the modification value corresponding to the field to be modified of the target modification instruction is different from the value of the key field corresponding to the target key flight data; the key field is the same field as the field to be modified corresponding to the target modification instruction.

[0064] S280. Modify the values of the keyword fields corresponding to each target critical flight data in the MG according to the modification values of the fields to be modified corresponding to the target modification instruction, so that the values of the keyword fields corresponding to each target critical flight data are the same as the modification values of the fields to be modified corresponding to the target modification instruction.

[0065] In this embodiment, in some special cases, when the critical flight data is still stored in the second data storage module and has not fallen into the first data storage module, the already modified critical flight data may be modified a second time. At this time, modify the critical flight data in the modification cache, so that when the critical flight data in the second data storage module falls back into the first data storage module, a corresponding modification instruction is generated to modify the original flight data or critical flight data in the first data storage module, so as to save resource occupancy and avoid frequent interaction between the data processing module and the database (the first data storage module). Save data processing time.

[0066] In an exemplary embodiment of the present application, the second data storage module is provided with a first storage unit and a second storage unit; wherein, the first storage unit is used to store calculation result data and original flight data not included in the first data storage module; the second storage unit is used to store critical flight data obtained according to the data processing module.

[0067] Specifically, the calculation result data and the original flight data not included in the first data storage module are new data for the first data storage module, while the critical flight data obtained according to the data processing module is modified data for the first data storage module. In the second data storage module of this embodiment, the new data and the modified data are stored separately, and all the new data in the first storage unit corresponds to one instruction when it falls into the first data storage module. And in one embodiment, the new data is preferentially processed.

[0068] Please refer to Figure 2 As shown, the embodiment of the present application provides a data processing method, which is applied to the above data processing system. The method includes:

[0069] S001. Obtain a target query instruction; wherein, the target query instruction has a corresponding target unique identifier sequence, a target departure location identifier, a target destination location identifier, and a target query time period.

[0070] S002. Obtain a number of original flight data and / or critical flight data from the first data storage module according to the target query instruction, and obtain a first data list Y=(Y1, Y2,..., Y i ,…, Y n); i = 1, 2, …, n; where n is the number of original flight data and / or critical flight data in the first data storage module whose corresponding execution time is within the target query time period, and the unique identification sequence, departure location identification, and destination location identification are all correspondingly the same as the target unique identification sequence, target departure location identification, and target destination location identification; Y i is the i-th original flight data or critical flight data corresponding to the target query instruction in the first data storage module.

[0071] S003. Obtain a number of critical flight data from the second data storage module according to the target query instruction, and obtain the second data list E = (E1, E2, …, E j , …, E m ); j = 1, 2, …, m; m is the number of critical flight data in the second data storage module whose corresponding execution time is within the target query time period, and the unique identification sequence, departure location identification, and destination location identification are all correspondingly the same as the target unique identification sequence, target departure location identification, and target destination location identification; E j is the j-th critical flight data corresponding to the target query instruction in the second data storage module; m ≤ n.

[0072] S004. Update Y according to E to obtain the third data list Y’ = (Y1’, Y2’, …, Y i ’, …, Y n ’); where Y i ’ is the i-th original flight data and / or critical flight data; if the unique identification sequence, departure location identification, destination location identification, and execution time corresponding to Y i ’ are the same as those corresponding to E j , then determine Y i ’ = E j ; if the unique identification sequence, departure location identification, destination location identification, and execution time corresponding to Y i ’ are all different from those of each critical flight data in E, then Y i ’ = Y i ; The third data list is used to process the target query instruction.

[0073] In an exemplary embodiment of the present application, an electronic device capable of implementing the above method is further provided.

[0074] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, various aspects of the present application can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuits", "modules", or "systems" here.

[0075] An electronic device according to this embodiment of the present application. The electronic device is only an example and should not impose any limitations on the functions and usage scopes of the embodiments of the present application.

[0076] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one of the above-mentioned processors, at least one of the above-mentioned memories, and a bus connecting different system components (including the memory and the processor).

[0077] Among them, the memory stores program codes, and the program codes can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present application described in the "Exemplary Method" section of this specification.

[0078] The memory may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory, and may further include a read-only memory (ROM).

[0079] The memory may also include a program / utility having a set (at least one) of program modules. Such program modules include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0080] The bus may represent one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure in a variety of bus structures.

[0081] The electronic device can also communicate with one or more external devices (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device that enables the electronic device to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface. Moreover, the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter. As shown in the figure, the network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0082] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0083] In an exemplary embodiment of the present application, there is also provided a computer-readable storage medium, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementation manners, various aspects of the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of the present specification.

[0084] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0085] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0086] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0087] The program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or, may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0088] In addition, the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, rather than for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0089] It should be noted that although several modules or units of the device for performing actions are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more of the above-mentioned modules or units may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided and embodied by multiple modules or units.

[0090] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A data processing system, characterized in that, The data processing system includes: a data processing module, a first data storage module, and a second data storage module; where: The first data storage module is used to store original flight data and critical flight data updated according to the second data storage module; the critical flight data is obtained by the data processing module after modifying the original flight data or critical flight data. The second data storage module is used to store critical flight data obtained according to the data processing module; wherein, the storage space of the second data storage module is smaller than that of the first data storage module, and the read / write speed of the second data storage module is greater than that of the first data storage module; the second data storage module has a corresponding preset data quantity threshold; each critical flight data in the second data storage module has a corresponding original flight data or critical flight data in the first data storage module; if the quantity of critical flight data stored in the second data storage module is equal to the preset data quantity threshold, then the original flight data corresponding to each critical flight data stored in the second data storage module is updated in the first data storage module.

2. The data processing system according to claim 1, wherein Each original flight data and each critical flight data have corresponding unique identification sequences, departure location identifications, destination location identifications, and executed times; the data processing module is used to perform the following steps: S110, obtain a target query instruction; wherein, the target query instruction has a corresponding target unique identification sequence, target departure location identification, target destination location identification, and target query time period. S120. Obtain a number of original flight data and / or critical flight data from the first data storage module according to the target query instruction, and obtain the first data list Y = (Y1, Y2,..., Y i ,..., Y n ); i = 1, 2,..., n; where n is the number of original flight data and / or critical flight data in the first data storage module whose corresponding executed time is within the target query time period, and the unique identification sequence, departure place identification, and destination identification are all correspondingly the same as the target unique identification sequence, target departure place identification, and target destination identification; Y i is the i-th original flight data or critical flight data corresponding to the target query instruction in the first data storage module; S130. Obtain a number of key flight data from the second data storage module according to the target query instruction, and obtain the second data list E = (E1, E2,..., E j ,..., E m ); j = 1, 2,..., m; m is the number of key flight data in the second data storage module whose corresponding executed time is within the target query time period, and the unique identification sequence, departure place identification, and destination identification are all correspondingly the same as the target unique identification sequence, target departure place identification, and target destination identification; E j is the j-th key flight data corresponding to the target query instruction in the second data storage module; m ≤ n; S140. If E is not empty, update Y according to E to obtain a third data list Y'=(Y1', Y2',..., Y i ',..., Y n '); where Y i ' is the i-th original flight data and / or critical flight data; if the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to Y i ' are the same as those of the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to E j , then determine that Y i ' = E j ; if the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to Y i ' are different from each critical flight data in E, then Y i ' = Y i ; The third data list is used to process the target query instruction.

3. The data processing system according to claim 2, wherein The first data storage module is further used to store schedule data, adjustment file data, and calculation result data; wherein, the schedule data is used to calculate the flight execution rate, and the adjustment file data is used to generate a target modification instruction to modify the original flight data and / or critical flight data.

4. The data processing system according to claim 3, wherein The data processing module is further used to perform the following steps: S150, perform flight execution rate calculation processing according to the schedule data corresponding to the target query instruction and Y'; and send the calculation result data obtained from the flight execution rate calculation processing to the second data storage module for storage.

5. The data processing system according to claim 4, wherein The second data storage module is further used to store the calculation result data obtained by the data processing module for flight execution rate calculation processing.

6. The data processing system according to claim 2, wherein, Each original flight data or critical flight data includes several fields; the second data storage module is further used to perform the following steps: S210. In response to detecting that the number of critical flight data stored in the second data storage module is equal to the preset data quantity threshold, obtain each piece of critical flight data in the second data storage module to obtain a list of critical flight data G = (G1, G2, …, G x , …, G y ); x = 1, 2, …, y; where y is the preset data quantity threshold corresponding to the second data storage module; G x is the x-th piece of critical flight data stored in the second data storage module; S220, obtain a number of first key clustering clusters according to G, so as to obtain a first clustering cluster list YJ = (YJ1, YJ2, …, YJ a , …, YJ b ); a = 1, 2, …, b; where b is the number of first key clustering clusters; YJ a is the cluster identifier of the a-th first key clustering cluster; all the field identifiers of the modified fields of the key flight data included in YJ a are the same; S230. According to YJ, obtain the second key clustering cluster list set EJ = (EJ1, EJ2,..., EJ a ,..., EJ b ); where EJ a is the second key clustering cluster list obtained by clustering all the key flight data in the a-th first key clustering cluster; EJ a = (EJ a，1 , EJ a，2 ,..., EJ a，c ,..., EJ a，f(a) ); c = 1, 2,..., f(a); f(a) is the number of the second key clustering clusters corresponding to EJ a ; EJ a，c is the cluster identifier of the c-th second key clustering cluster obtained by clustering all the key flight data in the a-th first key clustering cluster; the values of the modified fields of all the key flight data included in EJ a，c are the same; S240. Obtain a data update instruction list set ZG = (ZG1, ZG2,..., ZG a ,..., ZG b ); where ZG a is the data update instruction list corresponding to EJ a ; ZG a = (ZG a，1 , ZG a，2 ,..., ZG a，c ,..., ZG a，f(a) ); ZG a，c is the data update instruction corresponding to EJ a，c ; the data update instructions corresponding to any two second key clustering clusters are different; S250, update the original flight data and / or critical flight data stored in the first data storage module according to ZG.

7. A data processing method, characterized in that, The data processing method is applied to the data processing system according to any one of claims 1-6, and the method includes: S001, obtain a target query instruction; wherein, the target query instruction has a corresponding target unique identification sequence, target departure location identification, target destination location identification, and target query time period. S002, obtain a number of original flight data and / or key flight data from the first data storage module according to the target query instruction, and obtain the first data list Y = (Y1, Y2, …, Y i , …, Y n ); i = 1, 2, …, n; where n is the number of original flight data and / or key flight data in the first data storage module whose corresponding executed time is within the target query time period, and the unique identification sequence, departure place identification, and destination identification are all correspondingly the same as the target unique identification sequence, target departure place identification, and target destination identification; Y i is the i-th original flight data or key flight data corresponding to the target query instruction in the first data storage module; S003, obtain a number of key flight data from the second data storage module according to the target query instruction, and obtain the second data list E = (E1, E2, …, E j , …, E m ); j = 1, 2, …, m; m is the number of key flight data in the second data storage module whose corresponding executed time is within the target query time period, and the unique identification sequence, departure place identification, and destination identification are all corresponding and identical to the target unique identification sequence, target departure place identification, and target destination identification; E j is the j-th key flight data corresponding to the target query instruction in the second data storage module; m ≤ n; S004, update Y according to E to obtain the third data list Y’ = (Y1’, Y2’, …, Y i ’…, Y n ’); where Y i ’ is the i-th original flight data and / or critical flight data; if the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to Y i ’ are the same as those of the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to E j , then determine Y i ’ = E j ; if the unique identification sequence, departure location identification, destination location identification, and executed time corresponding to Y i ’ are different from each critical flight data in E, then Y i ’ = Y i ; the third data list is used to process the target query instruction.

8. A non-transitory computer-readable storage medium, characterized in that, At least one instruction or at least one program segment is stored in the storage medium, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the method according to claim 7.

9. An electronic device, characterized in that, Comprising a processor and the non-transitory computer-readable storage medium described in claim 8.

Images