Dynamic multi-task data checking method and system, medium and system
By employing a dynamic multi-task data inspection method and utilizing operation and maintenance interfaces and timestamp comparison technology, the problem of delayed timeliness in inspecting massive amounts of data at large hub airports has been solved, achieving efficient and accurate data consistency checks and reducing system operation risks.
Patent Information
- Application Number
- CN202511787000.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-01
- Publication Date
- 2026-02-27
AI Technical Summary
Existing technologies in departure systems of large hub airports face challenges such as delayed timeliness of massive data inspections, inflexible inspection results, and difficulty in meeting the consistency inspection requirements of various types of data, resulting in high system operation risks.
A dynamic multi-task data inspection method is adopted, which dynamically adjusts task configuration through the operation and maintenance interface, and combines timestamp comparison with historical records to achieve incremental inspection, reduce invalid inspection, and improve inspection efficiency and accuracy.
It has achieved automated and intelligent data consistency checks, which has reduced system operation risks, improved check speed and accuracy, and reduced waste of computing resources.
Smart Images
Figure CN121579924A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of data checking, and particularly relates to a dynamic multi-task data checking method, system, medium and system. BACKGROUND
[0002] As a leading aviation system service provider in China, the air traffic system supports the normal operation of the core business of many domestic airlines. Currently, with the process of migrating the core host systems such as departure and seat booking to an open platform, there are a large number of associated software and cross-domain systems involved between the open departure system and the original host departure system. This results in the existence of multiple isomorphic or heterogeneous business data, and the consistency between these data faces serious challenges. Especially in large hub airports with millions of passenger throughput, the amount of business data is extremely large.
[0003] In the existing data checking method, there are obvious defects, including a part still relying on manual checking, which is low in efficiency and prone to errors; another part of the automatic solution mostly adopts a single, full-quantity checking mode. This full-quantity checking method takes a very long time when facing massive data (for example, it may take about 1 hour to perform a full-quantity check on an airport with 20,000 people per day and 200 flights per day), resulting in a serious lag in the timeliness of the checking results. In addition, the existing method is usually difficult to flexibly and simultaneously check multiple types of data (such as flight, passenger, ticket, and load allocation), and cannot meet the comprehensive requirements of large hub airports for checking speed, accuracy, and flexibility.
[0004] The departure link is the final link for airlines to deliver services to passengers, involving flight data, passenger data, ticket data, additional service data, and other information, and there is a close correlation between these business data. Therefore, ensuring the consistency of data between related systems is a key goal to ensure the stable and reliable operation of the departure system. Implementing automatic, intelligent, and efficient data consistency checking can effectively reduce system operation risks and reduce the probability of faults caused by inconsistent data. SUMMARY
[0005] The present application provides a dynamic multi-task data checking method, system, medium and system to overcome the deficiencies of the prior art.
[0006] In a first aspect, the method comprises, starting a data checking task, reading the current task configuration, and determining the task type of the data checking task; According to the determined task type, dynamically adjusting the task configuration through the operation and maintenance interface, and obtaining the corresponding checkpoint from the modified task configuration; According to the modified task configuration, the to-be-checked data is acquired, consistency comparison verification is performed on the acquired to-be-checked data, a verification result and compared data are acquired; The verification result is analyzed, and according to the analysis result, the compared data is recorded, a record timestamp is generated and stored; The data update situation is monitored at a preset time interval, and according to the data update situation, the data checking task is cyclically run.
[0007] Further, the starting data checking task further comprises, before starting the data checking task, initializing the task running environment, and loading the default or last checking task configuration.
[0008] Further, the recording of the compared data according to the analysis result specifically comprises, the consistent data in the analysis result is directly recorded; the inconsistent data in the analysis result is traced, and a modification suggestion is given, and the inconsistent data and the modification suggestion are recorded.
[0009] Further, the cyclically running of the data checking task according to the data update situation specifically comprises, when the data is not updated, the data update situation is continuously monitored; otherwise, when the data is updated, it is judged whether there is historical record of the same type in the database, and whether the current time is greater than the timestamp recorded by the last task, according to the judgment result, the data checking task is triggered.
[0010] Further, the triggering of the data checking task according to the judgment result specifically comprises, when there is no historical data of the same type in the database, an incremental to-be-checked task list is directly formed, and a new round of data checking task cycle is triggered; when there is historical data of the same type in the database, and the current time is greater than the timestamp recorded by the last task, an incremental to-be-checked task list is formed, and a new round of data checking task cycle is triggered; otherwise, when there is historical data of the same type in the database, and the current time is less than the timestamp recorded by the last task, the data checking task is not triggered.
[0011] The second aspect, the system comprises: a task starting module, an operation and maintenance module, a data checking module, a data storage module and a cyclic running module; The task starting module is used for starting the data checking task, reading the current task configuration, and determining the task type of the data checking task; The operation and maintenance module is used for dynamically adjusting the task configuration through the operation and maintenance interface according to the determined task type, and acquiring the corresponding checking point from the modified task configuration; The data checking module is configured to acquire the data to be checked according to the modified task configuration, and perform consistency comparison and verification on the acquired data to be checked, acquire a verification result and compared data. The data storage module is configured to analyze the verification result, and record the compared data according to an analysis result, generate and store a record timestamp. The cycle running module is configured to monitor data update at a preset time interval, and cyclically run the data checking task according to the data update.
[0012] Further, the task starting module is further configured to initialize a task running environment before starting the data checking task, and load a default or last checking task configuration.
[0013] Further, the data storage module is specifically configured to, record consistent data in the analysis result directly, trace inconsistent data in the analysis result, and give a modification suggestion, and record the inconsistent data and the modification suggestion.
[0014] Further, the cycle running module is specifically configured to, when the data is not updated, continue to monitor the data update, otherwise, when the data is updated, determine whether there is historical record of the same type in the database, and compare whether the current time is greater than a timestamp recorded by a last task, according to a determination result, trigger the data checking task.
[0015] Further, the cycle running module is further configured to, when there is no historical data of the same type in the database, directly form an incremental task list to be checked, and trigger a new cycle of the data checking task, when there is historical data of the same type in the database, and the current time is greater than the timestamp recorded by the last task, form the incremental task list to be checked, and trigger the new cycle of the data checking task, otherwise, when there is historical data of the same type in the database, and the current time is less than the timestamp recorded by the last task, do not trigger the data checking task.
[0016] In a third aspect, a computer readable storage medium has a computer program stored therein, and the computer program is executed by a processor to implement the steps of any of the above dynamic multi-task data checking methods.
[0017] In a fourth aspect, an electronic device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication among each other through the communication bus. The memory is configured to store a computer program. Processor, for implementing the steps of the dynamic multi-task data checking method described above when executing the program stored on the memory.
[0018] Compared with the prior art, the present application has the following advantages: The present application realizes efficient, accurate and automated operation of data checking tasks by introducing dynamic configuration adjustment and intelligent triggering mechanism, can dynamically adjust the configuration according to the type through the operation and maintenance interface at the task start time, enhances the flexibility; and uses timestamp comparison and historical record judgment to intelligently identify data update state, so as to avoid invalid checking when the data does not change, and only check the changed part when the data is incrementally updated, greatly reducing the waste of computing resources and significantly improving the efficiency of large-scale data checking. Through the consistency comparison verification and inconsistent data tracing function, the quality and reliability of the data are effectively guaranteed.
[0019] 2. Through the closed loop formed by the cycle monitoring of the preset time interval, the whole process automation from task initialization, data acquisition, intelligent checking to result recording is realized, and the manual intervention is reduced.
[0020] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structures indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.
[0022] Figure 1 A dynamic multi-task data checking method flowchart of the present application is shown.
[0023] Figure 2 A dynamic multi-task data checking system module diagram of the present application is shown.
[0024] Figure 3 A data update condition monitoring method flowchart in the embodiment of the present application is shown. DETAILED DESCRIPTION
[0025] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.
[0026] The present application provides a dynamic multi-task data checking method based on variable configuration, as shown in the figure, the steps include, Figure 1 S1, starting a data checking task, reading the current task configuration, and determining the task type of the data checking task.
[0027] In this embodiment, the data checking task is started, and the current task type (such as flight, passenger, stowage, ticket, etc.) is read from the configuration file; wherein the task type defines the data category of this checking.
[0028] In this embodiment, before starting the data checking task, the present application also initializes the task running environment and loads the default or last checking task configuration.
[0029] S2, according to the determined task type, dynamically adjusting the task configuration through the operation and maintenance interface, and obtaining the corresponding checkpoint from the modified task configuration; wherein the checkpoint defines the specific data item or rule that needs to be compared for consistency.
[0030] In this embodiment, the task configuration is dynamically modified through the operation and maintenance interface, specifically including adding or modifying the task type according to the business requirements; obtaining the corresponding checkpoint according to the task type; specifying the data of a certain system as the baseline for comparison.
[0031] Optionally, the present application supports real-time configuration update, and the configuration change time is less than 200ms, ensuring the flexibility and real-time performance of the task.
[0032] S3, according to the modified task configuration, obtaining the data to be checked, and performing consistency comparison verification on the obtained data to be checked, obtaining the verification result and the compared data.
[0033] In the embodiment, the application obtains the verification result and the compared data in a layered and policy-based manner. After obtaining the same type of data from different systems, it is determined whether the field has special rules, such as inclusion rules, corresponding rules, and equal comparison when there is no rule, and the same is determined as consistent; the inclusion rule, such as the field data of system A being the string of the data of system B, is determined as consistent; the corresponding rule, that is, the field is translated according to the business rule, and the same meaning is represented in the business, and the same is determined as consistent according to the rule.
[0034] S4, analyze the verification result, and record the compared data according to the analysis result, generate and store a record timestamp.
[0035] In the embodiment, the application records the compared data according to the analysis result, specifically including directly recording the consistent data in the analysis result; tracing the inconsistent data in the analysis result, and giving a modification suggestion, and merging the inconsistent data and the modification suggestion.
[0036] S5, monitoring data update at a preset time interval, and running the data checking task cyclically according to the data update, such as Figure 3 .
[0037] In the embodiment, when the data is not updated, the data update is continuously monitored; otherwise, when the data is updated, it is determined whether there is the same type of historical record in the database, and whether the current time is greater than the timestamp recorded by the last task.
[0038] Optionally, when there is no same type of historical data in the database, an incremental to-be-checked task list is directly formed, a new round of data checking task cycle is triggered, and steps S1-S4 are repeated; when there is the same type of historical data in the database, and the current time is greater than the timestamp recorded by the last task, an incremental to-be-checked task list is formed, a new round of data checking task cycle is triggered, and steps S1-S4 are repeated; otherwise, when there is the same type of historical data in the database, and the current time is less than the timestamp recorded by the last task, the data checking task is not triggered; wherein, in the new round of data checking task, the data obtained in step S3 is mainly incremental update data, thereby improving the checking efficiency.
[0039] As shown in Figure 2 , the application also provides a dynamic multi-task data checking system, which comprises a task starting module, an operation and maintenance module, a data checking module, a data storage module and a cyclic running module.
[0040] 1, the task starting module is used for starting the data checking task, reading the current task configuration, and determining the task type of the data checking task.
[0041] In the embodiment, the task starting module starts the data checking task, and reads the current task type (such as flight, passenger, stowage, ticket, etc.) from the configuration file; wherein the task type defines the data category of the current checking.
[0042] In the embodiment, the task starting module initializes the task running environment before starting the data checking task, and loads the default or last checking task configuration.
[0043] 2. The operation and maintenance module is configured to dynamically adjust the task configuration through the operation and maintenance interface according to the determined task type, and obtain the corresponding checking point from the modified task configuration; wherein the checking point defines the specific data item or rule that needs to be compared for consistency.
[0044] In the embodiment, the operation and maintenance module dynamically modifies the task configuration through the operation and maintenance interface, specifically including adding or modifying the task type according to business requirements; obtaining the corresponding checking point according to the task type; and specifying the data of a certain system as the baseline for comparison.
[0045] Optionally, the operation and maintenance module also supports real-time configuration update, and the configuration change takes less than 200 ms, ensuring the flexibility and real-time performance of the task.
[0046] 3. The data checking module is configured to obtain the data to be checked according to the modified task configuration, and perform consistency comparison and verification on the obtained data to be checked, to obtain the verification result and the compared data.
[0047] In the embodiment, the data checking module obtains the verification result and the compared data through hierarchical and policy-based methods. After obtaining the same type of data from different systems, it is determined whether the field has special rules, such as inclusion rules, corresponding rules, and equal comparison when there are no rules, which is determined to be consistent; inclusion rules, such as A system field data being a string of B system data, which is determined to be consistent; and corresponding rules, i.e. translating the field according to business rules, which all represent the same meaning in business, and are determined to be consistent according to the rules.
[0048] 4. The data storage module is configured to analyze the verification result, and record the compared data according to the analysis result, to generate and store the record timestamp.
[0049] In the embodiment, the data storage module records the compared data according to the analysis result, specifically including directly recording the consistent data in the analysis result; tracing the inconsistent data in the analysis result, and giving modification suggestions, and merging the inconsistent data and the modification suggestions for recording.
[0050] 5. The cycle running module is configured to monitor the data update situation at a preset time interval, and cyclically run the data checking task according to the data update situation.
[0051] In the embodiment, when the data is not updated, the data update is continuously monitored; otherwise, when the data is updated, it is judged whether the same type of historical record exists in the database, and whether the current time is greater than the time stamp recorded by the last task is compared, according to the judgment result, the data checking task is triggered.
[0052] Optionally, when the same type of historical data does not exist in the database, an incremental to-be-checked task list is directly formed, and a new round of data checking task cycle is triggered; when the same type of historical data exists in the database, and the current time is greater than the time stamp recorded by the last task, an incremental to-be-checked task list is formed, and a new round of data checking task cycle is triggered; otherwise, when the same type of historical data exists in the database, and the current time is less than the time stamp recorded by the last task, the data checking task is not triggered; wherein, in the new round of data checking task, the data obtained in the data checking module is mainly incremental update data, so as to improve the checking efficiency.
[0053] Based on the above disclosure, the present application also provides an electronic device. The electronic device of the embodiment of the present application comprises at least one processor and at least one storage medium electrically connected, the storage medium is electrically connected with the processor, wherein the storage medium stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can execute the method as described above.
[0054] Based on the same inventive concept, the present application also provides a storage medium, the storage medium stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can execute the method as described above.
[0055] The above description and drawings sufficiently illustrate the embodiments of the present application to enable one skilled in the art to practice them. Other embodiments can include structural and other changes. The embodiments are merely representative of the possible variations. Individual components and functions are optional unless explicitly required, and the order of operations can be varied. Parts and features of some embodiments can be included or substituted for parts and features of other embodiments. The embodiments of the present application are not limited to the structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A dynamic multitasking data inspection method, characterized by, The method comprises, starting a data checking task, reading a current task configuration, and determining a task type of the data checking task; according to the determined task type, dynamically adjusting the task configuration through an operation and maintenance interface, and obtaining a corresponding checkpoint from the modified task configuration; according to the modified task configuration, obtaining to-be-checked data, and performing consistency comparison and verification on the obtained to-be-checked data to obtain a verification result and compared data; analyzing the verification result, and according to an analysis result, recording the compared data, generating, and storing a record timestamp; monitoring data update at a preset time interval, and according to the data update, cyclically running the data checking task.
2. The dynamic multitasking data inspection method of claim 1, wherein, The starting of the data checking task further comprises, before starting the data checking task, initializing a task running environment, and loading a default or last checking task configuration.
3. The method of claim 1, wherein, The recording of the compared data according to the analysis result specifically comprises, directly recording consistent data in the analysis result; tracing inconsistent data in the analysis result, and giving a modification suggestion, and merging the inconsistent data and the modification suggestion for recording.
4. The method of claim 1, wherein, The cyclic running of the data checking task according to the data update specifically comprises, when the data is not updated, continuing to monitor the data update; otherwise, when the data is updated, judging whether there is same type historical record in a database, and comparing whether a current time is greater than a timestamp recorded by a last task, according to a judgment result, triggering the data checking task.
5. The dynamic multitasking data inspection method of claim 4, wherein, The judgment result triggering the data checking task specifically comprises, when there is no same type historical data in the database, directly forming an incremental to-be-checked task list, and triggering a new round of data checking task cycle; when there is same type historical data in the database, and the current time is greater than the timestamp recorded by the last task, forming the incremental to-be-checked task list, and triggering the new round of data checking task cycle; otherwise, when there is same type historical data in the database, and the current time is less than the timestamp recorded by the last task, not triggering the data checking task.
6. A dynamic multitasking data inspection system, characterized by, The system comprises a task starting module, an operation and maintenance module, a data checking module, a data storage module, and a cyclic running module; The task starting module is used for starting a data checking task, reading a current task configuration, and determining a task type of the data checking task; The operation and maintenance module is used for according to the determined task type, dynamically adjusting the task configuration through an operation and maintenance interface, and obtaining a corresponding checkpoint from the modified task configuration; The data checking module is used for according to the modified task configuration, obtaining to-be-checked data, and performing consistency comparison and verification on the obtained to-be-checked data to obtain a verification result and compared data; The data storage module is used for analyzing the verification result, and according to an analysis result, recording the compared data, generating, and storing a record timestamp; The cyclic running module is used for monitoring data update at a preset time interval, and according to the data update, cyclically running the data checking task.
7. The dynamic multitasking data inspection system of claim 6, wherein, The task starting module is further used for, before starting the data checking task, initializing a task running environment, and loading a default or last checking task configuration.
8. The dynamic multitasking data inspection system of claim 6, wherein, The data storage module is specifically configured to, The consistent data in the analysis result is directly recorded in the record; the inconsistent data in the analysis result is traced, and a modification suggestion is given; and the inconsistent data and the modification suggestion are recorded.
9. The dynamic multitasking data inspection system of claim 6, wherein, The cycle running module is specifically configured to, when the data is not updated, continue to monitor the data update; otherwise, when the data is updated, judge whether there is historical record of the same type in the database, and compare whether the current time is greater than the time stamp recorded by the last task, according to the judgment result, trigger the data checking task.
10. The dynamic multitasking data inspection system of claim 9, wherein, The cycle running module is further configured to, When there is no historical data of the same type in the database, an incremental to-be-checked task list is directly formed, and a new round of data checking task cycle is triggered; when there is historical data of the same type in the database, and the current time is greater than the time stamp recorded by the last task, an incremental to-be-checked task list is formed, and a new round of data checking task cycle is triggered; otherwise, when there is historical data of the same type in the database, and the current time is less than the time stamp recorded by the last task, the data checking task is not triggered.
11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the dynamic multi-task data checking method steps of any one of claims 1-5.
12. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the dynamic multi-task data checking method steps of any one of claims 1-5. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the dynamic multi-task data checking method steps of any one of claims 1-5.