TCN data processing method, device and system based on configuration file and storage medium

Abstract

The invention provides a TCN data processing method, device and system based on a configuration file and a storage medium. The method comprises the following steps of: acquiring TCN (Transfer ss Control Number) standard data, and acquiring the TCN (Transfer ss Control Number) standard data according to the TCN (Transfer ss Control Number) standard data; acquiring a configuration file; wherein the configuration file is designed and configured according to application requirements; and according to the configuration file, processing the TCN standard data to obtain output data. The method only depends on a local data file system and does not depend on a remote database, so that the reliability of a data processing program is improved, and the timeliness of shared data is ensured; meanwhile, according to the screening conditions of different receivers in the configuration file and different output formats, output files of different formats are automatically processed, and the customization requirements of different receivers are met.

Description

Technical Field

[0001] The present disclosure belongs to the field of standard data processing, and in particular relates to a TCN data processing method, device, system and storage medium based on a configuration file. Background Art

[0002] The TCN (Transmission Control Number) data standard is one of the industry sales record data standards released by the Airline Rate Publishing Company (ATPCO), and is used for sales data exchange services between different civil aviation industry entities. Through this standard data exchange service, different airlines, ticket distribution system providers, ticket agents, ticket settlement service providers, civil aviation data processing companies, and other civil aviation participating organizations around the world can share ticket sales information according to the same data standard.

[0003] TCN standard data is divided into multiple records according to the content of the sales record, and each record focuses on a specific data topic. For example, record topics 1, 2, 3, 4, 5, 7, and 8 are basic information of sales records, market information, supplementary sales information, detailed amount information, itinerary information, payment information, fare calculation information, etc. Each record contains specific fields, and the start and end positions of each field in each record are fixed. When the string length of the field is less than the maximum length specified for the field, it needs to be filled with placeholders. The first two digits of each line of TCN records are used to mark the record number, such as 01 represents the record 1, and 02 represents the record 2. The total length of each record is fixed to 400 characters.

[0004] After receiving TCN data files, different civil aviation industry entities refer to the TCN data standards published by ATPCO, parse and store the data files, and apply them to related application systems. Then, based on the application types and needs of their downstream recipients, they filter the stored data and reassemble them into TCN data or data files in other formats for distribution.

[0005] Among the applications of TCN standard data, the most common application is to share TCN standard data files in a customized manner according to different data recipients. This process involves data range screening, data record screening, data field screening, data desensitization, data output conversion and other data operations. The current common processing method is to receive TCN standard data files from upstream data output organizations, parse TCN standard data files into database tables according to different records, and develop different data processing programs for data recipients of different application types to export data files in different formats. These data processing processes are generally composed of solidified programs. For data recipients of the same application type, the data output format is generally the same, and it is basically difficult to provide customized screening and output. And these methods basically need to go through three data operation processes, including receiving data files, parsing and storing data files, and exporting data files. Among these three data operation processes, the parsing and storing process has many uncertain factors and a long process, which affects the timeliness of data distribution. Summary of the invention

[0006] In view of the application characteristics and existing problems of the above-mentioned TCN standard data, the present disclosure provides a TCN data processing method and device based on a configuration file. After receiving the TCN data file, the exported file is directly screened and sent to the downstream according to the needs of the downstream recipient. The present disclosure only relies on the local data file system and does not rely on the remote database, which increases the reliability of the data processing program and ensures the timeliness of sharing data; at the same time, according to the screening conditions and output formats of different recipients in the configuration file, output files of different formats are automatically processed, solving the customization needs of different recipients.

[0007] On the one hand, a method for TCN data processing based on a configuration file is provided, comprising:

[0008] Get TCN (Transmission Control Number) standard data;

[0009] Obtaining a configuration file; wherein the configuration file is designed and configured according to application requirements;

[0010] According to the configuration file, the TCN standard data is processed to obtain output data.

[0011] In some embodiments, the configuration file includes at least one or more of a data format configuration of a TCN input file of TCN standard data; a parsing configuration of the TCN input file; a configuration of intermediate variables required for data analysis and calculation; a parsing configuration of the intermediate variables; a data format configuration of an output file; and an operation processing configuration of the output file.

[0012] In some embodiments, the processing of the TCN standard data according to the configuration file includes:

[0013] Configuration file parsing;

[0014] Output file initialization;

[0015] A TCN input file containing the TCN standard data is read, and the TCN input file is parsed according to the configuration file;

[0016] Output data item parsing and configuration;

[0017] Generation, finalization and output of output files.

[0018] In some embodiments, the configuration file parsing specifically includes: configuration parsing of the TCN input file format; configuration parsing of intermediate variables required for data analysis and calculation; configuration parsing of the data format of the output file; or configuration parsing of the output data items used for data output.

[0019] In some embodiments, the reading of the TCN input file containing the TCN standard data and parsing the TCN input file according to the configuration file specifically include: reading the file header of the input TCN input file according to the configuration file, and interpreting the field information in the file header of the TCN input file according to the configuration of the intermediate variable, and assigning the intermediate variable;

[0020] According to the data format configuration of the TCN input file, the input TCN standard data is read sequentially, and according to the configuration of the intermediate variables, the information of the TCN standard data is interpreted and the intermediate variables are assigned.

[0021] In some embodiments, the output data item parsing and configuration specifically includes: traversing all output data item configurations of the configuration file, and judging whether the variable value of the currently processed intermediate variable meets the filtering condition according to the filtering item configuration of each output data item configuration; for the variable value that meets the filtering condition, forming the output data for generating the output file according to the output format configuration of the output file of the configuration file.

[0022] In some embodiments, the generation, completion and output of the output file specifically include: writing the parsed output data into the output file according to the output format of the output file; and encrypting or compressing the output file according to the configuration file.

[0023] On the one hand, a TCN data processing device based on a configuration file is provided, comprising:

[0024] Standard data acquisition module, used to obtain TCN (Transmission Control Number) standard data;

[0025] A configuration file acquisition module, used to acquire a configuration file; wherein the configuration file is designed and configured according to application requirements;

[0026] The output data processing module is used to process the TCN standard data according to the configuration file to obtain output data.

[0027] On the one hand, a performance test asynchronous request data processing system is provided, the system comprising a processor and a memory, the processor executes computer instructions stored in the memory to implement the above-mentioned TCN data processing method based on the configuration file.

[0028] On the one hand, a computer-readable storage medium is provided, which stores computer instructions for causing a computer system to execute the above-mentioned TCN data processing based on the configuration file.

[0029] The solution provided by the present disclosure does not rely on a remote database, and only realizes the process of directly converting the input TCN file into the output TCN-related file through configuration file settings, while meeting the output format customization requirements of different downstream recipients. Compared with the current common TCN data file processing method, the program of the present disclosure is more robust, timely and flexible.

[0030] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or be understood by implementing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A flow chart of a method for TCN data processing based on a configuration file according to an embodiment of the present disclosure is shown;

[0033] Figure 2 A structural diagram of a device for TCN data processing based on a configuration file according to an embodiment of the present disclosure is shown;

[0034] Figure 3 A structure diagram of a TCN data processing system based on a configuration file according to an embodiment of the present disclosure is shown;

[0035] Figure 4 A computer-readable storage medium structure diagram according to an embodiment of the present disclosure is shown;

[0036] Figure 5 A specific flow chart of a method for TCN data processing based on a configuration file according to an embodiment of the present disclosure is shown;

[0037] Figure 6 A specific structural diagram of a device for TCN data processing based on a configuration file according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0039] There are many difficulties in the performance testing of asynchronous business software, which results in the inability to collect, count and analyze data for asynchronous requests as simply and quickly as synchronous business during the performance testing process. In response to this technical problem, the present disclosure provides the following technical solution.

[0040] On the one hand, the embodiment of the present disclosure provides a method for TCN data processing based on a configuration file, such as Figure 1 As shown, including:

[0041] First, obtain TCN (Transmission Control Number) standard data;

[0042] Second, obtaining a configuration file; wherein the configuration file is designed and configured according to application requirements;

[0043] Third, according to the configuration file, the TCN standard data is processed to obtain output data.

[0044] In one embodiment, the configuration file is designed and configured according to application requirements, and the various structures and interpretation procedures of the configuration file are designed, which mainly include the following parts:

[0045] 1) The format configuration and interpretation program of the input TCN input file, including the file header configuration of the TCN input file, the definition of the data content interpretation configuration of the TCN input file and the interpretation program design;

[0046] 2) Intermediate variable definition and source configuration design and interpretation procedures;

[0047] 3) Output format configuration design and interpretation procedures for output files;

[0048] 4) Design and configuration of output data items, including the configuration of the screening conditions of output data items, the output data format of output data items and other information.

[0049] In one embodiment, TCN (Transmission Control Number) standard data is obtained through one or more systems of one or more civil aviation owners.

[0050] In one embodiment, according to the designed and configured configuration file, processing the acquired TCN standard data includes: Figure 5 As shown:

[0051] Step S101: interpreting the configuration file;

[0052] The elements involved in the process of converting TCN data files from input files to output files are abstracted, including: 1) input files; 2) variables required for calculation; 3) output files.

[0053] The configuration information for these element designs includes:

[0054] 1) Input file format configuration; 2) Variable configuration required for calculation; 3) Output format configuration; 4) Output item configuration.

[0055] In one embodiment, the main information included in each configuration item of the configuration file is as follows in Table 1:

[0056]

[0057]

[0058]

[0059]

[0060]

[0061] Table 1: Configuration items of the configuration file

[0062] In Table 1, the configuration items marked with (*) indicate that there are multiple configuration items, which are array configuration items.

[0063] The configuration interpretation program reads the configuration file through the input program parameters and converts the content of the configuration file into various configuration data objects.

[0064] Convert the parameter values ​​in the program startup input parameters to the general variable list. For example, if the program startup input parameters include -dataDate=20240801-var2=xxxx, then the general variables include var2 of the variable data, and its values ​​are 20240801 and xxxx respectively. The value of this variable can be used in the calculation during the entire data processing process.

[0065] Step S102: output file initialization;

[0066] According to the output item configuration in the configuration data object read in the previous step, replace the variable value in the output file name and initialize the output file, including generating file objects (TCN files, general text files, Excel files and other file types), initializing the file header content for TCN files; initializing the title and header and other contents for EXCEL files, initializing the number of sales records, number of record rows and other information in the output file.

[0067] Step S103: TCN input file reading;

[0068] According to the file header format of the TCN input file, the configuration reads the file header of the input TCN input file, and according to the variable definition configuration, assigns these variables with the field information in the file header of the read TCN input file.

[0069] According to the record format configuration of the sales record of the TCN input file, the data record lines of the input TCN input file are read in row order, and according to the variable definition configuration, these variables are assigned with the data field information of the read TCN input file.

[0070] After reading all the records of a sales record set, proceed to the next step. After the next step, clear the sales records read this time from the memory.

[0071] The method to judge whether all the records of the sales record are completely read is to read the next record line again to the 01 segment group or to read to the end of the file of the TCN input file.

[0072] Step S104: TCN data screening and recording output:

[0073] Traverse all output item configurations of the configuration file, and determine whether the variable value of the current record meets the conditions according to the filter item configuration of each output item configuration, and determine whether the current TCN record meets the filter conditions. For records that meet the filter conditions, select the records and fields to be output according to the output format configuration, and perform calculations, conversions or desensitization according to the configuration, and output to the output file according to the specified output format (TCN format, text format, EXCEL format, etc.).

[0074] In one embodiment, the filter item configuration is a condition tree structure that can be infinitely extended downwards, and each filter item includes three optional parts: current filter condition, logic and filter item, and logic or filter item. Its decision table (Table 2) is as follows:

[0075] Current filter Logical and filter items Logical OR filter Judgment conclusion Filter is empty Any value Any value true false Any value Filter is empty false false Any value true true false Any value false false true Filter is empty Any value true true true Any value true true false Any value false

[0076] Table 2: Decision table

[0077] Step S105: TCN file output finalization processing steps:

[0078] When all the record lines of the TCN input file are read, filtered, and output, this step closes the input and output files, and reopens the output file in random write mode according to the configuration, and writes certain statistical values ​​of the output file to specific bytes of the output file (if the output format is still TCN format, rewrite the TCN file record line number field in the file header according to the total number of data lines in the file; if the output format is EXCEL, the statistical values ​​need to be recalculated after the file records are written completely). When the file is processed, encrypt, compress, and other operations are performed on the file according to the configuration.

[0079] On the one hand, the embodiment of the present disclosure provides a device for TCN data processing based on a configuration file, such as Figure 2 As shown, including:

[0080] Standard data acquisition module, used to obtain TCN (Transmission Control Number) standard data;

[0081] A configuration file acquisition module, used to acquire a configuration file; wherein the configuration file is designed and configured according to application requirements;

[0082] The output data processing module is used to process the TCN standard data according to the configuration file to obtain output data.

[0083] In one embodiment, the apparatus processes the acquired TCN standard data according to the designed and configured configuration file, and specifically includes: Figure 6 As shown:

[0084] Configuration interpretation module, used to read configuration information such as input file format configuration, variable definition and source configuration, output format configuration, output item configuration, etc.

[0085] The output file initialization module is used to initialize the output file according to the input item configuration (output file name, output file format), generate the file header part of the output file; and set the output configuration information corresponding to the output item.

[0086] The TCN input file reader module is used to read the input TCN data in row order according to the format configuration of the TCN file, and assign values ​​to these variables with the read TCN data field information according to the variable definition configuration.

[0087] The TCN data screening and record output module is used to determine whether the currently read TCN record meets the conditions based on the matching of the screening conditions with the variable values; the records that meet the conditions are written into the output file according to the output configuration.

[0088] The TCN file output finishing processing module is used to rewrite the statistical information of the output data and perform file encryption and compression according to the output configuration items.

[0089] On the one hand, if Figure 3 As shown, an embodiment of the present disclosure provides a TCN data processing system based on a configuration file, the system includes a processor and a memory, the processor executes computer instructions stored in the memory to implement the above-mentioned TCN data processing method based on the configuration file.

[0090] On the one hand, if Figure 4 As shown, the embodiment of the present disclosure provides a computer-readable storage medium 40 on which non-transitory computer-readable instructions 41 are stored. When the non-transitory computer-readable instructions 41 are executed by a processor, the TCN data processing method based on the configuration file of each embodiment of the present disclosure is executed.

[0091] As described above, the more common TCN data processing methods in the prior art basically include three processes: receiving TCN input files, parsing and storing them, and outputting TCN related files, and the format of the output files cannot be customized as needed.

[0092] The solution provided by the present disclosure does not rely on a remote database, and only realizes the process of directly converting the input TCN file into the output TCN-related file through configuration file settings, while meeting the output format customization requirements of different downstream recipients. Compared with the current common TCN data file processing method, the program of the present disclosure is more robust, timely and flexible.

[0093] Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for TCN data processing based on a configuration file, characterized in that: Get TCN (Transmission Control Number) standard data; Obtaining a configuration file; wherein the configuration file is designed and configured according to application requirements; According to the configuration file, the TCN standard data is processed to obtain output data.

2. The method according to claim 1, wherein: The configuration file at least includes a data format configuration of a TCN input file of TCN standard data; a parsing configuration of the TCN input file; a configuration of intermediate variables required for data analysis and calculation; and a parsing configuration of the intermediate variables; The data format configuration of the output file; and one or more of the operation processing configurations of the output file.

3. The method according to claim 1, wherein: The processing of the TCN standard data according to the configuration file includes: Configuration file parsing; Output file initialization; A TCN input file containing the TCN standard data is read, and the TCN input file is parsed according to the configuration file; Output data item parsing and configuration; Generation, finalization and output of output files.

4. The method according to claim 3, wherein: The configuration file parsing specifically includes: configuration parsing of the TCN input file format; configuration parsing of intermediate variables required for data analysis and calculation; configuration parsing of the data format of the output file; or configuration parsing of the output data items used for data output.

5. The method according to claim 3, wherein: The reading of the TCN input file containing the TCN standard data and parsing the TCN input file according to the configuration file specifically include: reading the file header of the input TCN input file according to the configuration file, and interpreting the field information in the file header of the TCN input file according to the configuration of the intermediate variable, and assigning the intermediate variable; According to the data format configuration of the TCN input file, the input TCN standard data is read sequentially, and according to the configuration of the intermediate variables, the information of the TCN standard data is interpreted and the intermediate variables are assigned.

6. The method according to claim 3, wherein: The output data item parsing and configuration specifically includes: traversing all output data item configurations of the configuration file, and judging whether the variable value of the currently processed intermediate variable meets the filtering condition according to the filtering item configuration of each output data item configuration; for the variable value that meets the filtering condition, forming the output data for generating the output file according to the output format configuration of the output file of the configuration file.

7. The method according to claim 3, wherein: The generation, closing and output of the output file specifically include: writing the parsed output data into the output file according to the output format of the output file; and encrypting or compressing the output file according to the configuration file.

8. A TCN data processing device based on a configuration file, characterized in that: Standard data acquisition module, used to obtain TCN (Transmission Control Number) standard data; A configuration file acquisition module, used to acquire a configuration file; wherein the configuration file is designed and configured according to application requirements; The output data processing module is used to process the TCN standard data according to the configuration file to obtain output data.

9. A performance test asynchronous request data processing system, the system comprising a processor and a memory, the processor executing computer instructions stored in the memory to implement any of the methods of claims 1-7.

10. A computer-readable storage medium storing computer instructions for causing a computer system to execute the method of any one of claims 1 to 7.

Images