File reading and writing method and system based on XML template and electronic equipment

Abstract

The invention provides a file read-write method and system based on an XML template and electronic equipment, and the main scheme is as follows: text files are converted into data to be put in storage, file related attributes are configured through the XML, and read-write batch put-in-storage operation is performed on the text files in a specified directory and a ready state. File fields needing to be stored are screened out, storage rules are screened through XML configuration data, to-be-stored text data are read and written, and to-be-operated data information is obtained; in the to-be-operated data information, selecting a to-be-operated table name and a data column configured by corresponding field mapping in the table through self-defined data screening to generate a corresponding table entry SQL, and performing persistent storage. According to the invention, by using the file storage configuration, only the corresponding file storage operation needs to be configured, and after the service is accessed, the service can be newly added, expanded and changed only by modifying part of the configuration, so that redundant code development in a hard coding mode in the service expansion is avoided, and the manpower investment of development, test, operation and maintenance is reduced.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a file reading and writing method, system and electronic equipment based on XML template. Background Art

[0002] As the volume of banking business increases, different systems have reconciliation business needs, and different channels have different data files, so there is a need to read and write a large number of reconciliation files into the warehouse. In the process of development using hard coding, we found that there was a lot of repeated code, and the file storage performance of different businesses would be slightly different. In addition, when the business files changed, operation and maintenance, redevelopment, restart, and retesting consumed a lot of manpower and time, and the flexibility was too poor. As the needs of business expansion continued to change, a lot of manpower was consumed, and a lot of repeated and redundant code was generated, and the resulting testing and operation and maintenance costs were greatly increased. This cost is unbearable in the Internet financial industry where the network is rapidly developing, so in order to simplify repeated file reading and writing and reduce labor costs, this method of configurable file reading and writing was developed;

[0003] The file reading and writing method based on XML template can solve the problems of hard-coded file data screening, elimination of storage, the need to restart the service after encoding, the inability to configure storage performance according to file size, and the need to repeatedly develop code to expand business scenarios through the configuration of file reading and writing methods. Flexibility is greatly improved, business expansion is simple, and maintenance and operation costs are greatly reduced. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the present invention provides a file reading and writing method, system and electronic device based on XML template, which are used to solve the problems in the prior art of hard-coded file data screening, elimination of warehousing, the need to restart the service after encoding is completed, the inability to configure the warehousing performance according to the file size, and the need to repeatedly develop code to expand business scenarios.

[0005] According to a first aspect of the present invention, the present invention provides a file reading and writing method based on an XML template, comprising:

[0006] Step 1. Convert text files into data and store them in the database. Configure the relevant attributes of the XML configuration file to perform batch reading and writing operations on the text files in the specified directory and ready state.

[0007] Step 2. Filter out the file fields that need to be stored, filter the storage rules through XML configuration data, read and write the text data to be stored, and obtain the data information to be operated;

[0008] Step 3. In the data information to be operated, through custom data filtering, select the table name to be operated and the data column configured in the corresponding field mapping in the table to generate the corresponding table entry SQL and persist it into the database.

[0009] Based on the above technical solution, the present invention can also make the following improvements.

[0010] Optional file read and write configuration includes:

[0011] Conventional file reading and writing: Generate persistence operations through mapping of file data column data, type, and table fields;

[0012] Read and write multiple identical files: Obtain all files of the same type that match the current file name through file batches and then perform regular file reading and writing;

[0013] Reading and writing complex data files: Through the configured persistent multi-table configuration, after the data file column separator and the number of data are adapted, the corresponding table is generated and stored in the database for the corresponding SQL persistence.

[0014] Optionally, the regular file is read and written as follows:

[0015] Declare that the file read and write same attributes are empty, which is used to define the current file read and write configuration as a regular file.

[0016] Optionally, the same multiple files are read and written as follows:

[0017] Declare that the file read and write same attribute is a positive integer or n, indicating that the current file read and write are the same multiple files.

[0018] Optionally, the complex file is read and written as follows:

[0019] In the persistent table configuration, multiple TABLE configuration nodes are declared, indicating that there are multiple table data to be persisted in the current data file.

[0020] Optionally, the filtering out of the file fields to be stored, filtering the storage rules through XML configuration data, reading and writing the text data to be stored, and obtaining the data information to be operated include:

[0021] Get the file directory, file ready status, file batch, file-related code, data start row, data batch, custom processing, and table name, delimiter, data column, column type, and row data quantity configuration of the XML file read and write files through the business number, and filter the tags for the above XML source data.

[0022] Optionally, the step of selecting the table name to be operated and the data column of the corresponding field mapping configuration in the table to generate the corresponding table entry SQL by customizing data screening includes:

[0023] Through XML configuration, the file is read and written by IO, and the insert SQL of the defined batch is generated according to the mapping column and column type and the persistence operation is performed. According to the corresponding service of the XML configuration filter, the code extension custom data filtering is supported, including:

[0024] S3.1: Customized extended data processing;

[0025] S3.2: Complex data processing;

[0026] S3.3: Define XML data columns, table columns, and type mappings.

[0027] Optionally, generating a batch of insert SQLs according to the mapping columns and column types and performing a persistence operation includes:

[0028] Through the generated SQL string, the total execution SQL is spliced, the database DAO is called, and the persistence operation is performed;

[0029] If the file reading and writing is not completed, continue to execute step S3.3 to read the file again;

[0030] If the file reading and writing is completed, the process ends.

[0031] According to a second aspect of the present invention, there is provided a file reading and writing system based on an XML template, comprising:

[0032] The data storage module is used to convert text files into data storage. By configuring the relevant attributes of the XML configuration file, the batch storage operation of the text files in the specified directory and ready state is performed;

[0033] The data reading and writing module is used to filter out the file fields that need to be stored, configure the data filtering and storage rules through XML, read and write the text data to be stored, and obtain the data information to be operated;

[0034] The mapping data generation module is used to select the table name to be operated and the data column of the corresponding field mapping configuration in the table through custom data screening in the data information to be operated, generate the corresponding table entry SQL and persist it into the database.

[0035] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for reading and writing files based on an XML template as described above is implemented.

[0036] Technical effects and advantages of the present invention:

[0037] The present invention provides a file reading and writing method, system and electronic device based on XML template. Due to the deepening of cooperation between banking business and various banking institutions, there are a large number of file reconciliation business requests, and different institutional businesses have different data storage requirements. In addition to normal business communication costs, manpower development, testing, and complex service operation and maintenance costs caused by file changes due to demand changes, by using file storage configuration, it is only necessary to configure the corresponding file storage operation, and after the business is connected, the configured reading and writing are performed. For business changes, only part of the configuration needs to be modified to achieve business addition, expansion, and change. It avoids redundant code development in the form of hard coding in business expansion, and greatly reduces the investment in development, testing, and operation and maintenance manpower. It can be of great help in business expansion, server deployment, operation, and development, reducing investment costs and greatly improving development efficiency.

[0038] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flowchart of a file reading and writing method based on XML template provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It is understandable that, based on the defects in the background technology, the present invention provides a file reading and writing method based on XML template, specifically as follows Figure 1 As shown, the following steps are included:

[0042] Step 1. Read and write files through business requests, convert text files into data and store them in the warehouse. Configure the relevant attributes of the XML file to perform batch reading and writing operations on text files in the specified directory and ready state;

[0043] The service request includes a timer or a service interface to read and write files; the tags of the XML rule template are as follows:

[0044] <files>: Root node, contains no attributes, no other type of labels, must contain at least one <file>Sub-tags;

[0045] <file>: File read and write configuration, must contain a <name> 、 <path> 、 <flag> 、 <start> 、 <batch> 、 <encode> 、 <filter> 、 <tablses>Subtags, attributes include:

[0046] id: represents a unique matching value;

[0047] desc:current <file>Description of the node

[0048] Business rules:

[0049] The id attribute of FILE is unique. A file can only exist in one configuration in the configuration, that is, the id is unique;

[0050] Processing logic:

[0051] When the project starts, the XML configuration is converted into the Java object FileReaderXmlVOs through the XML parsing tool Digester;

[0052] After the polling or service interface request is started by the timer, the id attribute corresponding to the business mapping in the request parameter is obtained, and the FileReaderXmlVO configuration properties are obtained from FileReaderXmlVOs.

[0053] In this embodiment, the file reading and writing configuration includes:

[0054] Regular file reading and writing: Generate persistence operations through mapping of file data column data, type, and table fields; the regular file reading and writing is as follows:

[0055] Declare that the file read and write same attribute is empty, which is used to define the current file read and write configuration as a regular file;

[0056] Read and write the same multiple files: through file batches, obtain all files of the same type that the current file name is adapted to, and then perform regular file reading and writing one by one. The same multiple file reading and writing is as follows:

[0057] Declare that the file read and write same attribute is a positive integer or n, indicating that the current file read and write are the same multiple files;

[0058] Reading and writing complex data files: Through the configured persistent multi-table configuration, after the data file column separator and the number of data are adapted, the corresponding table is generated and stored in the database for the corresponding SQL persistence.

[0059] The complex file is read and written as follows:

[0060] In the persistent table configuration, multiple TABLE configuration nodes are declared, indicating that there are multiple table data to be persisted in the current data file.

[0061] Step 2. Filter out the file fields that need to be stored, filter the storage rules through XML configuration data, read and write the text data to be stored, and obtain the data information to be operated;

[0062] In the above implementation, the file directory, file ready status, file batch, file-related code, data start row, data batch, custom processing Service, and table name, delimiter, data column, column type, and row data quantity configuration of the XML file read and write file are obtained through the business number;

[0063] The XML source data filtering tags are as follows:

[0064] S2.1: Files in the same batch (attribute mapping): a positive integer number of files or N.

[0065] <name>: Indicates the name of the file currently being read or written. The properties include the following:

[0066] desc: current node description

[0067] formate: The current file name is formatted in time type (such as: yyyyMMdd)

[0068] same: the current number of files (blank means 1, positive integer or N means infinite)

[0069] S2.2: File read and write attributes (attribute mapping)

[0070] <path>: The file path to be parsed. The properties include the following:

[0071] desc: current node description

[0072] formate: The current file path formatted time type (such as: yyyyMMdd)

[0073] <flag>: The file to be parsed is ready. The properties include the following:

[0074] desc: current node description

[0075] formate: The current file name is formatted in time type (such as: yyyyMMdd)

[0076] <start>: The starting line of the file to be parsed. The properties include the following:

[0077] desc: current node description

[0078] <batch>: The number of files to be parsed in a batch (the number of rows to be parsed at a time). The attributes include the following:

[0079] desc: current node description

[0080] <encode>: The file encoding to be parsed. The properties include the following:

[0081] desc: current node description

[0082] Processing logic:

[0083] Get the same attribute (NAME node same) in the FileNameVO attribute in FileReaderXmlVO, and format the file name according to the number of files configured by the same attribute and the formate attribute in FileNameVO;

[0084] a).same attribute is empty or 1, indicating that there is only one file of the same type to be read or written

[0085] b).same attribute is a positive integer greater than 1, indicating that the number of files of the same type to be read and written is the same as the specified number

[0086] c).same attribute is n, indicating that the number of files to be read and written is indefinite (read as many files as there are in the directory)

[0087] Get the formate attribute in the FilePathVO property in FileReaderXmlVO to format the file path

[0088] Get the formate attribute in the FileFlagVO attribute in FileReaderXmlVO to format the file ready file identification

[0089] Get the FileEncodeVO property in FileReaderXmlVO and set the encoding for reading and writing files

[0090] Get the FileStartVO property in FileReaderXmlVO and set the starting line for reading and writing file data

[0091] Get the FileBatchVO property in FileReaderXmlVO and set the number of records to be processed in one batch.

[0092] Step 3. In the data information to be operated, through custom data filtering, select the table name to be operated and the data column configured in the corresponding field mapping in the table to generate the corresponding table entry SQL and persist it into the database.

[0093] In the above implementation, the file is read and written by IO through XML configuration, and the insert SQL for the defined batch is generated according to the mapping column and column type, and the persistence operation is performed. Among them, according to the Service corresponding to the FILTER configured in XML, the code extension custom data filtering is supported, including:

[0094] S3.1: Customized extended data processing;

[0095] The XML source data cleaning tags are as follows:

[0096] <filter>: Custom Service for the file to be parsed, the properties include the following:

[0097] desc: current node description

[0098] Processing logic:

[0099] Get the FileFilterVO in FileReaderXmlVO and set the file data complex operation

[0100] a). If the custom Service attribute of the FILTER configuration is empty, there is no extended operation

[0101] b). If FILTER is configured with a custom Service attribute, the operation is performed according to the corresponding Service (for example: the status of a field in the filtered data is 1)

[0102] S3.2: Complex data processing;

[0103] <tablses>: The table configuration generated by the file to be parsed must contain one or more

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] <tsplit> 、 <tnums>Determine the table where the file data is entered

[0110] S3.3: Define XML data columns, table columns, and type mappings;

[0111] < / tnums> < / tsplit> Subtag, the attributes include the following: desc: current node description processing logic: get FileTablesVO in FileReaderXmlVO, set the file data table processing operation a). If there is only one TABLE subtag, it means there is only one mapped table b). If there are multiple TABLE subtags, it means there are multiple mapped tables. <tname> 、 <tsplit> 、 <tcolumn> 、 <tcoltype> 、 <tnums>Sub-tags

[0112] <tname>: The name of the table corresponding to persistence

[0113] <tsplit>: Data column segmentation flag (such as: "|", "|@|", ",")

[0114] <tcolumn>: Data columns and corresponding table column configuration

[0115] <tcoltype>: Data type corresponding to the data column

[0116] <tnums>: Total number of data columns after segmentation

[0117] Processing logic:

[0118] Get the FileTableVO in the FileTablesVO in the FileReaderXmlVO, and get the tname, tsplit, tcolumn, tcoltype, and tnums attributes

[0119] Create a For loop with the number of loops equal to the number of BATCH in FileReaderXmlVO

[0120] Read the current file data line DATA

[0121] Split the current data DATA according to the separator tsplit and the total number of splits tnums to obtain the array A

[0122] According to the data column tcolumn and data A, get the data MAP corresponding to the current table field column

[0123] Assemble the SQL string set List to be persisted in the current row through the data column type tcoltype, data MAP, and table name tname <filetosqlvo>

[0124] Finally, the corresponding data SQL statement is generated and persisted into the database.

[0125] The business request is a timer polling or service interface request. The corresponding XML configuration is obtained through the file type ID to read and write files and generate the corresponding table persistence operation.

[0126] Processing logic:

[0127] The SQL string generated in step S3.3 is concatenated with the total execution SQL, and the database DAO is called to perform the persistence operation;

[0128] If the file reading and writing is not completed, continue to execute step S3.3 to read the file again;

[0129] If the file reading and writing is completed, the process ends.

[0130] In summary, the embodiment of the present invention provides a file reading and writing method based on XML template. By using the file storage configuration, it is only necessary to configure the corresponding file storage operation. After the business is connected, the configured reading and writing are performed. For business changes, only part of the configuration needs to be modified to achieve business addition, expansion, and change. The purpose is to solve the problem of hard-coded file reading and writing, the need to restart the service after coding is completed, and the need to repeatedly develop code to expand business scenario file reading and writing.

[0131] According to a second aspect of the present invention, there is provided a file reading and writing system based on an XML template, the system comprising:

[0132] The data storage module is used to convert text files into data storage. By configuring the relevant attributes of the XML configuration file, the batch storage operation of the text files in the specified directory and ready state is performed;

[0133] The data reading and writing module is used to filter out the file fields that need to be stored, configure the data filtering and storage rules through XML, read and write the text data to be stored, and obtain the data information to be operated;

[0134] The mapping data generation module is used to select the table name to be operated and the data column of the corresponding field mapping configuration in the table through custom data screening in the data information to be operated, generate the corresponding table entry SQL and persist it into the database.

[0135] It can be understood that the file reading and writing system based on XML template provided by the present invention corresponds to the file reading and writing method based on XML template provided by the aforementioned embodiments. The relevant technical features of a file reading and writing system based on XML template can refer to the relevant technical features of a file reading and writing method based on XML template, which will not be repeated here.

[0136] The embodiment of the present invention further provides an electronic device, which may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor may call logic instructions in the memory to execute steps of a file reading and writing method based on an XML template.

[0137] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0139] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.< / filetosqlvo> < / tnums> < / tcoltype> < / tcolumn> < / tsplit> < / tname> < / tnums> < / tcoltype> < / tcolumn> < / tsplit> < / tname> : To persist the table attribute configuration, there must be another < / tablses> < / filter> < / encode> < / batch> < / start> < / flag> < / path> < / name> < / file> < / tablses> < / filter> < / encode> < / batch> < / start> < / flag> < / path> < / name> < / file> < / file> < / files>

Claims

1. A file reading and writing method based on XML template, characterized in that: The following steps are involved: Read and write files through business requests, convert text files into data and store them in the warehouse. Use XML configuration file attributes to read and write text files in the specified directory and ready state and store them in batches. Filter out the file fields that need to be stored, filter the storage rules through XML configuration data, read and write the text data to be stored, and obtain the data information to be operated; In the data information to be operated, through custom data filtering, select the table name to be operated and the data column of the corresponding field mapping configuration in the table to generate the corresponding table entry SQL and persist it into the database.

2. A file reading and writing method based on XML template according to claim 1, characterized in that: The reading and writing of files through business requests includes: Conventional file reading and writing: Generate persistence operations through mapping of file data column data, type, and table fields; Read and write multiple identical files: Obtain all files of the same type that match the current file name through file batches and then perform regular file reading and writing; Reading and writing complex data files: Through the configured persistent multi-table configuration, after the data file column separator and the number of data are adapted, the corresponding table is generated and stored in the database for the corresponding SQL persistence.

3. A file reading and writing method based on XML template according to claim 2, characterized in that: The conventional file reading and writing is represented as follows: Declare that the file read and write same attributes are empty, which is used to define the current file read and write configuration as a regular file.

4. The file reading and writing method based on XML template according to claim 2 is characterized in that: The same multi-file reading and writing is expressed as follows: Declare that the file read and write same attribute is a positive integer or n, indicating that the current file read and write are the same multiple files.

5. The file reading and writing method based on XML template according to claim 2 is characterized in that: The complex file reading and writing is represented as follows: In the persistent table configuration, multiple TABLE configuration nodes are declared, indicating that there are multiple table data to be persisted in the current data file.

6. The file reading and writing method based on XML template according to claim 1 is characterized in that: The method of filtering out the file fields to be stored, filtering the storage rules through XML configuration data, reading and writing the text data to be stored, and obtaining the data information to be operated includes: Get the file directory, file ready status, file batch, file-related code, data start row, data batch, custom processing, and table name, delimiter, data column, column type, and row data quantity configuration of the XML file read and write files through the business number, and filter the tags for the above XML source data.

7. The file reading and writing method based on XML template according to claim 1 is characterized in that: The method of selecting the table name to be operated and the data column of the corresponding field mapping configuration in the table to generate the corresponding table entry SQL by customizing data screening includes: Through XML configuration, the file is read and written by IO, and the insert SQL of the defined batch is generated according to the mapping column and column type and the persistence operation is performed. According to the corresponding service of the XML configuration filter, the code extension custom data filtering is supported, including: S3.1: Customized extended data processing; S3.2: Complex data processing; S3.3: Define XML data columns, table columns, and type mappings.

8. The file reading and writing method based on XML template according to claim 1 is characterized in that: The generating of insert SQL for defining a batch according to the mapping columns and column types and performing a persistence operation includes: Through the generated SQL string, the total execution SQL is spliced, the database DAO is called, and the persistence operation is performed; If the file reading and writing is not completed, continue to execute step S3.3 to read the file again; If the file reading and writing is completed, the process ends.

9. A file reading and writing system based on XML template, characterized in that: include: The data storage module is used to convert text files into data storage. By configuring the relevant attributes of the XML configuration file, the batch storage operation of the text files in the specified directory and ready state is performed; The data reading and writing module is used to filter out the file fields that need to be stored, configure the data filtering and storage rules through XML, read and write the text data to be stored, and obtain the data information to be operated; The mapping data generation module is used to select the table name to be operated and the data column of the corresponding field mapping configuration in the table through custom data screening in the data information to be operated, generate the corresponding table entry SQL and persist it into the database.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements a file reading and writing method based on an XML template as described in any one of claims 1 to 8.

Images