File analysis method and device based on database configuration relation, medium and product
Through the connection and configuration information mapping with MySQL database, data is parsed and saved, data interaction efficiency and security issues between financial institutions and agency agencies are solved, and efficient and flexible file parsing and data transmission are achieved.
Patent Information
- Application Number
- CN202411288085.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-31
AI Technical Summary
The efficiency and security of data interaction between financial institutions and agency sales agencies are difficult to ensure, especially when the data transmission file versions between different agency sales agencies are constantly upgraded and modified, there is a lack of efficient and flexible file analysis methods.
By establishing a connection with the MySQL database, reading the agency and version number information in the configuration maintenance relationship table, building a field mapping table, mapping the configuration information to Java objects, parsing the pending files and saving them in batches to the target database, combined with Java's JDBC framework and ibatis framework implement data processing and preservation.
It improves the efficiency and accuracy of data interaction between financial institutions and agency sales agencies, supports the transmission and analysis of different versions of data, enhances the flexibility and security of the system, and simplifies the development process of connecting new agency sales agencies.
Smart Images
Figure CN119226261A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of file parsing, and in particular to a file parsing method, device, medium and product based on database configuration relationship. Background Art
[0002] With the rapid growth of financial product agency sales, the traditional offline agency sales model is gradually shifting to an online model. In this process, the same financial institution often needs to connect with an increasing number of agency sales agencies. At the same time, in order to ensure the efficiency and security of data transmission, the data transmission file version interacting with the agency sales agencies is also constantly being upgraded and transformed.
[0003] In order to adapt to and connect with different sales agencies more efficiently and conveniently, an efficient and flexible file parsing method is needed. Summary of the invention
[0004] The purpose of this application is to provide a file parsing method, device, medium and product based on database configuration relationship, which improves the efficiency and accuracy of data interaction between financial institutions and sales agencies.
[0005] To achieve the above objectives, this application provides the following solutions:
[0006] In a first aspect, the present application provides a file parsing method based on a database configuration relationship, comprising:
[0007] Establish a connection with the MySQL database;
[0008] Read the configuration maintenance relationship table in the MySQL database, obtain the configuration information of the agency and the version number, and build a field mapping table based on the configuration information;
[0009] Mapping the configuration information into a Java object;
[0010] Get the file to be parsed;
[0011] Parsing the file to be parsed to obtain parsed data, and saving the parsed data in batches into a target database; the target database is a database of a financial institution;
[0012] Close the connection to the MySQL database.
[0013] Optionally, before establishing a connection to the MySQL database, also include:
[0014] In the MySQL database, create the configuration relationship maintenance table.
[0015] Optionally, establish a connection to a MySQL database, including:
[0016] Build a connection with the MySQL database through Java's JDBC framework.
[0017] Optionally, the fields in the configuration maintenance relationship table include: ID, sales agency, version number, file field name, target database field name, data type, data length limit, whether required, effective status and effective date.
[0018] Optionally, the configuration maintenance relationship table in the MySQL database is read to obtain configuration information of the sales agency and the version number, and a field mapping table is constructed based on the configuration information, including:
[0019] Based on the SQLMaps of the Java persistence layer framework ibatis, the query is executed through a query statement to retrieve the configuration information of the agency and version number from the configuration maintenance relationship table;
[0020] A field mapping table is constructed using the configuration information; wherein the key of the field mapping table is the file field name, and the value of the field mapping table is the target database field FieldInfo object.
[0021] Optionally, parsing the file to be parsed to obtain parsed data, and saving the parsed data in batches into a target database includes:
[0022] According to the file path and file name, use the Java file IO library FileInputStream to open and read the file to be parsed;
[0023] Use the read() method to read the file content of the file to be parsed byte by byte, and store the file in a byte array;
[0024] Split the file content into multiple lines according to the row separator rowSeparator, use the split() method of String to split the file content into a string array according to the row separator "\r\n", split each row of data into multiple fields according to the field separator fieldSeparator, and for each row of data, use the split() method to split it into a string array according to the field separator "\t";
[0025] Traverse each row of the split string array and perform logic verification. According to the field name and data type, use the reflection API property accessor to map the field value to the attribute of the Java object, and perform corresponding data type conversion according to the data type; the logic verification includes: whether it is required and the data length limit;
[0026] Create an ArrayList collection for storing parsed Java objects, add each row of parsed Java objects to the ArrayList collection, and use the object-relational mapping framework to save the parsed data in batches to the target database.
[0027] Optionally, the file parsing method based on database configuration relationship further includes:
[0028] Setting up an exception capture and processing mechanism to classify and process the captured exception information according to the error type; the classification processing includes: recording error logs, sending alarm notifications, and performing rollback operations;
[0029] The log information is recorded using the log framework; the log information includes: key operations, abnormal information and system operation status.
[0030] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned file parsing methods based on database configuration relationships.
[0031] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned file parsing methods based on database configuration relationships.
[0032] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned file parsing methods based on database configuration relationships.
[0033] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0034] The present application discloses a file parsing method, device, medium and product based on database configuration relationship. First, a connection with a MySQL database is established; then, a configuration maintenance relationship table in the MySQL database is read to obtain configuration information of a sales agency and a version number, and a field mapping table is constructed based on the configuration information; secondly, the configuration information is mapped to a Java object; the file to be parsed is obtained; thirdly, the file to be parsed is parsed to obtain parsed data, and the parsed data is saved in batches to a target database; the target database is a database of a financial institution; finally, the connection with the MySQL database is closed. Through simple configuration, the reading and parsing of data transmission files of different versions can be realized, thereby greatly improving the efficiency and accuracy of data interaction between financial institutions and sales agencies. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0036] Figure 1 A schematic diagram of a file parsing method based on a database configuration relationship provided in an embodiment of the present application;
[0037] Figure 2 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0039] The purpose of this application is to provide a file parsing method, device, medium and product based on database configuration relationship, aiming to improve the efficiency and accuracy of data interaction between financial institutions and sales agencies.
[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0041] In an exemplary embodiment, Figure 1 As shown, the file parsing method based on database configuration relationship in this embodiment includes:
[0042] Step 1: Establish a connection to the MySQL database.
[0043] As an optional implementation manner, before step 1, the method further includes:
[0044] In the MySQL database, create a configuration relationship maintenance table.
[0045] As an optional implementation, step 1 includes:
[0046] Build a connection with the MySQL database through Java's JDBC framework.
[0047] Specifically, when building a secure and stable connection with a MySQL database through the Java Database Connectivity (JDBC) framework of Java, it is necessary to configure information such as the database driver, connection Uniform Resource Locator (URL), user name and password.
[0048] As an optional implementation, the fields in the configuration maintenance relationship table include: ID, sales agency, version number, file field name, target database field name, data type, data length limit, required or not, effective status and effective date.
[0049] Specifically, the configuration relationship maintenance table is used to store the mapping relationship between the fields in the interactive data transmission files of different sales agencies (i.e., the files to be parsed) and the target database fields. By maintaining the relationship configuration maintenance table, the data mapping relationship of different sales agencies can be flexibly managed. This method not only supports the simultaneous connection of multiple sales agencies, but also improves the scalability and adaptability of the system, and can better meet the changes in business needs.
[0050] The meanings of the fields in the configuration maintenance relationship table are as follows:
[0051] ID is a unique identifier, which is used to uniquely identify each data record. Sales agency is the code of the sales agency, which is used to distinguish the configuration information of different sales agencies. Version number is the version number of the data configuration, which supports version management of configuration information. File field name is the field name in the file to be parsed. Target database field name is the corresponding field name in the target database. Data type is the data type of the field, such as string, integer, date, etc. Data length limit is the maximum length limit of the field. Whether required is the mark of whether the field is required. Effective status is the mark of whether the configuration is effective. Effective date is the date when the configuration takes effect.
[0052] Step 2: Read the configuration maintenance relationship table in the MySQL database, obtain the configuration information of the agency and version number, and build a field mapping table based on the configuration information.
[0053] As an optional implementation, step 2 includes:
[0054] Step 21: Based on the SQLMaps of the Java persistence layer framework ibatis, execute the query through the query statement to retrieve the configuration information of the agency and version number from the configuration maintenance relationship table.
[0055] Step 22: Use the configuration information to build a field mapping table (Map<String,FieldInfo> ); the key of the field mapping table is the file field name, and the value of the field mapping table is the target database field FieldInfo object.
[0056] Step 3: Map the configuration information into Java objects.
[0057] Step 4: Get the file to be parsed.
[0058] Step 5: parse the file to be parsed to obtain the parsed data, and save the parsed data in batches into a target database; the target database is a database of a financial institution.
[0059] As an optional implementation, step 5 includes:
[0060] Step 51: According to the file path and file name, use the Java file IO library FileInputStream to open and read the file to be parsed.
[0061] Step 52: Use the read() method to read the file content of the file to be parsed byte by byte, and store the file in a byte array.
[0062] Step 53: Split the file content into multiple lines according to the row separator rowSeparator, use the split() method of String to split the file content into a string array according to the row separator "\r\n", split each line of data into multiple fields according to the field separator fieldSeparator, and for each line of data, use the split() method to split it into a string array according to the field separator "\t".
[0063] Step 54: Traverse each row of the split string array and perform logical verification. According to the field name and data type, use the reflection API property accessor to map the field value to the attribute of the Java object, and perform the corresponding data type conversion according to the data type; the logical verification includes: whether it is required and the data length limit.
[0064] Specifically, data type conversion includes but is not limited to: Integer.parseInt(), Double.parseDouble().
[0065] Step 55: Create an ArrayList collection for storing parsed Java objects, add each row of parsed Java objects to the ArrayList collection, and use the Object-Relational Mapping (ORM) framework to save the parsed data in batches to the target database.
[0066] Step 6: Close the connection to the MySQL database.
[0067] Specifically, the close() method of the connection in the database connection object JDBC is called to close the connection to ensure that the system resources occupied by the connection are released back to the system in a timely and correct manner, prevent resource leakage, and ensure efficient management of system resources and optimization of application performance.
[0068] As an optional implementation, the file parsing method based on the database configuration relationship also includes:
[0069] Set up an exception capture and processing mechanism to classify and process the captured exception information according to the error type; classification processing includes: recording error logs, sending alarm notifications, and performing rollback operations.
[0070] Use the log framework to record log information; log information includes: key operations, exception information and system operation status.
[0071] Specifically, use the logging framework to record log information to ensure the robustness and reliability of the program for subsequent problem tracking and system maintenance.
[0072] Advantages of this application: By configuring the database and using Java technology to implement file parsing, the flexibility and configurability of the system are significantly improved, code redundancy is reduced, the development process of docking with new distribution agencies is accelerated, and the scalability and maintainability of the system are enhanced. All parsing rules are centrally managed in the database, which is convenient for dynamic adjustment and monitoring, and supports dynamic updates at runtime, ensuring the efficient operation of the system. In addition, the dual security measures of the database and Java are combined to effectively improve the overall security performance of the system.
[0073] In an exemplary embodiment, a computer device is provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a file parsing method based on a database configuration relationship.
[0074] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a file parsing method based on a database configuration relationship is implemented.
[0075] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements a file parsing method based on a database configuration relationship when executed by a processor.
[0076] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 2 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a file parsing method based on a database configuration relationship is implemented.
[0077] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0079] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0080] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0081] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0082] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A file parsing method based on database configuration relationship, characterized in that: The file parsing method based on database configuration relationship includes: Establish a connection with the MySQL database; Read the configuration maintenance relationship table in the MySQL database, obtain the configuration information of the agency and the version number, and build a field mapping table based on the configuration information; Mapping the configuration information into a Java object; Get the file to be parsed; Parsing the file to be parsed to obtain parsed data, and saving the parsed data in batches into a target database; the target database is a database of a financial institution; Close the connection to the MySQL database.
2. The file parsing method based on database configuration relationship according to claim 1, characterized in that: Before establishing a connection to the MySQL database, also include: In the MySQL database, create the configuration relationship maintenance table.
3. The file parsing method based on database configuration relationship according to claim 1, characterized in that: Establish a connection to the MySQL database, including: Build a connection with the MySQL database through Java's JDBC framework.
4. The file parsing method based on database configuration relationship according to claim 1, characterized in that: The fields in the configuration maintenance relationship table include: ID, sales agency, version number, file field name, target database field name, data type, data length limit, whether required, effective status and effective date.
5. The file parsing method based on database configuration relationship according to claim 1, characterized in that: Read the configuration maintenance relationship table in the MySQL database to obtain the configuration information of the agency and the version number, and build a field mapping table based on the configuration information, including: Based on the SQLMaps of the Java persistence layer framework ibatis, the query is executed through a query statement to retrieve the configuration information of the agency and version number from the configuration maintenance relationship table; A field mapping table is constructed using the configuration information; wherein the key of the field mapping table is the file field name, and the value of the field mapping table is the target database field FieldInfo object.
6. The file parsing method based on database configuration relationship according to claim 1, characterized in that: Parsing the file to be parsed to obtain parsed data, and saving the parsed data in batches into a target database, including: According to the file path and file name, use the Java file IO library FileInputStream to open and read the file to be parsed; Use the read() method to read the file content of the file to be parsed byte by byte, and store the file in a byte array; Split the file content into multiple lines according to the row separator rowSeparator, use the split() method of String to split the file content into a string array according to the row separator "\r\n", split each row of data into multiple fields according to the field separator fieldSeparator, and for each row of data, use the split() method to split it into a string array according to the field separator "\t"; Traverse each row of the split string array and perform logic verification. According to the field name and data type, use the reflection API property accessor to map the field value to the attribute of the Java object, and perform corresponding data type conversion according to the data type; the logic verification includes: whether it is required and the data length limit; Create an ArrayList collection for storing parsed Java objects, add each row of parsed Java objects to the ArrayList collection, and use the object-relational mapping framework to save the parsed data in batches to the target database.
7. The file parsing method based on database configuration relationship according to claim 1, characterized in that: The file parsing method based on database configuration relationship also includes: Setting up an exception capture and processing mechanism to classify and process the captured exception information according to the error type; the classification processing includes: recording error logs, sending alarm notifications, and performing rollback operations; The log information is recorded using the log framework; the log information includes: key operations, abnormal information and system operation status.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the file parsing method based on database configuration relationship as described in any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the file parsing method based on database configuration relationship described in any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the file parsing method based on database configuration relationship described in any one of claims 1 to 7 is implemented.