A net worth data analysis system

Abstract

The application provides a net value data analysis system, relates to the technical field of net value data analysis processing, and can reduce the customized development workload, shorten the time from days to minutes for accessing new formats; format changes do not require code modification, the system is more stable, maintenance cost is reduced; a unified analysis engine has strong reusability, reduces system redundancy; supports rapid expansion of data sources, adapts to the frequently changed submission formats of the financial industry; is configurable, traceable and reviewable throughout, and improves data accuracy and compliance.

Description

Technical Field

[0001] This invention relates to the field of net asset value (NAV) data parsing and processing technology, and more specifically, to a NAV data parsing system. Background Technology

[0002] As a distribution agent, we receive a large amount of net asset value (NAV) data in various formats from the custodian institutions authorized by the private fund managers, including email body tables, Excel attachments, PDF attachments, and other forms. We need to clean, convert, and match the NAV data with funds before storing it in the database for internal operations and information disclosure. Existing technologies typically develop custom processing programs for each reporting format to convert heterogeneous data into a unified format before storing it in the database. However, different custodian institutions use different formats, and even the same custodian institution may frequently change its format due to changes in reporting requirements. Therefore, existing solutions have the following drawbacks: large amount of custom program development work and long development cycle; code modification is required when the format changes or new sources are added; high system maintenance costs, poor scalability, and low stability. Therefore, we propose an improvement to this problem by proposing a net asset value data parsing system. Summary of the Invention

[0003] This invention provides a net asset value (NAV) data parsing system, comprising: Configuration Management Module: Used to provide configuration units; Data source access module: Automatically retrieves net asset value files, identifies file types, and converts them into structured text. Configuration driver parsing engine: used to read configuration rules; Data verification and matching module: Automatically matches fund entities according to the configured matching rules and outputs the verification results; Automatic data entry module: Writes the parsed data into the standard library according to the configuration.

[0004] As a preferred technical solution of this application, the configuration unit is a visual configuration interface or a structured configuration interface, which can define data source type, file parsing rules, data cleaning rules, fund matching rules and database mapping rules.

[0005] As a preferred technical solution of this application, the configuration unit is stored in XML, JSON or database.

[0006] As a preferred technical solution in this application, the configuration rules are configured as configuration files, and text extraction, table positioning, field extraction, data cleaning and transformation are performed according to the configuration rules.

[0007] As a preferred technical solution of this application, the configuration rules are implemented using a rule engine to realize the parsing logic. The rule engine is Drools or QLExpress.

[0008] As a preferred technical solution of this application, the configuration-driven parsing engine introduces a natural language processing keyword extraction method when locating tables and extracting fields, which helps to locate table boundaries and the location of fields and improves configuration fault tolerance.

[0009] As a preferred technical solution in this application, the net asset value file data formats retrieved by the data source access module include email body tables, Excel attachments, and PDF attachments.

[0010] As a preferred technical solution of this application, it also includes: Pre-configure parsing rules for a certain type of hosting institution or a certain type of reporting format in the system; The system retrieves net asset value files periodically or triggered by specific times, including emails, Excel files, and PDFs. The parsing engine automatically matches the corresponding configuration based on file characteristics; Extract key fields according to configuration. Key fields include net asset value, date, fund name, unit net asset value, and cumulative net asset value. Perform cleaning, format standardization, and outlier handling; Match fund entities according to configuration rules; If the verification passes, the data is automatically entered into the database; otherwise, it enters the manual review queue.

[0011] As a preferred technical solution of this application, the configured parsing rules include: specifying the worksheet name of the data file, the starting row position of the data, and the column mapping relationship corresponding to each field.

[0012] As a preferred technical solution in this application, the data cleaning rules include: converting date fields to a unified date format, retaining a specified number of decimal places in net value fields, and filtering out null values ​​and outliers.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: This application can reduce the workload of custom development, shortening the time to integrate with new formats from days to minutes; format changes do not require code modification, making the system more stable and reducing maintenance costs; the unified parsing engine has strong reusability and reduces system redundancy; it supports rapid expansion of data sources to adapt to the frequently changing reporting formats in the financial industry; it is fully configurable, traceable, and verifiable, improving data accuracy and compliance. Attached Figure Description

[0014] Figure 1 A schematic diagram of the net asset value data parsing system provided in this application; Figure 2 Flowchart of the net asset value data parsing system provided for this application; Figure 3 The parsing rule configuration diagram provided for this application; Figure 4The email content adaptation verification diagram provided for this application. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0017] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0018] Example 1, please refer to Figures 1-4 A net asset value (NAV) data parsing system, comprising: Configuration Management Module: Used to provide configuration units; Data source access module: Automatically retrieves net asset value (NAV) files, identifies file types, and converts them into structured text; enables automated and unified access to NAV files from multiple sources, eliminating barriers to heterogeneous data formats; Configuration driver parsing engine: used to read configuration rules; Data verification and matching module: Automatically matches fund entities according to the configured matching rules and outputs the verification results; Automatic data insertion module: Writes the parsed data into the standard database according to the configuration, supports log retention and rollback; relies on fixed configuration to standardize the data insertion logic, avoiding field confusion and data mis-storage issues. At the same time, the log retention and rollback mechanism can quickly repair data insertion anomalies, ensuring the integrity and stability of database data.

[0019] Example 2 further optimizes the net asset value data parsing system provided in Example 1. Specifically, the configuration unit is a visual configuration interface or a structured configuration interface, which can define data source type, file parsing rules, data cleaning rules, fund matching rules and database mapping rules. The visual configuration interface or structured configuration interface can transform abstract rule logic into intuitive graphical operations, allowing business personnel without programming skills to independently complete rule configuration and modification, greatly improving configuration efficiency and reducing the system operation and maintenance threshold.

[0020] Example 3 further optimizes the net asset value data parsing system provided in Example 1. Specifically, the configuration unit is XML, JSON, or database storage configuration; XML and JSON formats are lightweight, easy to transmit, and easy to parse, and are suitable for temporary configuration scenarios; database storage supports persistent management and fast retrieval of massive configurations, and is suitable for long-term complex business scenarios. Multiple storage modes are compatible to meet different operation and maintenance needs.

[0021] Example 4 further optimizes the net asset value data parsing system provided in Example 1. Specifically, the configuration rules are configured in a configuration file, and text extraction, table positioning, field extraction, data cleaning and transformation are performed according to the configuration rules, without the need to write any code.

[0022] Example 5 further optimizes the net asset value data parsing system provided in Example 1. Specifically, the configuration rules are implemented using a rule engine to realize the parsing logic. The rule engine is Drools or QLExpress. The system integrates mainstream open-source rule engines such as Drools and QLExpress, transforming static configuration rules into dynamically executable business rule expressions. It supports complex multi-condition judgments, logical operations, and data linkage rules, and can realize personalized and complex net asset value data parsing, matching, and cleaning logic.

[0023] Example 6 further optimizes the net asset value data parsing system provided in the above examples. Specifically, when configuring the parsing engine to locate tables and extract fields, a natural language processing (NLP) keyword extraction method is introduced to assist in locating table boundaries and field positions, improving configuration fault tolerance. Traditional pure configuration parsing has zero fault tolerance for file format changes, and parsing will fail if there are slight row or column offsets or table format changes. Natural language processing (NLP) assisted positioning can adapt to small format changes without frequent configuration modifications, reducing the parsing failure rate.

[0024] Furthermore, the net asset value (NAV) file data formats retrieved by the data source access module include email body tables, Excel attachments, and PDF attachments. Currently, custodian institutions mainly use email tables, Excel, and PDF formats for NAV reporting. This multi-format compatibility can fully cover industry reporting scenarios, eliminating the need for manual file format conversion and achieving full coverage of data access.

[0025] Furthermore, it also includes: Pre-configure parsing rules for a certain type of hosting institution or a certain type of reporting format in the system; The system retrieves net asset value files periodically or triggered by specific times, including emails, Excel files, PDFs, etc. The parsing engine automatically matches the corresponding configuration based on file characteristics; Extract key fields according to configuration. Key fields include net asset value, date, fund name, unit net asset value, cumulative net asset value, etc. Perform cleaning, format standardization, and outlier handling; The fund entity is matched according to the configuration rules; the matching strategies used include exact matching of fund names, fuzzy matching of fund names, matching of custodian codes, or a combination thereof. If the verification passes, the data is automatically entered into the database; otherwise, it enters the manual review queue.

[0026] The configured parsing rules include: specifying the worksheet name of the data file, the starting row position of the data, and the column mapping relationship of each field.

[0027] Data cleaning rules include: converting date fields to a uniform date format, retaining a specified number of decimal places for net value fields, and filtering out null and outlier values.

[0028] Furthermore, the configuration and execution logic of this application are completely decoupled, and rule changes do not affect the main program; it supports dynamic loading of configurations without restarting the service; a single parsing engine adapts to multi-source heterogeneous data; and a configuration-driven multi-level matching strategy improves the automatic matching rate.

[0029] Furthermore, this application replaces custom code with configuration, enabling access through visual configuration without the need to write parsing programs for each format, achieving zero-code or low-code expansion; format changes do not require modification of system code, and custodians only need to modify configuration items to take effect when adjusting formats, reducing maintenance costs; a unified parsing engine adapts to multi-source heterogeneous data, with one engine supporting multiple net asset value reporting formats such as email body, Excel, and PDF, demonstrating strong scalability; configuration-driven full-process automation, from data extraction, cleaning, matching to warehousing, the entire process is configuration-driven, improving stability and reusability.

[0030] like Figure 2 As shown, let's take email processing as an example: Start collecting and managing emails, and parse the emails according to the configured parsing rules. If the parsing fails, mark it as an exception and collect and manage emails again. If the parsing is successful, write it to the raw database for manual and automatic quality inspection. If it passes, write it to the standard database and end. If it fails, mark it as an exception and write it to the raw database again.

[0031] Taking the net asset value data in Excel format submitted by a certain custodian institution as an example: Create a new configuration in the system, with the data source type being Excel, specifying Sheet1, and the data starting row being the 3rd row. Configure the field mapping: column A corresponds to the fund name, column D corresponds to the unit net value, and column F corresponds to the net value date. Configure the cleaning rules to convert the date field to yyyy-MM-dd format, retain 4 decimal places in the net value field, and filter out null and outlier values. Configure matching rules to match fund entities by fund name plus custodian code; After receiving email attachments, the system automatically identifies and matches the configuration, and completes the parsing, cleaning, matching and storage operations without the need for development, deployment or system restart.

[0032] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0033] Obviously, the embodiments described above are merely some embodiments of the present invention, not all embodiments. The accompanying drawings show preferred embodiments of the present invention, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of this invention.

Claims

1. A net asset value (NAV) data parsing system, characterized in that, include: Configuration Management Module: Used to provide configuration units; Data source access module: Automatically retrieves net asset value files, identifies file types, and converts them into structured text. Configuration driver parsing engine: used to read configuration rules; Data verification and matching module: Automatically matches fund entities according to the configured matching rules and outputs the verification results; Automatic data entry module: Writes the parsed data into the standard library according to the configuration.

2. The net asset value data parsing system according to claim 1, characterized in that, The configuration unit is a visual configuration interface or a structured configuration interface, which can define data source type, file parsing rules, data cleaning rules, fund matching rules and database mapping rules.

3. The net asset value data parsing system according to claim 1, characterized in that, The configuration unit is stored in XML, JSON, or a database.

4. The net asset value data parsing system according to claim 1, characterized in that, The configuration rules are configured in a configuration file, and text extraction, table positioning, field extraction, data cleaning and transformation are performed according to the configuration rules, without the need to write any code.

5. The net asset value data parsing system according to claim 1, characterized in that, The configuration rules use a rule engine to implement the parsing logic. The rule engine can be Drools or QLExpress.

6. The net asset value data parsing system according to claim 1, characterized in that, The configuration-driven parsing engine incorporates natural language processing keyword extraction during table positioning and field extraction to assist in locating table boundaries and field positions, thereby improving configuration fault tolerance.

7. The net asset value data parsing system according to claim 1, characterized in that, The net asset value (NAV) file data formats retrieved by the data source access module include email body tables, Excel attachments, and PDF attachments.

8. The net asset value data parsing system according to claim 1, characterized in that, Also includes: Pre-configure parsing rules for a certain type of hosting institution or a certain type of reporting format in the system; The system retrieves net asset value files periodically or triggered by specific times, including emails, Excel files, and PDFs. The parsing engine automatically matches the corresponding configuration based on file characteristics; Extract key fields according to configuration. Key fields include net asset value, date, fund name, unit net asset value, and cumulative net asset value. Perform cleaning, format standardization, and outlier handling; Match fund entities according to configuration rules; If the verification passes, the data is automatically entered into the database; otherwise, it enters the manual review queue.

9. The net asset value data parsing system according to claim 8, characterized in that, The configured parsing rules include: specifying the worksheet name of the data file, the starting row position of the data, and the column mapping relationship of each field.

10. The net asset value data parsing system according to claim 8, characterized in that, Data cleaning rules include: converting date fields to a uniform date format, retaining a specified number of decimal places for net value fields, and filtering out null and outlier values.

Images