Method and system for cleaning and analyzing information management net value mail based on large model

Abstract

The invention discloses an information management net value mail cleaning and analyzing method and system based on a large model, and the method comprises the steps: obtaining a mail of a user mailbox, and extracting and storing the metadata of the mail; performing multiple identification on the mails based on a large model, confirming net value mails, and excluding non-net value mails; aiming at net value mail contents in different formats, adopting a corresponding extraction mode and carrying out data verification; standardizing the extracted net value data according to a preset term mapping table and a standardized data dictionary; performing automatic analysis based on the net value data after standardization processing; and carrying out visual display on the analysis result. According to the method, automatic identification and acquisition of net value mails are realized, net value data in mails in different formats can be extracted and automatically analyzed, and analysis results such as income trend and risk assessment are generated in real time. The method solves the problems that in the prior art, the efficiency of obtaining and analyzing the net value data in the asset management mail is low, errors are prone to occurring, format adaptation is difficult, and a dynamic analysis result is difficult to generate in time.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence net asset value email processing, and in particular to a method and system for cleaning and parsing asset management net asset value emails based on a large model. Background Technology

[0002] In the financial investment field, individual investors, families, or small and medium-sized investment institutions need to continuously track the net asset value (NAV) data of investment products (such as public funds, private funds, and bank wealth management products) to assess investment returns and adjust investment strategies. Currently, fund companies, securities companies, and third-party wealth management platforms mainly send regular (such as daily or weekly) NAV reports to investors via email. These emails contain key data such as product name, date, unit NAV, cumulative NAV, and rate of return.

[0003] With the surge in the number of asset management products, a single institution can send hundreds to thousands of net asset value emails per day, and there is no unified standard for the format of emails from different products and institutions: some use structured tables, some use unstructured paragraph descriptions, and some embed images or encrypted PDF attachments.

[0004] Currently, the industry mainly employs the following solutions for acquiring and analyzing net investment value data: Manual processing solution: Professional staff open each email, identify and extract net asset value data, manually verify the format, and then enter it into the system. The steps of this solution are "email screening → data location → manual transcription → format verification". Processing a single email takes about 3-5 minutes, which limits the daily processing volume. In addition, misunderstandings of terminology can easily lead to data errors (such as mistakenly recording "adjusted net asset value" as "unit net asset value").

[0005] Basic email filtering tools: Some email services have built-in email categorization functions (such as tagging by sender or keywords), which can classify net worth emails into specific folders. However, this can only filter emails and cannot extract net worth data from the email content. Subsequent data processing still needs to be done manually.

[0006] Financial data platform integration solution: Some third-party financial platforms (such as financial apps) provide net asset value query functions, but the data source relies on institutional API interfaces, and the update is delayed (usually 1-2 days behind), and the products covered are limited, which cannot include all customized products held by investors (such as private equity products), resulting in insufficient data integrity.

[0007] Simple OCR extraction tools: A few tools support OCR text recognition for image-formatted net worth reports in emails, but they can only extract text content and cannot recognize data logic (such as the relationship between "unit net worth" and its corresponding value). Furthermore, they have poor adaptability to the format differences of plain text emails (such as tables and paragraph layouts), resulting in low extraction accuracy.

[0008] Existing technologies for acquiring and analyzing net asset value data in asset management emails suffer from problems such as low efficiency, long processing time, susceptibility to human error, difficulties in data integration due to inconsistent email formats across different institutions, and difficulty in generating dynamic analysis results in a timely manner. Summary of the Invention

[0009] The purpose of this invention is to propose a method for cleaning and parsing asset management net asset value emails based on a large model, aiming to solve the problems of low efficiency, error-proneness, difficulty in format adaptation, and difficulty in generating dynamic analysis results in a timely manner when acquiring and analyzing net asset value data in asset management emails using existing technologies.

[0010] This invention is implemented as follows: a method for cleaning and parsing asset management net asset value emails based on a large model, the method comprising: Retrieve emails from the user's inbox, extract and store the metadata of the emails; Based on a large model, emails are identified through multiple identification processes to confirm net worth emails and exclude non-net worth emails. For different formats of net asset value email content, appropriate extraction methods are used and data verification is performed; The extracted net value data is standardized based on the pre-defined terminology mapping table and standardized data dictionary. Automated analysis is performed based on standardized net asset value data; The analysis results will be visualized.

[0011] In this embodiment of the invention, the step of performing multiple identifications on emails based on a large model to confirm net worth emails and exclude non-net worth emails includes: Pre-set sender feature database and historical sender list for sender identification; A pre-defined keyword library for net investment value is used to match keywords in email titles or the first paragraph of the email body. By using a pre-trained lightweight BERT model, semantic recognition and analysis are performed on the email body to identify net worth emails and exclude non-net worth emails.

[0012] In this embodiment of the invention, the extraction method and data verification for net asset value email content in different formats are as follows: HTML emails: Locate data regions using structured parsing tools; extract key-value pairs using regular expressions; identify entities in unstructured text using a pre-trained Named Entity Recognition (NER) model; PDF attachment emails: By calling the OCR engine to recognize text in images or PDF content, the text is converted into editable text; for table-type images, the table area is located through a table detection algorithm, and then the row and column data are extracted. Emails with Excel or CSV attachments: Parse attachments using Python's pandas library, directly read table data, and associate it with the corresponding product; Data validation: Filters erroneous data through numerical validity checks and date continuity checks; including numerical validity checks and date continuity checks; Numerical reasonableness verification: Check whether the net value is within a reasonable range; Date continuity check: Checks whether the extracted dates are consecutive and excludes duplicate date data.

[0013] In this embodiment of the invention, the automated analysis based on standardized net asset value data includes: The standardized net value data is preprocessed. Based on the preprocessed net asset value data, calculate the core indicators for investment analysis; Perform sequence trend analysis and pattern recognition on the net value data; Conduct risk warning and assessment for single products; Provide personalized analysis and product recommendations.

[0014] In this embodiment of the invention, the preprocessing of the standardized net asset value data includes: Duplicate date data is removed through time series verification; For missing dates, net asset value data was completed using linear interpolation. Link user holdings information with product net asset value data.

[0015] In this embodiment of the invention, in the step of calculating the core indicators for investment analysis based on the preprocessed net asset value data, The core indicators include return indicators and risk indicators. The return indicators include daily return, cumulative return, and annualized return. The risk indicators include volatility, maximum drawdown, and Sharpe ratio.

[0016] In this embodiment of the invention, in the step of performing sequence trend analysis and pattern recognition on the net value data, The method for performing series trend analysis on net asset value data is as follows: For short-term trends, a short-term net asset value moving average is plotted using a moving average model. When the short-term moving average crosses above the long-term moving average, it is marked as an "uptrend signal," and vice versa as a "downtrend signal." For periodic analysis, Fourier transform is used to perform spectral analysis on net value data over medium- to long-term time periods to identify seasonal fluctuation patterns. The method for pattern recognition of net asset value data is as follows: The K-means clustering algorithm is used to cluster the net asset value curves of similar products, identify the similarity between the current product and the curves of historically strong / weak products, and help to judge future trends.

[0017] In this embodiment of the invention, the step of conducting single-product risk warning and assessment includes: The system presets Level 1 and Level 2 warning conditions. When a Level 1 warning condition is met, a Level 1 warning is triggered; when a Level 2 warning condition is met, a Level 2 warning is triggered. The isolated forest algorithm is used to detect net asset value outliers and identify potential risk events. Generate a risk assessment report; the risk assessment report includes a single product risk rating and an asset allocation risk matrix.

[0018] In this embodiment of the invention, in the step of performing personalized analysis and product recommendation, The methods for performing personalized analysis include, but are not limited to, the following: Performance is calculated based on the user's holding period; or Generate a suitability score based on investment objectives and provide suggestions; The methods for making product recommendations include, but are not limited to, the following: Based on collaborative filtering algorithms, recommend alternative products that have similar investment style characteristics to the user's holdings but have performed better recently; or, Based on users' historical interaction data with products, the similarity between products is calculated, and product recommendations are made using item-based collaborative filtering; or, Based on the product's historical net asset value data, calculate the Pearson correlation coefficient, and then recommend products with low correlation based on the Pearson correlation coefficient.

[0019] Another objective of this invention is to propose a large-scale model-based asset management net asset value email cleaning and parsing system. The system includes: The email retrieval module is used to retrieve emails from the user's email address, extract and store the metadata of the emails; The net asset value email recognition module is used to perform multiple recognitions on emails based on a large model, confirming net asset value emails and excluding non-net asset value emails. The net asset value (NAV) data extraction module is used to extract NAV data from emails of different formats using appropriate methods and to perform data verification. The net asset value (NAV) data standardization processing module is used to standardize the extracted NAV data according to a preset terminology mapping table and a standardized data dictionary. The net asset value (NAV) data analysis module is used for automated analysis based on standardized NAV data. The analysis results display module is used to visualize the analysis results.

[0020] Beneficial effects of the present invention This invention discloses a method and system for cleaning and parsing asset management net asset value (NAV) emails based on a large-scale model. The method includes: acquiring emails from a user's inbox; extracting and storing the email's metadata; performing multi-factor identification on the emails based on a large-scale model to confirm NAV emails and exclude non-NAV emails; employing appropriate extraction methods and performing data verification for NAV emails of different formats; standardizing the extracted NAV data according to a preset terminology mapping table and a standardized data dictionary; performing automated analysis based on the standardized NAV data; and visualizing the analysis results. This invention enables automatic identification and acquisition of NAV emails, extracts NAV data from emails of different formats, performs automated analysis, and generates real-time analysis results such as return trends and risk assessments. This invention solves the problems of low efficiency, error-proneness, difficulty in format adaptation, and inability to generate timely dynamic analysis results in existing technologies for acquiring and analyzing NAV data from asset management emails. Attached Figure Description

[0021] Figure 1 This is a flowchart of a preferred embodiment of the present invention for cleaning and parsing asset management net asset value emails based on a large model; Figure 2 This is a structural diagram of a preferred embodiment of the present invention for a large-scale model-based asset management net asset value email cleaning and parsing system. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. For ease of explanation, only the parts related to the embodiments of this invention are shown. It should be understood that the specific embodiments described herein are merely for explaining this invention and are not intended to limit this invention.

[0023] This invention proposes a method and system for cleaning and parsing asset management net asset value (NAV) emails based on a large-scale model. The method includes: acquiring emails from a user's inbox; extracting and storing the email's metadata; performing multi-factor identification on the emails based on a large-scale model to confirm NAV emails and exclude non-NAV emails; employing appropriate extraction methods and performing data verification for NAV emails of different formats; standardizing the extracted NAV data according to a preset terminology mapping table and a standardized data dictionary; performing automated analysis based on the standardized NAV data; and visualizing the analysis results. This invention enables automatic identification and acquisition of NAV emails, extracts NAV data from emails of different formats, performs automated analysis, and generates real-time analysis results such as return trends and risk assessments. This invention solves the problems of low efficiency, error-proneness, difficulty in format adaptation, and inability to generate timely dynamic analysis results in existing technologies for acquiring and analyzing NAV data from asset management emails.

[0024] Figure 1 This is a flowchart of a preferred embodiment of the present invention for a method of cleaning and parsing asset management net asset value emails based on a large model; the method includes the following steps: S1, retrieve emails from the user's mailbox, extract and store the metadata of the emails; The metadata of the email includes, but is not limited to, sender, subject, sending time, content type, and email body.

[0025] In this embodiment of the invention, a secure connection can be established with the user's email address (such as NetEase Mail, QQ Mail, Enterprise Mail, etc.) via the IMAP / POP3 protocol. After authorization by the user, emails can be synchronized at regular intervals (configurable to 8:00 AM or 8:00 PM daily) or in real time.

[0026] In another embodiment of the present invention, the OAuth2.0 authorization mechanism is used to achieve secure email access and avoid storing account passwords in plaintext; new emails are obtained through an incremental synchronization algorithm, thereby reducing the amount of data transmission and reducing the pressure on the email server. The method of obtaining new emails through incremental synchronization is common knowledge in the industry and will not be described in detail here.

[0027] In this embodiment of the invention, multiple email configurations are supported (e.g., up to 10 email accounts can be associated). After synchronization, email metadata (sender, title, sending time, content type) is stored in a local database to provide a data foundation for subsequent identification.

[0028] S2, based on a large model, performs multiple identifications on emails to confirm net worth emails and exclude non-net worth emails. This invention provides a method for performing multiple identifications on emails based on a large model, confirming net worth emails and excluding non-net worth emails, including the following steps (S21-S23): S21, Preset sender feature database and historical sender list, and perform sender identification; For example, in one embodiment of the present invention, if the sender's features match successfully or the sender is a historical sender, then the sender of the high-probability net worth email is marked and the process proceeds to step S22; otherwise, the process proceeds directly to step S22. In this embodiment of the invention, the sender characteristic database includes the official email domains of mainstream financial institutions, for example: Public funds: @chinaamc.com, @efunds.com, @huatai-pb.com, etc.; Securities firm asset management: @citics.com, @gtjas.com, etc.; Bank wealth management: @cmbchina.com, @icbc.com.cn, etc.; Third-party platforms: @lufunds.com, @zhaocaijing.com, etc.; In this embodiment of the invention, the sender feature matching involves extracting the domain name portion of the email's From field and performing a precise match with a domain name database. S22, a preset investment net value keyword library, performs keyword matching on email titles or the first paragraph of email body; For example, in one embodiment of the present invention, if the keywords in both the email title and the first paragraph of the email body match successfully, then the email is marked as a net worth email and proceeds to step S3; otherwise, proceeds to step S23. For example, in one embodiment of the present invention, if it is determined that the email title is successfully matched but the keywords in the first paragraph of the email body are not successfully matched, then proceed to step S23; The investment net asset value-related keywords include: Net asset value related information: Net asset value report, unit net asset value, cumulative net asset value, net asset value update, etc. Return-related information: return reports, portfolio returns, monthly returns, rate of return, etc. Reports include: investment reports, operational reports, quarterly reports, annual reports, etc. Account-related information: asset changes, holding details, account reports, etc. The method for keyword matching in email titles or the first paragraph of email body is through regular expression matching; it includes the following steps (steps S221-S224): S221, Define the keyword list; The keywords include various expressions that may appear in the net asset value email; for example, keyword = ['net asset value', 'unit net asset value', 'cumulative net asset value', 'rate of return', 'holdings', 'assets', 'investment report']; S222 connects the list of keywords using the pipe symbol (|) to form a regular expression pattern; For example, construct a regular expression, pattern = r'Net Asset Value|Unit Net Asset Value|Cumulative Net Asset Value|Return|Holdings|Assets|Investment Report'; S223, Extract the title and first paragraph of the body from email data; For example, the first 200 characters of the main text or the first paragraph; S224, Use regular expressions to search for matches in the headings and the first paragraph of the body text; For example, the email subject line could be: "Notification of Updated Net Asset Value of ChinaAMC Growth Fund". The regular expression matching found "unit net value" and determined it to be a net value email; S23 uses a pre-trained lightweight BERT model to perform semantic recognition and analysis on the email body, identifying net worth emails and excluding non-net worth emails. In this embodiment of the invention, a lightweight BERT model is pre-trained to perform semantic understanding of the email body, identifying core semantics of net worth emails such as net worth, rate of return, and cumulative returns, while excluding non-net worth emails (such as spam or marketing emails). The model parameters are updated by having users manually label misjudged emails, continuously improving recognition accuracy. Misjudged emails are those whose model predictions do not match the true labels, including incorrectly identifying non-net worth emails as net worth emails and incorrectly identifying net worth emails as non-net worth emails.

[0029] An embodiment of the present invention provides a method for training a lightweight BERT model, comprising the following steps (steps S231-S235): S231, Prepare labeled data: Collect an email dataset and manually label each email as either "net worth email" or "non-net worth email". S232, Model Learning: The email dataset is input into the BERT model for training (fine-tuning). The model attempts to predict the category of the emails and compares the prediction results with the true labels.

[0030] S233, Parameter Update: Based on the prediction error, the parameters inside the model are fine-tuned through backpropagation of the algorithm. During the fine-tuning process, the model adjusts hundreds of millions of internal parameters, gradually learning to strongly associate specific words, sentence structures, and contextual patterns (i.e., "core semantics") with "net worth email" tags, enhancing the recognition of net worth-related contexts, and learning to ignore promotional and advertising language.

[0031] S234, Forming a dedicated model: After training, the model becomes a dedicated model for recognizing net worth emails, which can make decisions based on the learned, deep semantic features and accurately identify the context of core semantics.

[0032] S235, Continuous Optimization and Domain Adaptation: Through continuous learning via user feedback (marking misjudged emails), the model continuously adapts to the email styles received by specific users and the expression habits of specific financial companies, further enhancing its specialization.

[0033] S3 employs appropriate extraction methods and performs data verification for net asset value email content in different formats. In this embodiment of the invention, the extraction method used for net worth email content in different formats is as follows: HTML emails: Locate data areas (such as table tags) using structured parsing tools.

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] Figure 2

[0059]

[0060]

[0061]

[0062]

[0063]

[0064] 1. Bullet list; 2. Extract key-value pairs (e.g., "Product Name: XXX" "Unit Net Value: 1.0523"); 3. Identify entities (e.g., product name, numerical value, date) in unstructured text using a pre-trained Named Entity Recognition (NER) model; 4. PDF attachment emails: Perform text recognition on images or PDF content using an OCR engine (e.g., Tesseract-OCR) and convert it into editable text; 5. For table-type images, locate table areas using table detection algorithms (e.g., YOLOv5 table detection model) and then extract row and column data. 6. Excel / CSV attachment emails: Parse attachments using Python's pandas library, directly read table data, and associate it with the corresponding product. 7. Data validation: Filter erroneous data through numerical reasonableness and date continuity checks; including numerical reasonableness and date continuity checks; Numerical reasonableness check: Check if the net value is within a reasonable range (e.g., 0.5-5.0, but the specific range can be adjusted according to the product type). Date continuity check: Check if the extracted dates are consecutive and exclude duplicate date data.For example, in one embodiment, a pre-trained Named Entity Recognition (NER) model is used to recognize entities (such as product names, numerical values, dates, etc.) in unstructured text, including the following steps (steps S31-S35): S31, Model Selection: Select a pre-trained NER model; for example, a BERT-based model that has been trained on a large corpus and can recognize common entity types (such as names, locations, times, etc.); in practical applications, to adapt to the financial field, a NER model can be fine-tuned for financial text, or a general model can be used and then adapted to the domain; S32, Text Preprocessing: Preprocess the unstructured text in the email body; including segmentation and cleaning (removing irrelevant characters, standardizing format, etc.); S33, Entity Recognition: Input the preprocessed text into the NER model, and the model recognizes the entities in the text and labels their types; for example, the model recognizes "1.03" as a number. S34, Post-processing: Based on business needs, post-process the entities identified by the model; for example: merging adjacent entities (e.g., merging "unit" and "net value" into "unit net value" as a single entity); entity normalization (unifying different date formats to a standard format, converting percentage values ​​to decimals, etc.); relation extraction (attempting to establish relationships between entities, such as associating values ​​with product names); S35, Output: Outputting the identified entities in a structured form; for example, in JSON format, containing entity text, type, position in the text, etc.; S4, Standardizing the extracted net value data according to a preset terminology mapping table and a standardized data dictionary; the terminology mapping table stores the correspondence between terms used by different organizations and standard terms. For example, if one institution uses "net value per unit" and another uses "net value per unit price", they are both mapped to the standard term "net value per unit" defined in the standardized data dictionary; for example, "annualized rate of return" corresponds to "annual return rate", etc.; the standardized data dictionary stores the specifications that the standardized data should follow, including the definition of standard terms, data format, data type, value range, etc., and defines the target format for standardization.For example, data format is standardized: date format is standardized to "YYYY-MM-DD", numerical values ​​are retained to 4 decimal places, and the rate of return is converted to percentage form (e.g., 0.032 → 3.2%); unique product identifier: a unique product ID is generated by combining "institution name + full product name + code" to solve the problem of different names for the same product (e.g., "Huaxia Growth Fund" and "Huaxia Growth Hybrid (000001)" are considered the same product); in this embodiment of the invention, terms in the terminology mapping table are automatically aligned using a fuzzy matching algorithm; the fuzzy matching algorithm can be Jaro-Winkler similarity algorithm, cosine similarity algorithm, edit distance, etc., all of which are common industry knowledge and will not be elaborated here; in this embodiment of the invention, the standardized net asset value data is stored. The central database supports multi-dimensional indexing (e.g., by product, date, institution, etc.) for easy subsequent analysis and retrieval; S5, based on the standardized net asset value data, perform automated analysis; This embodiment of the invention provides a method for automated analysis based on standardized net asset value data, including the following steps (steps S51-S55): S51: Preprocess the standardized net asset value data; The standardized net asset value data includes basic product information (e.g., name, institution, type, etc.), historical net asset value data (e.g., date, unit net asset value, cumulative net asset value, etc.), and user holding information (e.g., holding shares, holding cost, holding date, etc.). This invention provides a method for preprocessing standardized net asset value (NAV) data, including the following steps (steps S511-S513): S511, removing duplicate date data through time series verification; removing duplicate date data through time series verification involves identifying multiple records of the same product on the same day, and then selecting the best record to retain based on data source priority or timestamp, while deleting other duplicate records; S512, completing NAV data for missing dates using linear interpolation; for example, if a product is missing NAV on 2023-10-07, then the median value is calculated based on the NAVs on 2023-10-06 and 2023-10-08; the linear interpolation formula is: y=y1+((x-1)×(y2-y1)) / (x2-x1); where x1=2023-10-06, y1=1.0350; x2=2023-10-08, y2=1.0400; x=2023-10-07, where y represents the missing net value for 2023-10-07; in the time series, x1, x2, x can be numerical values ​​corresponding to dates (such as timestamps or consecutive integers).For simplicity, consecutive integers are used, i.e., it is assumed that adjacent dates differ by 1; calculate the net value on 2023-10-07: y=1.0350+((7-6)×(1.0400-1.0350)) / (8-6)=1.0375; therefore, after interpolation, the net value on 2023-10-07 is 1.0375; S513, associate user holding information with product net value data; in this embodiment of the invention, association can be made through the product's unique ID to generate a product-holding mapping table; ensure accurate correspondence in return calculation. S52: Based on the preprocessed net asset value data, calculate the core indicators for investment analysis; in this embodiment of the invention, the core indicators include return indicators and risk indicators. The return indicators include daily return, cumulative return, and annualized return; the risk indicators include volatility, maximum drawdown, and Sharpe ratio; return indicators: daily return = (daily unit net asset value - previous day's unit net asset value) × holding shares; cumulative return = (daily unit net asset value - holding cost net asset value) × holding shares; annualized return = [(daily unit net asset value / holding cost net asset value)^(365 / holding days) - 1] × 100%; In one specific implementation, a calculation task is triggered periodically via an SQL script, setting the calculation of all indicators to be completed within 10 minutes after the daily net asset value data update; Risk indicators: Volatility = Standard deviation of net asset value returns within the interval; Maximum drawdown = (Highest net asset value within the interval - Lowest net asset value within the interval) / Highest net asset value within the interval × 100%; Sharpe ratio = (Annualized return - Risk-free rate) / Annualized volatility; In this embodiment of the invention, the statistical functions are implemented using Python's numpy library, and users can customize the calculation interval (e.g., the past 7 days, the past 30 days, the past 90 days); Volatility reflects short-term volatility risk; The risk-free rate can be taken by default as the national average for the same period. Bond yield (e.g., 3%); S53: Perform sequence trend analysis and pattern recognition on the net asset value data; An embodiment of the present invention provides a method for sequence trend analysis of net asset value data as follows: For short-term trends, a moving average (MA) model is used to draw net asset value moving averages for short time periods (e.g., 5-day, 10-day, 30-day). When the short-term moving average (5-day) crosses above the long-term moving average (30-day), it is marked as an "upward trend signal," and vice versa as a "downward trend signal." For periodic analysis, Fourier transform is used to perform spectral analysis on net asset value data for medium- and long-term time periods (e.g., the past year); thereby identifying seasonal fluctuation patterns (e.g., the impact of quarterly interest payment cycles on bond products). The visualization presentation of this embodiment of the present invention is achieved by drawing a net asset value trend chart using ECharts, automatically marking trend inflection points (e.g., days with a single-day increase exceeding 3% or a decrease exceeding 5%). The present invention provides a method for pattern recognition of net asset value data: clustering the net asset value curves of similar products (such as equity funds) based on the K-means clustering algorithm, identifying the similarity between the curves of the current product and historically strong / weak products, and assisting in judging future trends.Even if the K-means clustering algorithm is used to cluster the net asset value curves of similar products (such as equity funds), the net asset value curves of historical products are divided into multiple clusters according to shape similarity (such as strong, mediocre, weak, etc.). Then, the distance between the current product's net asset value curve and the center of each cluster (such as Euclidean distance) is calculated to find the most similar cluster. The historical performance of this cluster is used to help judge the future trend of the current product. S54: Perform single product risk warning and assessment; This embodiment of the invention provides a method for performing risk warning and assessment, including the following steps (steps S541-S543): S541, preset the first-level warning conditions and the second-level warning conditions. When the first-level warning condition is met, the first-level warning is triggered; when the second-level warning condition is met, the second-level warning is triggered. For example, a primary warning condition can be preset as follows: the daily decline of a single product exceeds a first threshold, the cumulative decline over a period exceeds a second threshold, or the maximum drawdown over a period exceeds a third threshold; or a primary warning condition can be a combination of a single product's daily decline exceeding a first threshold, a cumulative decline over a period exceeding a second threshold, and a maximum drawdown over a period exceeding a third threshold; for example, a single product's daily decline exceeding a first threshold or a 7-day maximum drawdown exceeding a third threshold; in one specific implementation, a primary warning is automatically triggered when a single product's daily decline exceeds 5%, its cumulative decline over the past 7 days exceeds 10%, or its maximum drawdown over the past 7 days exceeds 20%; for example, a secondary warning condition can be preset as follows: the overall return of the entire account falls below a fourth threshold or the return of a single product falls below a fifth threshold; in one specific implementation, a secondary warning is triggered when the overall return of the entire account falls below 10%. It should be noted that in practical applications, similar primary and secondary warning conditions to those in this embodiment can be set according to actual circumstances, all of which are within the scope of protection of this invention. The primary and secondary warning conditions exemplified in this embodiment are not intended to limit the scope of protection of this invention. S542 utilizes the Isolation Forest algorithm to detect outliers in net asset value (NAV) (such as sudden rises and falls deviating from historical fluctuation ranges) and identify potential risk events (such as fund manager changes or defaults in heavily held stocks). The Isolation Forest algorithm is a highly efficient unsupervised anomaly detection method. Its core principle is that outlier data points often have the characteristic of being "few but distinct," thus they can be quickly isolated by randomly partitioning the feature space. In the constructed binary tree, outliers are usually closer to the root node, while normal points require more partitioning steps to be isolated. In NAV anomaly detection, NAV data (which can be NAV sequences, return sequences, etc.) is input into the Isolation Forest algorithm, which returns an anomaly score for each point. Thresholds can be set to mark outliers. These outliers may correspond to abnormal NAV fluctuations, thus indicating potential risk events. The Isolation Forest algorithm is a commonly used method in the industry and will not be elaborated further here.S543, Generate a risk assessment report; the risk assessment report includes a single product risk rating and an asset allocation risk matrix; the single product risk rating includes low, medium, and high risk levels, rated based on a comprehensive score of volatility, maximum drawdown, and Sharpe ratio (e.g., out of 100 points, ≥80 points is low risk, 60-79 points is medium risk, and <60 points is high risk); the asset allocation risk matrix calculates risk exposure according to investment type (equity, bond, mixed), and indicates concentration risk when the proportion of a single type of asset exceeds a preset threshold (e.g., 60%). S55: Perform personalized analysis and product recommendations; the personalized analysis includes, but is not limited to, the following methods: calculating performance based on the user's holding period; or generating a suitability score based on investment objectives and providing prompts; for example, calculating performance based on the user's holding period (short-term < 6 months, medium-term 6-12 months, long-term > 12 months) and comparing investment returns for different periods; for example, generating a suitability score based on investment objectives (conservative / aggressive), and prompting "risk and objective do not match" if a conservative user holds high-volatility products; the product recommendations include, but are not limited to, the following methods: based on collaborative filtering algorithms, recommending alternative products with similar investment style characteristics to the user's held products but with better recent performance (such as other products managed by the same fund manager); or, Based on users' historical interaction data with products (such as browsing, purchasing, holdings, etc.), the similarity between products is calculated, and product recommendations are made using item-based collaborative filtering; or, based on the historical net asset value data of the products, the Pearson correlation coefficient is calculated, and then products with low correlation are recommended based on the Pearson correlation coefficient; the recommendation of alternative products with similar investment style characteristics to the user's holding products but with better recent performance based on the collaborative filtering algorithm specifically involves: calculating similarity (such as cosine similarity or Euclidean distance) by analyzing the investment style characteristics of the user's holding products (such as fund category, investment industry preference, risk level, historical volatility, Sharpe ratio, etc.), and finding other products with similar styles to the user's holding products and selected by similar user groups or with better recent performance based on collaborative filtering for recommendation.For example, in one embodiment, the product recommendation method includes the following steps (S551a-S551d): S551a, collecting product feature data; for example: product feature = [fund category, industry allocation ratio, top ten holdings, fund manager, risk level, historical volatility, Sharpe ratio, maximum drawdown...]; S551b, performing one-hot encoding on categorical features and standardizing numerical features; S551c, calculating the similarity between products (e.g., using cosine similarity); it should be noted that in this embodiment, product style similarity is used as item similarity; S551d, based on the products held by the user, finding products similar to these held products, and filtering out products with better recent performance as recommendations; the calculation of product similarity based on the user's historical interaction data with products, and using item-based collaborative filtering for product recommendation specifically involves: constructing a user-product matrix (each user's holding weight for each product, such as holding amount or whether they hold the product), and then calculating the similarity between products (through the similarity of user holding vectors), which is item-based collaborative filtering in collaborative filtering. For example, in another embodiment, the product recommendation method includes the following steps (S552a-S552d): S552a, constructing a user-product holding matrix; where rows represent users, columns represent products, and values ​​represent holding amounts or 0 / 1 (indicating whether a holding exists). S552b, calculating the similarity between products (through the similarity of user holding vectors, such as cosine similarity); S552c, for the products held by the target user, identifying similar products for each held product. S552d, ranking the similar products according to similarity weights and considering recent performance (such as rate of return), recommending products that rank higher and outperform the user's current holdings. In this embodiment of the invention, the step of calculating the Pearson correlation coefficient based on the historical net asset value data of the products, and then recommending low-correlation products based on the Pearson correlation coefficient, uses the following formula for calculating the Pearson correlation coefficient: r = \frac {nΣXY-(ΣX)(ΣY)} {\sqrt {[nΣX²-(ΣX)²][nΣY²-(ΣY)²}}; where X and Y are the net asset value sequences of the two products, and n is the number of data points (sample size); the correlation coefficient r ranges from -1 to 1, r>0 indicates positive correlation, r<0 indicates negative correlation, and r=0 indicates no linear correlation. The closer |r| is to 1, the higher the linear correlation; the closer |r| is to 0, the lower the linear correlation. In practical applications, assets with low or even negative correlation with existing holdings are often added to a portfolio to diversify risk. The correlation coefficient between any two products can be obtained based on the correlation analysis of historical net asset value data. Then, for the products currently held by the user, other products with low correlation coefficients (e.g., correlation coefficients close to 0 or negative) are sought as recommendation candidates.By recommending products with low correlation, the system helps users build a more diversified investment portfolio, reducing the overall portfolio volatility. S6, the analysis results are visualized. In some embodiments of this invention, the analysis results can be displayed through a web or mobile interface, using visualization components (such as ECharts) to generate charts. The data overview page displays core indicators such as total net asset value, daily return, and cumulative return, and a dashboard displays the asset distribution ratio. The product details page displays the net asset value trend chart, return details (by day / week / month), risk indicators, and prediction curves for a single product. High-risk warnings (such as net asset value falling below the stop-loss line) are pushed via pop-ups or SMS notifications. Standardized data or analysis results can be exported to Excel and PDF formats to meet users' offline analysis needs. Corresponding to the intelligent question-answering method described in the above embodiments, a structural block diagram of a large-model-based asset management net asset value email cleaning and parsing system provided by this invention is shown. For ease of explanation, only the parts related to the embodiments of this application are shown. The system includes: an email acquisition module for acquiring emails from user email addresses, extracting and storing the email metadata; a net worth email identification module for performing multiple identifications on emails based on a large model, confirming net worth emails and excluding non-net worth emails; a net worth data extraction module for using appropriate extraction methods and performing data verification for net worth email content in different formats; a net worth data standardization processing module for standardizing the extracted net worth data according to a preset terminology mapping table and a standardized data dictionary; a net worth data analysis module for performing automated analysis based on the standardized net worth data; and an analysis result display module for visualizing the analysis results.Furthermore, the net asset value (NAV) email recognition module also includes a first recognition module for pre-setting a sender feature library and a historical sender list to identify the sender; a second recognition module for pre-setting an investment NAV keyword library to match keywords in the email title or the first paragraph of the email body; and a third recognition module for performing semantic recognition analysis on the email body using a pre-trained BERT lightweight model to identify NAV emails and exclude non-NAV emails. Furthermore, the NAV data extraction module includes: an HTML email extraction module for locating data regions using structured parsing tools; extracting key-value pairs using regular expressions; and identifying entities in unstructured text using a pre-trained Named Entity Recognition (NER) model; and a PDF attachment email module for performing text recognition on images or PDF content using an OCR engine to convert it into editable text; and for table-type images, using a table detection algorithm (locating table regions and then extracting row and column data; Excel / CSV)... The attachment email module uses Python's pandas library to parse attachments, directly read table data, and associate it with the corresponding product. The data validation module filters erroneous data through numerical reasonableness and date continuity checks. This includes a numerical reasonableness check module to verify if the net asset value (NAV) is within a reasonable range, and a date continuity check module to check if the extracted dates are consecutive and exclude duplicate dates. Further, the NAV data analysis module includes: a preprocessing module to preprocess the standardized NAV data; an investment analysis core indicator calculation module to calculate core investment analysis indicators based on the preprocessed NAV data; a sequence trend analysis and pattern recognition module to perform sequence trend analysis and pattern recognition on the NAV data; and a risk warning and assessment module to conduct single-product risk warnings and assessments. The preprocessing module includes an assessment module; a personalized analysis and product recommendation module for performing personalized analysis and product recommendations. Further, the preprocessing module includes: a duplicate date data removal module for removing duplicate date data through time series verification; a missing date net asset value data completion module for completing missing date net asset value data using linear interpolation; and an association module for associating user holdings information with product net asset value data. Further, in the investment analysis core indicator calculation module, the core indicators include return indicators and risk indicators. The return indicators include daily return, cumulative return, and annualized return; the risk indicators include volatility, maximum drawdown, and Sharpe ratio. Further, the risk warning and assessment module includes: an early warning module for presetting primary and secondary early warning conditions. When the primary early warning condition is met, a primary early warning is triggered; when the secondary early warning condition is met, a secondary early warning is triggered.The net asset value (NAV) outlier detection module is used to detect NAV outliers and identify potential risk events using the Isolation Forest algorithm. The risk assessment report generation module generates a risk assessment report, which includes a single-product risk rating and an asset allocation risk matrix. Further, the personalized analysis and product recommendation module includes: a personalized analysis module for calculating return performance based on the user's holding period; or generating a suitability score based on investment objectives and providing prompts; and a product recommendation module for recommending alternative products with similar investment style characteristics to the user's held products but with better recent performance, based on a collaborative filtering algorithm; or calculating the similarity between products based on the user's historical interaction data and using item-based collaborative filtering for product recommendations; or calculating the Pearson correlation coefficient based on the product's historical NAV data and then recommending low-correlation products based on the Pearson correlation coefficient. Those skilled in the art will understand that all or part of the steps in the above embodiments can be implemented by program instructions and related hardware. The program can be stored in a computer-readable storage medium, such as ROM, RAM, disk, or optical disk. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for cleaning and parsing asset management net asset value emails based on a large model, characterized in that, The method includes: Retrieve emails from the user's inbox, extract and store the metadata of the emails; Based on a large model, emails are identified through multiple identification processes to confirm net worth emails and exclude non-net worth emails. For different formats of net asset value email content, appropriate extraction methods are used and data verification is performed; The extracted net value data is standardized based on the pre-defined terminology mapping table and standardized data dictionary. Automated analysis is performed based on standardized net asset value data; The analysis results will be visualized.

2. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 1, characterized in that, The steps of performing multiple identifications on emails based on a large model to confirm net worth emails and exclude non-net worth emails include: Pre-set sender feature database and historical sender list for sender identification; A pre-defined keyword library for net investment value is used to match keywords in email titles or the first paragraph of the email body. By using a pre-trained lightweight BERT model, semantic recognition and analysis are performed on the email body to identify net worth emails and exclude non-net worth emails.

3. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 1, characterized in that, The extraction and data validation methods employed for different formats of net asset value email content are as follows: HTML emails: Locate data regions using structured parsing tools; extract key-value pairs using regular expressions; identify entities in unstructured text using a pre-trained Named Entity Recognition (NER) model; PDF attachment emails: By calling the OCR engine to recognize text in images or PDF content, the text is converted into editable text; for table-type images, the table area is located through a table detection algorithm, and then the row and column data are extracted. Emails with Excel or CSV attachments: Parse attachments using Python's pandas library, directly read table data, and associate it with the corresponding product; Data validation: Filter out erroneous data through numerical reasonableness validation and date continuity validation; This includes verification of numerical validity and verification of date continuity; Numerical reasonableness verification: Check whether the net value is within a reasonable range; Date continuity check: Checks whether the extracted dates are consecutive and excludes duplicate date data.

4. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 1, characterized in that, The automated analysis based on the standardized net asset value data includes: The standardized net value data is preprocessed. Based on the preprocessed net asset value data, calculate the core indicators for investment analysis; Perform sequence trend analysis and pattern recognition on the net value data; Conduct risk warning and assessment for single products; Provide personalized analysis and product recommendations.

5. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 4, characterized in that, The preprocessing of the standardized net value data includes: Duplicate date data is removed through time series verification; For missing dates, net asset value data was completed using linear interpolation. Link user holdings information with product net asset value data.

6. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 4, characterized in that, In the step of calculating the core investment analysis indicators based on the preprocessed net asset value data... The core indicators include return indicators and risk indicators. The return indicators include daily return, cumulative return, and annualized return. The risk indicators include volatility, maximum drawdown, and Sharpe ratio.

7. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 4, characterized in that, In the step of performing sequence trend analysis and pattern recognition on the net value data The method for performing series trend analysis on net asset value data is as follows: For short-term trends, a moving average model is used to draw a net value moving average for a short period. When the short-term moving average crosses above the long-term moving average, it is marked as an "uptrend signal," and vice versa as a "downtrend signal." For periodic analysis, Fourier transform is used to perform spectral analysis on net value data over medium- to long-term time periods to identify seasonal fluctuation patterns. The method for pattern recognition of net asset value data is as follows: The K-means clustering algorithm is used to cluster the net asset value curves of similar products, identify the similarity between the current product and the curves of historically strong / weak products, and help to judge future trends.

8. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 1, characterized in that, The steps for conducting single-product risk warning and assessment include: The system presets Level 1 and Level 2 warning conditions. When a Level 1 warning condition is met, a Level 1 warning is triggered; when a Level 2 warning condition is met, a Level 2 warning is triggered. The isolated forest algorithm is used to detect net asset value outliers and identify potential risk events. Generate a risk assessment report; the risk assessment report includes a single product risk rating and an asset allocation risk matrix.

9. The asset management net asset value email cleaning and parsing method based on a large model as described in claim 4, characterized in that, In the steps of personalized analysis and product recommendation... The methods for performing personalized analysis include, but are not limited to, the following: Performance is calculated based on the user's holding period; or Generate a suitability score based on investment objectives and provide suggestions; The methods for making product recommendations include, but are not limited to, the following: Based on collaborative filtering algorithms, recommend alternative products that have similar investment style characteristics to the user's holdings but have performed better recently; or, Based on users' historical interaction data with products, the similarity between products is calculated, and product recommendations are made using item-based collaborative filtering; or, Based on the product's historical net asset value data, calculate the Pearson correlation coefficient, and then recommend products with low correlation based on the Pearson correlation coefficient.

10. A system for cleaning and parsing asset management net asset value emails based on a large model, characterized in that, The system includes: The email retrieval module is used to retrieve emails from the user's email address, extract and store the metadata of the emails; The net asset value email recognition module is used to perform multiple recognitions on emails based on a large model, confirming net asset value emails and excluding non-net asset value emails. The net asset value (NAV) data extraction module is used to extract NAV data from emails of different formats using appropriate methods and to perform data verification. The net asset value (NAV) data standardization processing module is used to standardize the extracted NAV data according to a preset terminology mapping table and a standardized data dictionary. The net asset value (NAV) data analysis module is used for automated analysis based on standardized NAV data. The analysis results display module is used to visualize the analysis results.

Images