A method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset data.

Abstract

This invention discloses a method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset (NPA) data, belonging to the field of NPA data processing technology. The method includes the following steps: S1, data collection and preprocessing, integrating raw data from multiple channels, completing format unification and preliminary deduplication; S2, layered data cleaning, obtaining clean and consistent data through anomaly detection, missing value repair, redundancy removal, and standardization; S3, multi-dimensional labeling, constructing a labeling system for basic information, risk level, disposal attributes, and compliance attributes, completing accurate labeling; S4, structured summarization, constructing a layered model, classifying and archiving related data, and generating standardized and reusable results. This invention improves data quality and processing efficiency, realizing the transformation of data from messy to standardized, adapting to financial supervision, providing data support for intelligent disposal of NPAs, and reducing labor costs and data error rates.

Description

Technical Field

[0001] This invention relates to the field of non-performing asset data processing technology, specifically to a method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset data. Background Technology

[0002] The content in this section only provides background information related to this invention and may not constitute prior art.

[0003] Non-performing assets refer to assets held by financial institutions, enterprises, etc., that are difficult to recover principal and interest or realize on time and in full. They mainly include substandard, doubtful, and loss-class financial assets. Their disposal process involves multiple stages such as acquisition, due diligence, valuation, pricing, and disposal. Data, as the core foundation for non-performing asset disposal, directly affects disposal efficiency and risk control effectiveness.

[0004] Currently, there are many pain points in the processing of non-performing asset (NPA) data: First, the data sources are scattered and disorganized. The original NPA data comes from multiple channels such as banks, asset management companies, and judicial organs, and the data formats include spreadsheets, scanned documents, and other forms. There is no unified processing standard, and disposal teams need to sort it out from scratch after taking over, which consumes a lot of manpower and time. Second, the data quality is uneven. There are a large number of outliers, missing values, and redundant data in the original data. Moreover, the data formats are not uniform and the terminology is not standardized, making manual verification difficult and easily overlooking key information, which affects the formulation of disposal plans and risk assessment. Third, the labeling system is imperfect. Existing labeling methods mostly focus on a single dimension (such as risk level) and lack multi-dimensional labeling that combines regulatory requirements and disposal needs. The labeling results cannot fully support the application of the entire process of due diligence, valuation, and disposal. Fourth, the aggregation method is not standardized. Data aggregation is mostly simple stacking, lacking structured organization, poor data correlation, and unable to achieve rapid query, traceability, and reuse. It is also difficult to meet the regulatory requirements of financial regulatory authorities for the authenticity, completeness, and compliance of NPA data.

[0005] Existing data processing methods are mostly designed for general-purpose data and lack adaptation to the professionalism, complexity, and regulatory orientation of non-performing asset (NPA) data, thus failing to effectively address the aforementioned pain points. For example, conventional data cleaning methods do not consider the correlation between NPA debt and mortgage information, resulting in logical flaws in the cleaned data; conventional labeling methods do not incorporate NPA regulatory classification standards and disposal requirements, leading to insufficient practicality of the labeling results; and conventional aggregation methods do not construct a structured model adapted to the entire NPA disposal process, failing to support intelligent disposal and regulatory reporting. With the deepening of digital reforms, asset management companies are gradually transforming into multi-party collaborative risk governance platforms, increasing the demands for standardization, accuracy, and structure in NPA data. There is an urgent need for a NPA data processing method that adapts to industry needs and regulatory requirements, transforming data from chaotic to standardized, improving data processing efficiency and quality, and providing support for intelligent NPA disposal. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset data. This method solves the technical problems in existing non-performing asset data processing, such as data disorder, low quality, non-standard labeling, low summarization efficiency, and inability to adapt to regulatory requirements and disposal needs.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: This application provides a method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset data, including the following steps: S1. Data Acquisition and Preprocessing Raw data on non-performing assets (NPAs) is collected from financial institutions, asset management companies, and judicial authorities. This raw data includes various types of data related to NPAs, such as debt documents, mortgage information, debtor data, legal documents, and financial vouchers. Considering the large number of scanned paper documents and image-based documents in the NPA data collection, a combination of API interface integration, OCR recognition, and direct database connection is used for data collection. OCR recognition is used to extract text data from scanned documents, image-based legal documents, and financial vouchers to ensure the comprehensiveness of data collection. API interface integration is used to obtain structured data from financial institutions and asset management companies, while direct database connection is used to obtain relevant case data from judicial authorities.

[0008] The collected raw data undergoes format parsing to standardize data storage format. Specifically, this includes: standardizing date format to "YYYY-MM-DD" to avoid logical errors caused by inconsistent date formats; standardizing amount format to numeric, removing redundant information such as symbols and units; and standardizing text encoding to UTF-8 to ensure text data compatibility. Simultaneously, by calculating the hash value of each data entry and comparing these hash values, initial duplicate data removal is performed, retaining the first occurrence of valid data, reducing the workload of subsequent data processing, and initially improving data quality.

[0009] S2, Layered Data Cleaning The preprocessed data undergoes layered cleaning to ensure its cleanliness, consistency, and accuracy, meeting the core requirements of non-performing asset disposal and supervision. This process includes the following sub-steps: S21. Anomaly Detection: A combination of statistical analysis and machine learning algorithms is used to comprehensively detect outliers in the data. For numerical data such as debt amount and overdue days, the 3σ principle is used to detect outliers, i.e., data deviating from the mean by more than three standard deviations. For textual data such as debtor names and ownership certificate numbers, invalid text and data with incorrect formats are detected by setting text length thresholds and format validation rules. For time-based data such as overdue start dates and disposal periods, time logic validation is used to detect data that is inconsistent with the actual business logic (such as negative overdue days or disposal periods longer than the asset's lifespan).

[0010] S22. Missing Value Repair: A categorized repair strategy is adopted, with differentiated repair for different types of missing fields to ensure the integrity of key data. For key fields such as asset number, debt amount, debtor identity information, and ownership certificate number, data is supplemented by linking data from multiple channels. For example, missing case information is supplemented by linking the debtor's identity information to the judicial database. For non-key fields (such as remarks), the mean, median, or pattern value of the same type of data is used for filling. For missing fields that cannot be supplemented in the above ways, they are marked as "to be supplemented" and a manual verification reminder is triggered to ensure that data missingness is traceable and processable, meeting the requirements of financial regulators for data authenticity and completeness.

[0011] S23. Redundant Data Removal: This is divided into field-level redundancy removal and record-level redundancy removal. Field-level redundancy removal deletes completely duplicate or highly related redundant fields (e.g., if both "debt amount" and "loan amount" exist and are completely identical, delete one of the fields). Record-level redundancy removal uses a fuzzy matching algorithm, setting a similarity threshold (e.g., 95%), to remove duplicate records with identical core information (asset number, debtor, debt amount) but minor differences in details, thus avoiding low processing efficiency and analytical bias caused by data redundancy.

[0012] S24. Data Standardization: This includes coding standardization, terminology standardization, and value range standardization. Coding standardization follows unified coding rules in financial regulation, uniformly coding asset types, risk levels, and disposal methods to ensure data standardization. Terminology standardization unifies professional terms related to non-performing assets; for example, the expressions "debt-for-equity swap assets" and "mortgage assets" are standardized according to regulatory requirements to avoid misunderstandings caused by inconsistent terminology. Value range standardization maps numerical data of different magnitudes (such as debt amount and valuation amount) to a unified value range, facilitating subsequent data comparison and analysis.

[0013] After data cleaning is completed, a cleaning report is generated. The cleaning report includes the amount of original data, the amount and type of abnormal data, the status of missing value repair, the amount of redundant data removed, and the cleaning completion rate. It is used to trace the data cleaning process, meet compliance audit requirements, and ensure that the data cleaning process is supervised and traceable.

[0014] S3, Multi-dimensional annotation Based on the non-performing asset regulatory classification standards (substandard, doubtful, and loss-related) and disposal needs, a multi-dimensional labeling system is constructed to achieve accurate labeling of standardized clean data, providing support for subsequent disposal plan formulation, risk assessment, and valuation pricing. Specifically, this includes the following: S31. Labeling System Construction: The labeling system includes four core dimensions: basic information dimension, risk level dimension, disposal attribute dimension, and compliance attribute dimension. These dimensions are interconnected and complementary, comprehensively covering the core characteristics and application needs of non-performing assets. Basic information dimensions include asset number, asset type (debt, equity, physical assets), debt amount, debtor information (name / company name, identity code / unified social credit code, contact information), ownership information (ownership certificate number, ownership type), overdue start date, and overdue days. These are used to characterize the basic attributes of non-performing assets and serve as the basis for subsequent labeling and summarization. Risk level dimensions: Referring to the financial regulatory risk classification standards, it is divided into substandard, doubtful, and loss categories. The specific risk level is determined by combining the number of overdue days, credit impairment, and debtor's ability to perform. For example, overdue for more than 90 days without significant credit impairment is marked as substandard; overdue for more than 270 days or debtor evading bank debts is marked as doubtful; and overdue for more than 360 days or debtor entering bankruptcy liquidation proceedings is marked as loss. The disposal attribute dimensions include disposal priority (high, medium, low), suitability of disposal methods (debt settlement with assets, equity transfer, debt restructuring, public auction, etc.), valuation range, and recovery expectations (recovery probability, expected recovery amount), which are used to support the formulation and optimization of disposal plans and improve disposal efficiency. Compliance attributes include: completeness of information (whether it is complete), legality of ownership (whether there are any ownership disputes), and compliance of due diligence (whether it complies with the principle of two-person investigation and whether the necessary verification has been completed), which are used to meet the review requirements of financial regulatory authorities and ensure that the non-performing asset disposal process is compliant and legal.

[0015] S32. Annotation Implementation: A combination of rule matching and machine learning is used to complete multi-dimensional annotation, balancing annotation efficiency and accuracy. Rule matching and annotation: Used for annotation of basic information dimension and compliance attribute dimension. It has preset regulatory rules, ownership verification rules and data integrity verification rules, and completes the annotation by matching fields. For example, it matches the ownership legality rule according to the ownership certificate number to determine whether there is an ownership dispute of the asset; and it judges whether the data is complete according to the completeness of the data fields. Machine learning annotation: used for annotation of risk level dimension and disposal attribute dimension. It trains classification models (such as random forest and logistic regression models) based on historical annotation data, inputs feature parameters in standardized clean data (such as overdue days, debt amount, debtor's financial status, and collateral status), and outputs the corresponding annotation results; Manual verification and correction: After the annotation is completed, manual sampling verification is carried out according to a preset ratio (such as 5%) to correct the annotation deviation and ensure that the annotation accuracy rate is not less than 98%, so as to avoid annotation errors from affecting subsequent processing and risk assessment.

[0016] S4, Structured Summary A hierarchical structured aggregation model is constructed to classify, archive, and integrate labeled data, generating standardized and reusable structured data results. This enables data to be queryable, traceable, and reusable, adapting to the needs of the entire non-performing asset disposal process. Specifically, this includes the following sub-steps: S41. Construction of a hierarchical structured summary model: The model includes a base layer, an annotation layer, and an application layer. These three layers are interconnected, forming a complete structured system. Basic layer: Stores the standardized clean data obtained in step S2, classifies and stores it according to asset type, and ensures the traceability of the original data; Labeling layer: It associates and stores the correspondence between labeling information and original data, establishes the association between labeling information and basic data through asset number, and realizes that the labeling information can be queried and modified; Application layer: Based on different needs such as non-performing asset disposal, valuation, and regulatory reporting, different types of structured results are generated, including basic data tables, risk labeling tables, and disposal adaptation tables.

[0017] S42. Categorization, Archiving, and Integration: Data across various dimensions is linked through asset identification numbers. Basic information, risk assessment, disposal attributes, and compliance information are integrated and categorized according to asset type, risk level, and disposal priority to create a structured dataset. For example, basic information, risk level, disposal suitability plan, and compliance verification results for the same asset can be stored together for one-click querying and linked analysis.

[0018] S43. Structured Output: The structured output supports exporting to Excel, CSV, and database table formats, which are compatible with the existing data systems of asset management companies and financial institutions. It can be directly used in scenarios such as due diligence, valuation and pricing, disposal plan formulation, and regulatory reporting, enabling rapid data reuse and improving disposal efficiency.

[0019] S5, Data Update and Verification Establish a dynamic data update mechanism to collect real-time changes in non-performing asset data (such as updates to overdue days, changes in disposal progress, and valuation adjustments), and update the cleaning, labeling, and summarizing results synchronously to ensure data timeliness; regularly (e.g., monthly) verify the completeness, accuracy, and compliance of structured results, triggering anomaly warnings for those that fail verification, and notifying relevant personnel to handle them, ensuring that the data always meets regulatory requirements and disposal needs, and achieving full lifecycle management of the data.

[0020] The beneficial effects of this invention are as follows: 1. This invention adapts to the characteristics of non-performing asset data and regulatory requirements, solving the problem of insufficient targeting of existing data processing methods: Combining the characteristics of non-performing asset data, such as dispersed sources, complex types, and strong regulatory orientation, this invention designs a layered data cleaning process to address industry pain points such as scanned document identification, abnormal data, and missing data. At the same time, it strictly follows the principles of authenticity, cleanliness, and compliance of financial regulation to ensure that the data meets regulatory review requirements and avoids disposal risks caused by non-compliant data.

[0021] 2. This invention constructs a multi-dimensional annotation system to improve data usability: Unlike existing single-dimensional annotation methods, the annotation system constructed in this invention covers four core dimensions: basic information, risk level, disposal attributes, and compliance attributes. It not only meets the requirements of financial supervision for the risk classification of non-performing assets, but also supports the entire process of disposal plan formulation, valuation and pricing, and maximizes the value of data.

[0022] 3. This invention achieves structured data aggregation and improves processing efficiency: Through a hierarchical structured aggregation model, the cleaned and labeled data is associated, integrated, classified, and archived to generate standardized and reusable structured results. This solves the problems of messy and poor correlation in existing data aggregation, enables rapid data query, traceability, and reuse, adapts to the application needs of different scenarios, significantly reduces the cost of manual data processing, and improves the efficiency of non-performing asset disposal.

[0023] 4. This invention significantly improves data processing efficiency and quality: It adopts technologies such as OCR recognition and machine learning to replace traditional manual data processing methods, reducing human intervention and lowering the data error rate. At the same time, through dynamic updates and regular verification mechanisms, it ensures the timeliness, accuracy and completeness of data, providing reliable data support for the intelligent disposal and risk management of non-performing assets, and helping asset management companies to transform from "manual collection" to "intelligent operation". Attached Figure Description

[0024] Figure 1 The present invention provides a flowchart of a method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset data; Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] Example 1 This embodiment provides a method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset data. The specific steps are as follows: S1. Data Acquisition and Preprocessing We collected raw data on non-performing assets acquired by an asset management company, including data on 1,000 debt-related non-performing assets. The data sources included structured data (debt amount, debtor information, overdue days, etc.) from bank API interfaces, case data (filing time, execution status, etc.) directly connected to judicial databases, and scanned paper documents (loan contracts, mortgage certificates). We used OCR recognition technology to extract text data from the scanned documents to ensure comprehensive data collection.

[0027] The collected raw data underwent format parsing to unify the data storage format: date formats were standardized to "YYYY-MM-DD", e.g., "2024.10.05" and "2024 / 10 / 05" were uniformly changed to "2024-10-05"; amount formats were standardized to numeric, removing units such as "ten thousand yuan" and "yuan", e.g., "5 million yuan" was changed to 5,000,000; and text encoding was standardized to UTF-8. Simultaneously, the hash value of each data entry was calculated, and 120 duplicate data entries were removed by comparing hash values, retaining 880 valid preprocessed data entries.

[0028] S2, Layered Data Cleaning S21. Abnormal Data Detection: The 3σ principle was used to detect numerical data such as debt amount and overdue days, and 35 abnormal data entries were detected (such as negative debt amount or overdue days exceeding 1000 days); 18 invalid text data entries were detected by passing the text length threshold (debtor's name length is not less than 2 characters) and format validation (identity code is 18 digits); 7 data entries with contradictory time settings were detected by time logic validation (such as overdue start date being later than the current date), for a total of 60 abnormal data entries detected.

[0029] S22. Missing Value Repair: For the detected missing fields, 12 key fields such as asset number and debtor identity information are missing. These are filled by linking the bank database and judicial database. 38 non-key fields (such as remarks information) are missing. These are filled by using pattern values ​​of the same type of data. 2 missing fields that cannot be filled are marked as "to be filled" and a manual verification reminder is triggered.

[0030] S23. Redundant Data Removal: Field-level redundancy removal deletes 4 duplicate fields such as "Loan Amount" and "Debt Amount"; Record-level redundancy removal uses a fuzzy matching algorithm (similarity threshold of 95%) to remove 25 duplicate records with the same core information.

[0031] S24. Data Standardization: In accordance with the unified coding rules of financial supervision, asset types (debt-based assets are coded as 01) and risk levels (substandard assets 02, doubtful assets 03, and loss assets 04) are uniformly coded; professional terminology is standardized, and "debt-for-equity swap assets" is uniformly expressed as "pledged assets"; the amount of debt is mapped to the value range of [0,1] for easy subsequent analysis.

[0032] After the data cleaning is completed, a cleaning report is generated, showing 1,000 original data entries, 60 abnormal data entries, 50 missing value repair entries, 25 redundant data entries removed, and a cleaning completion rate of 91.5%.

[0033] S3, Multi-dimensional annotation 3.1 Annotation System Construction: The four-dimensional annotation system designed in this invention is adopted, as follows: Basic information dimensions include: asset number, asset type (debt), debt amount, debtor's name, ID number, contact information, ownership certificate number, overdue start date, and overdue days; Risk level dimension: Referring to the financial regulatory risk classification standards, those overdue for 90-270 days and without significant credit impairment are marked as substandard; those overdue for 270-360 days or with debtors evading debts are marked as doubtful; and those overdue for more than 360 days or with debtors entering bankruptcy liquidation are marked as loss. Disposal attribute dimensions: Disposal priority is determined based on the amount of debt and expected recovery (amounts greater than 5 million and recovery probability greater than 60% are high priority), the suitability of disposal method is determined based on the collateral situation (property mortgage is suitable for debt settlement or public auction), valuation range is determined based on market conditions and asset status, and expected recovery is determined based on the debtor's ability to perform its obligations. Compliance attribute dimension: The completeness of the documents is judged based on whether the loan contract, mortgage certificate and due diligence report are complete; the legality of ownership is judged based on the ownership certificate number; and the compliance of due diligence is judged based on whether the principle of two-person investigation is implemented.

[0034] 3.2 Labeling Implementation: A combination of rule matching and machine learning was adopted. The basic information dimension and compliance attribute dimension were labeled using rule matching. For example, the legality of ownership was verified based on the ownership certificate number to determine whether there was any ownership dispute. The risk level dimension and disposal attribute dimension were labeled using a pre-trained random forest model. Input features such as overdue days, debt amount, and mortgage status were used to output the labeling results. After labeling, 44 records (5%) were manually sampled and verified, and 2 labels were corrected for labeling deviations. The final labeling accuracy rate was 99.5%.

[0035] S4, Structured Summary S41. Construct a hierarchical structured aggregation model, including a basic layer, a labeling layer, and an application layer: The basic layer stores 880 standardized clean data entries, categorized by debt assets; the labeling layer associates labeling information with basic data, establishing a link through asset numbers; the application layer generates a basic data table, a risk labeling table, and a disposal adaptation table.

[0036] S42. Classification, archiving and integration: Basic information, risk labeling, disposal attributes and compliance information are linked and integrated through asset number, and archived according to risk level (sub-class, doubtful, loss class), with 420 sub-class entries, 310 doubtful entries and 150 loss entries, forming a structured dataset.

[0037] S43. Structured Output: Export the structured dataset to Excel format, adapt it to the asset management company's existing data system, and use it directly for due diligence, valuation and pricing, and disposal plan formulation, enabling rapid data reuse.

[0038] S5. Data Update and Verification: Establish a dynamic data update mechanism to collect information such as overdue days updates and changes in processing progress in real time, and update the cleaning, labeling, and summary results synchronously; verify the structured results monthly. This month's verification found 3 data anomalies (overdue days not updated), triggering an anomaly warning and notifying relevant personnel to make corrections to ensure data timeliness and accuracy.

[0039] In this embodiment, the non-performing asset data processed by the method of the present invention has significantly improved data quality, with the abnormal data rate decreasing from 6% to 0.5%, the labeling accuracy rate reaching 99.5%, and the structured results being directly adaptable to the entire disposal process. The time spent manually sorting data is reduced by 80%, and the disposal efficiency is greatly improved. At the same time, the data complies with financial regulatory requirements, effectively reducing disposal risks.

[0040] Example 2 This embodiment targets a mix of equity and physical non-performing assets to verify the applicability of the method of the present invention. The specific steps are as follows: S1. Data Acquisition and Preprocessing The project collected raw data on 800 mixed-type non-performing assets divested by a financial institution, including 300 equity-related non-performing assets (containing equity registration documents, shareholder information, and equity valuation reports) and 500 physical non-performing assets (containing real estate and equipment mortgage certificates and scanned copies of valuation reports). Data collection was achieved through API interfaces connecting to the financial institution's equity database, direct database connections to the real estate registration center, and OCR recognition of scanned documents (valuation reports and mortgage certificates). OCR recognition focused on extracting core information such as the location, area, and appraised value of the physical assets.

[0041] Unified data storage format: Date format is unified as "YYYY-MM-DD", equity valuation and physical asset valuation are unified as numerical type, and text encoding is unified as UTF-8; calculate the hash value of each data, remove 95 duplicate data (duplicate equity registration and duplicate physical mortgage information), and retain 705 valid preprocessed data.

[0042] S2, Layered Data Cleaning S21. Anomaly Detection: The 3σ principle was used to detect numerical data such as equity valuation and physical asset valuation, and 28 anomalies were detected (such as valuation of 0, far exceeding the reasonable market range); through text format verification (equity code is 6 digits, real estate registration number is 15 digits), 12 invalid text data were detected; through time logic verification, 5 data were detected that showed contradictions between the equity duration and disposal cycle and expired physical mortgage period, for a total of 45 anomalies.

[0043] S22. Missing value repair: 10 key fields (equity code, real estate registration number, appraised value) are missing, which are supplemented by linking the equity registration center and real estate registration center databases; 32 non-key fields (remarks, asset usage status) are missing, which are filled with values ​​of the same data pattern; 1 field cannot be supplemented, which is marked as "to be supplemented" and triggers manual verification.

[0044] S23. Redundant data removal: At the field level, remove three duplicate fields such as "equity value" and "equity valuation", "physical valuation" and "mortgage value"; at the record level, use a fuzzy matching algorithm (similarity threshold of 92%) to remove 20 duplicate records with the same core information (equity code / real estate registration number, asset type, valuation value).

[0045] S24. Data Standardization: In accordance with the financial regulatory coding rules, equity assets are coded as 02 and physical assets as 03, while the risk level codes remain unchanged; professional terminology is standardized, and the expressions "equity pledge" and "physical mortgage" are standardized; equity valuation and physical asset valuation are mapped to the [0,1] value range to facilitate cross-type asset comparison.

[0046] After cleaning, a cleaning report was generated. The original data consisted of 800 records, 45 records of abnormal data, 42 records of missing values ​​were repaired, and 20 records of redundant data were removed. The cleaning completion rate was 91.9%.

[0047] S3, Multi-dimensional annotation 3.1 The labeling system has been adjusted to adapt to mixed assets: the basic information dimension adds equity ratio and physical asset location / area; the risk level dimension is adjusted in combination with equity liquidity and physical asset realization difficulty (e.g., equity that cannot be traded or physical assets that are seized are labeled as loss); the disposal attribute dimension adds equity disposal methods (equity transfer, liquidation) and physical asset disposal methods (auction, sale); the compliance attribute dimension adds equity ownership verification and physical asset mortgage validity verification.

[0048] 3.2 Labeling Implementation: Rule matching labels basic information and compliance attributes; machine learning (logistic regression model) labels risk level and disposal attributes, inputting characteristic parameters such as equity liquidity and difficulty of physical realization; manual sampling verification of 35 cases (5%), correction of 1 deviation, labeling accuracy rate of 99.7%.

[0049] S4, Structured Summary A hierarchical model is constructed, with the base layer storing data categorized by equity and physical assets; the annotation layer linking annotation information with the base data; and the application layer generating equity data tables, physical asset data tables, and a mixed asset summary table. Data across various dimensions is linked by asset number, archived by asset type and risk level, and the structured output can be exported to CSV format for compatibility with financial institution asset management systems.

[0050] S5. Data Update and Verification: Real-time updates on equity pledge status and physical asset seizure status, monthly verification. This month, two physical asset pledges were found to have expired, triggering an alert and correction to ensure data compliance.

[0051] In this embodiment, the abnormal data rate of mixed assets decreased from 5.6% to 0.3%, the labeling accuracy rate was 99.7%, the manual processing time was reduced by 82%, and the structured results can be directly used for cross-type asset valuation and disposal, adapting to the needs of mixed non-performing asset processing.

[0052] Example 3 This embodiment verifies the efficiency of the method of the present invention for a large amount of non-performing asset data. The specific steps are as follows: S1. Data Acquisition and Preprocessing This project collected data on 5,000 non-performing debt assets acquired in bulk by an asset management company. The data sources included API connections with multiple banks, batch data exported from judicial authorities, and 1,200 scanned loan contracts. Multi-threaded API integration, batch OCR recognition (supporting multiple scanned document formats), and batch database import were employed to improve data collection efficiency.

[0053] After unifying the data format, the data was deduplicated in batches using hash values, removing 620 duplicate records and retaining 4,380 valid preprocessed records. The preprocessing took 4 hours (compared to 12 hours using traditional methods).

[0054] S2, Layered Data Cleaning Using a batch processing mode, abnormal data detection was performed in parallel with machine learning algorithms based on the 3σ principle, detecting 210 abnormal data entries; missing value repair was achieved by batch associating and supplementing through multiple channels, repairing 45 missing key fields and 150 missing non-key fields; redundant data was batch removed, with 5 duplicate fields deleted at the field level and 85 duplicate records removed at the record level; data was standardized through batch coding and unified terminology.

[0055] After cleaning was completed, the cleaning report showed a completion rate of 93.2% and a time of 6 hours (traditional methods require 24 hours).

[0056] S3, Multi-dimensional annotation Basic information and compliance attributes were labeled in batches using rule matching, and risk level and disposal attribute labels were labeled in batches using machine learning models. 219 samples were manually sampled and verified (5%), and 3 deviations were corrected. The labeling accuracy rate was 99.4%, and the labeling time was 8 hours (traditional manual labeling takes 72 hours).

[0057] S4, Structured Summary The hierarchical model integrates data in batches, categorizes and archives it according to risk level and disposal priority, generates batch structured data tables, supports batch database import, adapts to the large-scale asset management needs of asset management companies, and takes 3 hours to summarize.

[0058] S5. Data Update and Verification: Establish a batch data update mechanism to synchronize data changes from multiple channels in real time, perform monthly batch verification, and ensure that the response time for abnormal data alerts is ≤1 hour, thereby ensuring the timeliness and compliance of large-scale data.

[0059] In this embodiment, the total processing time for large-scale data is 21 hours, which is 84% ​​shorter than the traditional method (132 hours). The abnormal data rate is reduced from 4.2% to 0.4%, and the annotation accuracy is 99.4%, fully demonstrating the efficiency and scalability of the method of the present invention.

[0060] Comparative Example 1 Using traditional manual processing methods, 1,000 non-performing debt asset data, identical to those in Example 1, were processed. The specific process included: manually organizing scanned documents, manually entering data, manually checking for duplicates, detecting abnormal data, repairing missing values, manually labeling data by single risk level, and manually organizing and summarizing the data.

[0061] Processing results: Total time spent: 160 hours; 28 anomaly data were missed, with an anomaly rate of 2.8%; missing value repair was incomplete, with 15 key fields not being supplemented; annotation only covered risk levels, with an annotation accuracy rate of 88%; the summarized data was an unstructured document, which could not be directly reused and required secondary processing; there was no data update and verification mechanism, resulting in poor data timeliness and inability to guarantee compliance.

[0062] Comparative Example 2 Using existing general data cleaning, labeling, and summarizing methods, 1,000 non-performing debt asset data, the same as in Example 1, were processed. The specific process was as follows: using general data cleaning tools to remove duplicates and detect anomalies, simply filling in missing values, labeling in a single dimension, and simply classifying and summarizing.

[0063] Processing results: Total processing time: 48 hours; Abnormal data rate: 3.5% (data correlation of non-performing assets was not considered, and logically abnormal data was omitted); Missing value repair accuracy rate: 75%; Only risk level was marked, which cannot support the entire disposal process; The aggregated data has poor correlation, low structure, and poor adaptability, and cannot be directly connected to the financial regulatory system and disposal system; There is no dynamic update and verification mechanism, and the data compliance does not meet the standards.

[0064] Comparative Example 3 A simplified version of the method of this invention is adopted, which removes the multi-dimensional annotation steps and only performs data cleaning and simple summarization to process 1,000 non-performing debt asset data, the same as in Example 1.

[0065] Processing results: Total time taken: 32 hours, abnormal data rate: 0.8% (cleaning effect is close to that of this invention); however, there is no labeling information, making it impossible to conduct risk assessment or formulate disposal plans, and the data has extremely low practicality; the aggregated data is merely a collection of basic information, which cannot achieve rapid querying and reuse, and cannot meet the needs of supervision and disposal.

[0066] Performance Comparison Table of Examples and Comparative Examples: .

[0067] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0068] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for cleaning, multi-dimensional labeling, and structured summarization of non-performing asset data, characterized in that, Includes the following steps: S1. Data Collection and Preprocessing: Collect raw data on non-performing assets from financial institutions, asset management companies, and judicial authorities. The raw data includes debt information, mortgage information, debtor data, legal documents, and financial vouchers. Parse the raw data to unify the data storage format, and perform preliminary deduplication through hash verification to obtain preprocessed data. S2. Layered data cleaning: The preprocessed data is sequentially subjected to anomaly detection, missing value repair, redundant data removal, and data standardization to obtain standardized clean data. S3. Multi-dimensional labeling: Based on the non-performing asset regulatory classification standards and disposal needs, a multi-dimensional labeling system is constructed. The labeling system includes basic information dimension, risk level dimension, disposal attribute dimension, and compliance attribute dimension. A combination of rule matching and machine learning is used to complete the multi-dimensional labeling of standardized clean data and generate labeled data. S4. Structured Summary: Construct a hierarchical structured summary model to classify, archive, and integrate the labeled data to generate standardized structured results, including a basic data table, a risk labeling table, and a treatment adaptation table.

2. The method according to claim 1, characterized in that, In step S1, the data acquisition adopts a combination of API interface connection, OCR recognition, and direct database connection. OCR recognition is used to extract text data from scanned documents, image-based legal documents, and financial vouchers. The format unification includes date format being unified as "YYYY-MM-DD", amount format being unified as numeric, and text encoding being unified as UTF-8. The preliminary deduplication is performed by calculating the hash value of each data item, comparing the hash values ​​to complete the preliminary removal of duplicate data, and retaining the first occurrence of valid data.

3. The method according to claim 1, characterized in that, In step S2, the abnormal data detection adopts a combination of statistical analysis and machine learning algorithms, specifically including: detecting outliers in numerical data through the 3σ principle, detecting outliers in text data through text length threshold and format verification, and detecting outliers in overdue days and processing cycles through time logic verification; the missing value repair adopts a classification repair strategy, when key fields are missing, they are supplemented by associating data from multiple channels, and when non-key fields are missing, they are filled with the mean, median or pattern value of the same type of data, and those that cannot be supplemented are marked as "to be supplemented" and trigger manual verification reminders.

4. The method according to claim 1, characterized in that, In step S2, the redundant data removal includes field-level redundancy removal and record-level redundancy removal. Field-level redundancy removal deletes completely duplicate or highly related redundant fields. Record-level redundancy removal uses a fuzzy matching algorithm to remove duplicate records with consistent core information but minor differences in details. The data standardization includes coding standardization, terminology standardization, and value range standardization. Among them, coding standardization encodes asset types and risk levels according to the unified coding rules of financial supervision.

5. The method according to claim 1, characterized in that, In step S3, the basic information dimension includes asset number, asset type, debt amount, debtor information, ownership information, and overdue start date, used to characterize the basic attributes of non-performing assets; the risk level dimension refers to the financial regulatory risk classification standards and is divided into substandard, doubtful, and loss categories, and the specific risk level is determined by combining the number of overdue days, credit impairment, and debtor's repayment ability; the disposal attribute dimension includes disposal priority, suitability of disposal method, valuation range, and recovery expectation, used to support the formulation of disposal plan; the compliance attribute dimension includes data completeness, ownership legality, and due diligence compliance, used to meet regulatory review requirements.

6. The method according to claim 1, characterized in that, In step S3, the rule matching is used for labeling the basic information dimension and compliance attribute dimension. Preset regulatory rules, ownership verification rules, and data integrity verification rules are used to complete the labeling through field matching. The machine learning is used for labeling the risk level dimension and disposal attribute dimension. The classification model is trained based on historical labeling data, the feature parameters in the standardized clean data are input, and the corresponding labeling results are output. After the labeling is completed, the labeling deviation is corrected by manual sampling verification.

7. The method according to claim 1, characterized in that, In step S4, the hierarchical structured aggregation model includes a base layer, a labeling layer, and an application layer. The base layer stores standardized clean data, the labeling layer associates and stores the correspondence between labeling information and original data, and the application layer generates different types of structured results according to the requirements of non-performing asset disposal, valuation, and regulatory reporting. The association integration establishes the association relationship of data in each dimension through asset numbering.

8. The method according to claim 1, characterized in that, It also includes step S5, data update and verification: establish a dynamic data update mechanism to collect change information of non-performing asset data in real time, and update the cleaning, labeling and summary results synchronously; regularly verify the completeness, accuracy and compliance of structured results, trigger an anomaly warning if the verification fails, notify relevant personnel to handle it, and ensure that the data always meets regulatory requirements and disposal needs.

9. The method according to claim 1, characterized in that, In step S2, after the data cleaning is completed, a cleaning report is generated. The cleaning report includes the amount of original data, the amount and type of abnormal data, the status of missing value repair, the amount of redundant data removed, and the cleaning completion rate, which is used to trace the data cleaning process.

10. The method according to claim 1, characterized in that, In step S4, the structured results can be exported as Excel, CSV, or database table formats.

Images