National library big data clustering analysis processing method and system

Abstract

The invention relates to the technical field of financial data analysis, and discloses a national library big data clustering analysis processing method and system. The method comprises the following steps: establishing a data standard of data scenario application for collected financial data, performing source data verification on the collected financial data, performing statistics and de-duplication operation on the financial data passing the source data verification at the same time, completing the processing of the financial data, inputting the processed financial data into a clustering analysis layer at the same time, and performing clustering analysis on the financial data. Data clustering is carried out on input processed financial data through a clustering analysis layer, and centralized and real-time processing of the financial data is realized in a mode that the data subjected to data clustering is verified in batches through distributed calculation by a task scheduling control mechanism; and finally, carrying out secondary matching and account checking on the financial data after verification processing, and converting the financial data into corresponding data files to be output so as to ensure the accuracy and effectiveness of the financial data.

Description

Technical Field

[0001] The present invention relates to the technical field of financial data analysis, and in particular to a method and system for clustering analysis and processing of treasury big data. Background Art

[0002] Due to the huge scale, wide variety and rapid growth of historical treasury data, its processing and use need to be deeply integrated with internal supervision and inspection, accounting file management, income, refund, expenditure, correction, cash management, accounting voucher management and transfer, non-tax revenue and centralized payment supervision, statistical analysis and other business requirements, making it extremely difficult to use treasury data in scenarios such as online review, value extraction, analysis and mining, cross-validation, and trend forecasting.

[0003] The prior art CN115203317A uses a blockchain storage unit to store financial data on the chain in real time, ensuring the security of data storage while also being able to conveniently call data; it then uses the collected data to conduct targeted analysis of the financial data using a node data analysis unit, but fails to solve the difficulties in convenient scenario-based use such as online review, value extraction, analysis and mining, cross-validation, and trend prediction, and has great limitations. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the shortcomings of the existing technology, the present invention provides a method for clustering analysis and processing of treasury big data, which has the advantages of multi-source, security, and accuracy. It solves the problem that it is extremely difficult to use treasury data in scenarios such as online consultation, value extraction, analysis and mining, cross-validation, and trend prediction.

[0006] (II) Technical solution

[0007] In order to solve the technical problems that it is extremely difficult to conveniently use the above-mentioned treasury data in scenarios such as online consultation, value extraction, analysis and mining, cross-validation, and trend prediction, the present invention provides the following technical solutions:

[0008] The present invention discloses a method for clustering analysis and processing of national treasury big data, which specifically comprises the following steps:

[0009] S1. Collect financial data to establish data standards for scenario-based applications, and perform source data verification on the collected financial data;

[0010] S2. Perform statistics and de-duplicate operations on the financial data after the source data verification to obtain processed data;

[0011] S3, inputting the processed financial data into the clustering analysis layer, and performing data clustering on the input processed financial data through the clustering analysis layer to obtain clustered financial data, where the clustered financial data includes data features, financial data types, and financial data contents;

[0012] S4, verifying the data after data clustering in batches through distributed computing by means of a task scheduling control mechanism;

[0013] S5. Perform secondary matching and reconciliation on the verified financial data and convert it into corresponding data files for output.

[0014] The present invention completes the processing of financial data by establishing data standards for scenario-based applications for collected financial data, and performs source data verification on the collected financial data. At the same time, it performs statistics and deduplication operations on the financial data after the source data verification, and inputs the processed financial data into a clustering analysis layer. The clustering analysis layer performs data clustering on the input processed financial data, and the data after data clustering is verified in batches through distributed computing by a task scheduling control mechanism to achieve centralized and real-time processing of financial data. Finally, the verified financial data is matched and reconciled for the second time, and converted into corresponding data files for output, so as to ensure the accuracy and effectiveness of the financial data.

[0015] Preferably, the collecting of financial data to establish data standards for scenario-based applications and performing source data verification on the collected data includes the following steps:

[0016] S11. Collect financial data and establish data standards for scenario-based applications based on financial data types;

[0017] The financial data from various data sources in the treasury can be divided into two types of data, namely financial business data and financial voucher data;

[0018] Since financial business data and financial voucher data have the characteristics of time series data, the structure definitions of financial business data and financial voucher data include three parts: timestamp, dimension, and indicator;

[0019] The timestamp T is defined in a standard time format, accurate to milliseconds, the dimension D is the specific location information of the collection point and the location information of the data source point, and the indicator M represents the financial business data or financial voucher data collected by the collection point;

[0020] When M is financial business data, it is specifically one of the following: accounting book data, financial report and payment transaction data, reconciliation data, and account opening and closing registration data, that is, each M includes one financial business data;

[0021] When M is financial voucher data, it is specifically one of accounting file management, income, return, expenditure, correction, cash management, accounting voucher management and transfer, non-tax revenue and centralized payment supervision, and statistical analysis, that is, each M includes one financial voucher data;

[0022] After the financial business data and financial voucher data structure are defined, the collected financial data is saved in the form of A = (T, D, M);

[0023] Among them, A represents the account, T represents the timestamp, D represents the dimension, and M represents the indicator;

[0024] S12. Perform source data verification on the collected data based on the financial business data and financial voucher data structure definition;

[0025] By comparing the financial data sent by the data source point and the received data at the collection point one by one according to the saved timestamps, when the timestamps of the financial data sent by the data source point and the received data at the collection point are consistent, it is determined that the source data verification has passed;

[0026] When the timestamps of the financial data sent by the data source point and the received data at the collection point are inconsistent, it is determined that the source data verification fails, indicating that there are errors in the received data at the collection point, and the erroneous data in the received data at the collection point is deleted.

[0027] The present invention completes the standardization and process-based management of financial data by establishing and defining data standards for scenario-based applications of financial data, thereby facilitating the scenario-based comprehensive utilization of financial data.

[0028] Preferably, the statistical and duplicate removal operations on the financial data after the source data verification include the following steps:

[0029] Convert the collected and saved financial data into binary and set the length threshold of the binary data;

[0030] Divide the collected and saved financial data into two parts and ensure that the two parts have the same length;

[0031] The first half of the binary data after the division is set to save the timestamp data and dimension data of the collected financial data;

[0032] Set the second half of the binary data after division to save the indicator data of the collected financial data;

[0033] Use hash algorithm to traverse the timestamp data and dimension data in financial data;

[0034] Build a long array and set a hash function at the same time, and save the hash value calculated by the hash function each time in the constructed array;

[0035] When the hash values ​​of the first half of the two sets of data calculated by the hash algorithm are consistent, a second hash algorithm is used to calculate the second half of the two sets of data;

[0036] When the hash values ​​of the first half of the two sets of data calculated by the hash algorithm are consistent, the two sets of data are determined to be duplicate data, and the set of data traversed later is deleted;

[0037] After the traversal is completed, the financial data converted into binary will be restored and arranged according to the timestamps in the restored financial data;

[0038] The arranged financial data is set as the processed financial data.

[0039] The present invention uses a hash algorithm to establish an array to store hash values ​​calculated by a hash function in the array, and ensures that the hash values ​​in the array are unique to perform statistics and deduplication operations on financial data. At the same time, the accuracy and reliability of the deduplication operation of the financial data are guaranteed by two hash calculations.

[0040] Preferably, the step of inputting the processed financial data into the cluster analysis layer and performing data clustering on the input processed financial data through the cluster analysis layer comprises the following steps:

[0041] S31. Feature extraction of financial data based on deep learning network;

[0042] S32, clustering the financial data after feature extraction, and outputting a clustering result, where the clustering result includes: features of the financial data after clustering and the clusters to which the features of the financial data after clustering belong;

[0043] S33. Cluster the corresponding financial data based on the characteristics of the clustered financial data to obtain clustered financial data.

[0044] Preferably, the feature extraction of financial data based on the deep learning network includes the following steps:

[0045] The deep learning network consists of three structural layers, namely the convolutional layer, the pooling structure layer and the data fully connected layer;

[0046] S311, selecting part of the processed financial data from all the processed financial data to form a processed financial data training set;

[0047] S312, inputting the processed financial data in the processed financial data training set into the deep learning network until the deep learning network converges to obtain a trained deep learning network;

[0048] The back propagation training loss function is used in the deep learning network. The function expression is as follows:

[0049]

[0050] Where B represents the back propagation training loss function of financial data, θ represents the convolution layer in the deep learning network, n represents the weight of the convolution kernel, y(i) represents the number of times the financial data is trained during the propagation process, i represents the i-th training, and h(x) represents the back propagation of financial data at node x;

[0051] When the value of the back-propagation training loss function drops below the set threshold, the deep learning network converges and a trained deep learning network is obtained;

[0052] S313. Input all processed financial data into the trained deep learning network, and output the extracted features of the financial data.

[0053] Preferably, clustering the financial data after feature extraction comprises the following steps:

[0054] Clustering the extracted features of financial data by calculating the similarity between the extracted features of financial data;

[0055] The similarity calculation formula is as follows:

[0056]

[0057] Wherein, sim represents the similarity value, c represents the cth financial data feature, e represents the eth financial data feature, and g represents the angle between the cth financial data feature and the eth financial data feature;

[0058] When the similarity value is in [0,1], it means that the two data features are similar, and the two data features are placed in the same cluster;

[0059] When the similarity value is in [-1,0), it means that the two data features are not similar;

[0060] Traverse all the extracted features of financial data, divide all the extracted features of financial data, and output the clustering results.

[0061] The present invention realizes feature extraction and clustering of financial data by using a deep learning network, and ensures the accuracy and real-time performance of feature extraction and clustering of financial data through the continuously convolved and updated deep learning network.

[0062] Preferably, the step of verifying the clustered data in batches by distributed computing through a task scheduling control mechanism comprises the following steps:

[0063] Setting processing priorities based on the type of financial data in the clustered financial data;

[0064] Set the processing priority of financial voucher data higher than that of financial business data;

[0065] The batch verification process using distributed computing includes the following steps:

[0066] S41, predetermine a polynomial that uses both the data source point and the collection point as divisors;

[0067] S42, after the polynomial is determined, binary division operation is performed on the original financial data to be sent and the determined polynomial to calculate a check code;

[0068] S43, attaching the calculated check code to the end of the original financial data to be sent, and when receiving the original financial data, the collection point performs calculations based on the determined polynomial and the received original financial data, and compares the calculation result with the check code at the end of the received original financial data;

[0069] When the calculation result is consistent with the check code at the end of the received original financial data, the check is passed, indicating that the data transmission is normal;

[0070] When the calculation result is inconsistent with the check code at the end of the received original financial data, the check fails, indicating that the data transmission is abnormal.

[0071] The present invention uses a batch verification processing method, sets a polynomial with both the data source point and the collection point used as a divisor, and verifies the financial data by calculating the verification code of the sent financial data through the set polynomial, thereby ensuring the accuracy of the financial data.

[0072] Preferably, the secondary matching and reconciliation of the verified financial data comprises the following steps:

[0073] Perform partition sum verification on the verified financial data, and compare the partition summed financial data with the saved statistical data;

[0074] When the summed financial data is inconsistent with the saved statistical data, traverse the financial data to locate the error information;

[0075] When the summed financial data is consistent with the saved statistical data, the secondary matching reconciliation is passed, indicating that there is no erroneous data in the collected financial data.

[0076] The present invention also discloses a national treasury big data cluster analysis processing system, comprising: an application support module, a cluster analysis module, a data processing module and a basic support module;

[0077] The basic support module is used to store middleware and financial data resources;

[0078] The cluster analysis module is used to build stable, efficient and reliable data computing and model services and perform cluster analysis on the collected data through application service clusters, task scheduling frameworks, cache cluster calls and load balancing access;

[0079] The data processing module is used to perform statistics and deduplication operations on financial data;

[0080] The application support module is used to provide an access interface for users to interact and implement data access for different scenarios.

[0081] (III) Beneficial effects

[0082] Compared with the prior art, the present invention provides a method for clustering analysis and processing of treasury big data, which has the following beneficial effects:

[0083] 1. The invention completes the processing of financial data by establishing data standards for scenario-based applications of collected financial data, and performing source data verification on the collected financial data. At the same time, the financial data after the source data verification is statistically analyzed and deduplicated. The processed financial data is input into a clustering analysis layer, and the input processed financial data is clustered through the clustering analysis layer. The data after data clustering is verified in batches through distributed computing by a task scheduling control mechanism, so as to realize centralized and real-time processing of financial data. Finally, the verified financial data is matched and reconciled for the second time, and converted into corresponding data files for output, so as to ensure the accuracy and validity of the financial data.

[0084] 2. This invention completes the standardization and process-based management of financial data by establishing and defining data standards for scenario-based applications of financial data, thereby facilitating the scenario-based comprehensive utilization of financial data.

[0085] 3. By using the hash algorithm, an array is established to store the hash values ​​calculated by the hash function in the array, and the unique hash values ​​in the array are guaranteed to perform statistics and deduplication operations on the financial data. At the same time, the accuracy and reliability of the deduplication operation of the financial data are guaranteed by two hash calculations.

[0086] 4. This invention realizes feature extraction and clustering of financial data by using a deep learning network, and ensures the accuracy and real-time performance of feature extraction and clustering of financial data through continuous convolution and updating of the deep learning network.

[0087] 5. The invention uses a batch verification processing method, sets a polynomial with both the data source point and the collection point used as a divisor, and verifies the financial data by calculating the verification code of the sent financial data through the set polynomial, thereby ensuring the accuracy of the financial data. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 It is a schematic diagram of the flow structure of the treasury big data clustering analysis and processing method of the present invention. DETAILED DESCRIPTION

[0089] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0090] The present invention discloses a method for clustering analysis and processing of national treasury big data, which specifically comprises the following steps:

[0091] S1. Collect financial data to establish data standards for scenario-based applications, and perform source data verification on the collected financial data;

[0092] The process of collecting financial data, establishing data standards for scenario-based applications, and verifying the source data of the collected data includes the following steps:

[0093] S11. Collect financial data and establish data standards for scenario-based applications based on financial data types;

[0094] The financial data from various data sources in the treasury can be divided into two types of data, namely financial business data and financial voucher data;

[0095] Furthermore, since the financial business data and the financial voucher data have the characteristics of time series data, the structure definitions of the financial business data and the financial voucher data both include three parts: timestamp, dimension, and index;

[0096] The timestamp T is defined in a standard time format, accurate to milliseconds, the dimension D is the specific location information of the collection point and the location information of the data source point, and the indicator M represents the financial business data or financial voucher data collected by the collection point;

[0097] Further, after the definition of the financial business data and financial voucher data structure is completed, the collected financial data is saved in the form of A = (T, D, M);

[0098] Among them, A represents the account, T represents the timestamp, D represents the dimension, and M represents the indicator;

[0099] When M is financial business data, it is specifically one of the following: accounting book data, financial report and payment transaction data, reconciliation data, and account opening and closing registration data; that is, each M includes one financial voucher data;

[0100] When M is financial voucher data, it specifically refers to one of the following: accounting file management, income, return, expenditure, correction, cash management, accounting voucher management and transfer, non-tax revenue and centralized payment supervision, and statistical analysis; that is, each M includes one financial voucher data.

[0101] S12. Perform source data verification on the collected data based on the financial business data and financial voucher data structure definition;

[0102] By comparing the financial data sent by the data source point and the received data at the collection point one by one according to the saved timestamps, when the timestamps of the financial data sent by the data source point and the received data at the collection point are consistent, it is determined that the source data verification has passed;

[0103] When the timestamps of the financial data sent by the data source point and the received data at the collection point are inconsistent, it is determined that the source data verification fails, indicating that there is an error in the received data at the collection point, and the erroneous data in the received data at the collection point is deleted;

[0104] S2. Perform statistics and de-duplicate operations on the financial data after the source data verification to obtain processed data;

[0105] The following steps are included to perform statistics and deduplication on the financial data after the source data has passed verification:

[0106] Convert the collected and saved financial data into binary and set the length threshold of the binary data;

[0107] Divide the collected and saved financial data into two parts and ensure that the two parts have the same length;

[0108] The first half of the binary data after the division is set to save the timestamp data and dimension data of the collected financial data;

[0109] Set the second half of the binary data after division to save the indicator data of the collected financial data;

[0110] Furthermore, the timestamp data and dimension data in the financial data are traversed through the hash algorithm;

[0111] Build a long array and set a hash function at the same time, and save the hash value calculated by the hash function each time in the constructed array;

[0112] When the hash values ​​of the first half of the two sets of data calculated by the hash algorithm are consistent, a second hash algorithm is used to calculate the second half of the two sets of data;

[0113] When the hash values ​​of the first half of the two sets of data calculated by the hash algorithm are consistent, the two sets of data are determined to be duplicate data, and the set of data traversed later is deleted;

[0114] After the traversal is completed, the financial data converted into binary will be restored and arranged according to the timestamps in the restored financial data;

[0115] The arranged financial data is set as the processed financial data;

[0116] S3, inputting the processed financial data into the clustering analysis layer, and performing data clustering on the input processed financial data through the clustering analysis layer to obtain clustered financial data, where the clustered financial data includes data features, financial data types, and financial data contents;

[0117] Inputting the processed financial data into the clustering analysis layer, and performing data clustering on the input processed financial data through the clustering analysis layer includes the following steps:

[0118] S31. Feature extraction of financial data based on deep learning network;

[0119] The deep learning network consists of three structural layers, namely the convolutional layer, the pooling structure layer and the data fully connected layer;

[0120] S311, selecting part of the processed financial data from all the processed financial data to form a processed financial data training set;

[0121] S312, inputting the processed financial data in the processed financial data training set into the deep learning network until the deep learning network converges to obtain a trained deep learning network;

[0122] The back propagation training loss function is used in the deep learning network. The function expression is as follows:

[0123]

[0124] Where B represents the back propagation training loss function of financial data, θ represents the convolution layer in the deep learning network, n represents the weight of the convolution kernel, y(i) represents the number of times the financial data is trained during the propagation process, i represents the i-th training, and h(x) represents the back propagation of financial data at node x;

[0125] When the value of the back-propagation training loss function drops below the set threshold, the deep learning network converges and a trained deep learning network is obtained;

[0126] S313, input all processed financial data into the trained deep learning network, and output the extracted features of the financial data;

[0127] S32, clustering the financial data after feature extraction, and outputting a clustering result, where the clustering result includes: features of the financial data after clustering and the clusters to which the features of the financial data after clustering belong;

[0128] Clustering the extracted features of financial data by calculating the similarity between the extracted features of financial data;

[0129] The similarity calculation formula is as follows:

[0130]

[0131] Wherein, sim represents the similarity value, c represents the cth financial data feature, e represents the eth financial data feature, and g represents the angle between the cth financial data feature and the eth financial data feature;

[0132] When the similarity value is in [0,1], it means that the two data features are similar, and the two data features are placed in the same cluster;

[0133] When the similarity value is in [-1,0), it means that the two data features are not similar;

[0134] Traverse all the extracted features of financial data, divide all the extracted features of financial data, and output the clustering results

[0135] S33, clustering the corresponding financial data based on the characteristics of the clustered financial data to obtain clustered financial data;

[0136] S4, verifying the data after data clustering in batches through distributed computing by means of a task scheduling control mechanism;

[0137] The data clustering is processed in batches through distributed computing by using a task scheduling control mechanism, including the following steps:

[0138] Setting processing priorities based on the type of financial data in the clustered financial data;

[0139] Set the processing priority of financial voucher data higher than that of financial business data;

[0140] Furthermore, the batch verification process using a distributed computing method includes the following steps:

[0141] S41, predetermine a polynomial that uses both the data source point and the collection point as divisors;

[0142] S42, after the polynomial is determined, binary division operation is performed on the original financial data to be sent and the determined polynomial to calculate a check code;

[0143] S43, attaching the calculated check code to the end of the original financial data to be sent, and when receiving the original financial data, the collection point performs calculations based on the determined polynomial and the received original financial data, and compares the calculation result with the check code at the end of the received original financial data;

[0144] When the calculation result is consistent with the check code at the end of the received original financial data, the check is passed, indicating that the data transmission is normal;

[0145] When the calculation result is inconsistent with the check code at the end of the received original financial data, the check fails, indicating that the data transmission is abnormal;

[0146] S5. Perform secondary matching and reconciliation on the verified financial data and convert it into corresponding data files for output;

[0147] The second matching and reconciliation of the verified financial data includes the following steps:

[0148] Perform partition sum verification on the verified financial data, and compare the partition summed financial data with the saved statistical data;

[0149] When the summed financial data is inconsistent with the saved statistical data, traverse the financial data to locate the error information;

[0150] When the summed financial data is consistent with the saved statistical data, the secondary matching reconciliation is passed, indicating that there is no erroneous data in the collected financial data;

[0151] The present invention also discloses a national treasury big data cluster analysis processing system, comprising: an application support module, a cluster analysis module, a data processing module and a basic support module;

[0152] The basic support module is used to store middleware and financial data resources;

[0153] The cluster analysis module is used to build stable, efficient and reliable data computing and model services and perform cluster analysis on the collected data through application service clusters, task scheduling frameworks, cache cluster calls and load balancing access;

[0154] The data processing module is used to perform statistics and deduplication operations on financial data;

[0155] The application support module is used to provide an access interface to interact with users and implement data access for different scenarios.

[0156] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for clustering analysis and processing of treasury big data, characterized in that: The following steps are involved: S1. Collect financial data to establish data standards for scenario-based applications, and perform source data verification on the collected financial data; S2. Perform statistics and de-duplicate operations on the financial data after the source data verification to obtain processed data; S3, inputting the processed financial data into the clustering analysis layer, and performing data clustering on the input processed financial data through the clustering analysis layer to obtain clustered financial data, where the clustered financial data includes data features, financial data types, and financial data contents; S4, verifying the data after data clustering in batches through distributed computing by means of a task scheduling control mechanism; S5. Perform secondary matching and reconciliation on the verified financial data and convert it into corresponding data files for output.

2. A method for clustering analysis and processing of national treasury big data according to claim 1, characterized in that: The process of collecting financial data, establishing data standards for scenario-based applications, and verifying the source data of the collected data includes the following steps: S11. Collect financial data and establish data standards for scenario-based applications based on financial data types; The financial data from various data sources in the treasury are divided into two categories: financial business data and financial voucher data; The structure definitions of financial business data and financial voucher data include three parts: timestamp, dimension, and indicator; The timestamp T is defined in a standard time format, accurate to milliseconds, the dimension D is the specific location information of the collection point and the location information of the data source point, and the indicator M represents the financial business data or financial voucher data collected by the collection point; After the financial business data and financial voucher data structure are defined, the collected financial data is saved in the form of A = (T, D, M); Among them, A represents the account, T represents the timestamp, D represents the dimension, and M represents the indicator; S12. Perform source data verification on the collected data based on the financial business data and financial voucher data structure definition; By comparing the financial data sent by the data source point and the received data at the collection point one by one according to the saved timestamps, when the timestamps of the financial data sent by the data source point and the received data at the collection point are consistent, it is determined that the source data verification has passed; When the timestamps of the financial data sent by the data source point and the received data at the collection point are inconsistent, it is determined that the source data verification fails, indicating that there are errors in the received data at the collection point, and the erroneous data in the received data at the collection point is deleted.

3. The method for clustering analysis and processing of national treasury big data according to claim 1, characterized in that: The statistical and duplicate removal operations on the financial data after the source data verification include the following steps: Convert the collected and saved financial data into binary and set the length threshold of the binary data; Divide the collected and saved financial data into two parts and ensure that the two parts have the same length; The first half of the binary data after the division is set to save the timestamp data and dimension data of the collected financial data; Set the second half of the binary data after division to save the indicator data of the collected financial data; Use hash algorithm to traverse the timestamp data and dimension data in financial data; Build a long array and set a hash function at the same time, and save the hash value calculated by the hash function each time in the constructed array; When the hash values ​​of the first half of the two sets of data calculated by the hash algorithm are consistent, a second hash algorithm is used to calculate the second half of the two sets of data; When the hash values ​​of the first half of the two sets of data calculated by the hash algorithm are consistent, the two sets of data are determined to be duplicate data, and the set of data traversed later is deleted; After the traversal is completed, the financial data converted into binary will be restored and arranged according to the timestamps in the restored financial data; The arranged financial data is set as the processed financial data.

4. The method for clustering analysis and processing of national treasury big data according to claim 1, characterized in that: The step of inputting the processed financial data into the cluster analysis layer and clustering the input processed financial data through the cluster analysis layer includes the following steps: S31. Feature extraction of financial data based on deep learning network; S32, clustering the financial data after feature extraction, and outputting a clustering result, where the clustering result includes: features of the financial data after clustering and the clusters to which the features of the financial data after clustering belong; S33. Cluster the corresponding financial data based on the characteristics of the clustered financial data to obtain clustered financial data.

5. A method for clustering analysis and processing of national treasury big data according to claim 4, characterized in that: The feature extraction of financial data based on the deep learning network includes the following steps: The deep learning network consists of three structural layers, namely the convolutional layer, the pooling structure layer and the data fully connected layer; S311, selecting part of the processed financial data from all the processed financial data to form a processed financial data training set; S312, inputting the processed financial data in the processed financial data training set into the deep learning network until the deep learning network converges to obtain a trained deep learning network; The back propagation training loss function is used in the deep learning network. The function expression is as follows: Where B represents the back propagation training loss function of financial data, θ represents the convolution layer in the deep learning network, n represents the weight of the convolution kernel, y(i) represents the number of times the financial data is trained during the propagation process, i represents the i-th training, and h(x) represents the back propagation of financial data at node x; When the value of the back-propagation training loss function drops below the set threshold, the deep learning network converges and a trained deep learning network is obtained; S313. Input all processed financial data into the trained deep learning network, and output the extracted features of the financial data.

6. A method for clustering analysis and processing of national treasury big data according to claim 4, characterized in that: The clustering process of the financial data after feature extraction comprises the following steps: Clustering the extracted features of financial data by calculating the similarity between the extracted features of financial data; The similarity calculation formula is as follows: Wherein, sim represents the similarity value, c represents the cth financial data feature, e represents the eth financial data feature, and g represents the angle between the cth financial data feature and the eth financial data feature; When the similarity value is in [0,1], it means that the two data features are similar, and the two data features are placed in the same cluster; When the similarity value is in [-1,0), it means that the two data features are not similar; Traverse all the extracted features of financial data, divide all the extracted features of financial data, and output the clustering results.

7. The method for clustering analysis and processing of national treasury big data according to claim 1, characterized in that: The data clustering is verified in batches by distributed computing through a task scheduling control mechanism, and includes the following steps: Setting processing priorities based on the type of financial data in the clustered financial data; Set the processing priority of financial voucher data higher than that of financial business data; The batch verification process using distributed computing includes the following steps: S41, predetermine a polynomial that uses both the data source point and the collection point as divisors; S42, after the polynomial is determined, binary division operation is performed on the original financial data to be sent and the determined polynomial to calculate a check code; S43, attaching the calculated check code to the end of the original financial data to be sent, and when receiving the original financial data, the collection point performs calculations based on the determined polynomial and the received original financial data, and compares the calculation result with the check code at the end of the received original financial data; When the calculation result is consistent with the check code at the end of the received original financial data, the check is passed, indicating that the data transmission is normal; When the calculation result is inconsistent with the check code at the end of the received original financial data, the check fails, indicating that the data transmission is abnormal.

8. The method for clustering analysis and processing of national treasury big data according to claim 1, characterized in that: The second matching and reconciliation of the verified financial data includes the following steps: Perform partition sum verification on the verified financial data, and compare the partition summed financial data with the saved statistical data; When the summed financial data is inconsistent with the saved statistical data, traverse the financial data to locate the error information; When the summed financial data is consistent with the saved statistical data, the secondary matching reconciliation is passed, indicating that there is no erroneous data in the collected financial data.

9. A system for implementing the method for clustering analysis and processing of national treasury big data according to any one of claims 1 to 8, characterized in that: include: Application support module, cluster analysis module, data processing module and basic support module; The basic support module is used to store middleware and financial data resources; The cluster analysis module is used to build stable, efficient and reliable data computing and model services and perform cluster analysis on the collected data through application service clusters, task scheduling frameworks, cache cluster calls and load balancing access; The data processing module is used to perform statistics and deduplication operations on financial data; The application support module is used to provide an access interface to interact with users and implement data access for different scenarios.

Images