A financial data management method and system based on artificial intelligence
Through the AI-based financial data management method, combined with the Apriori algorithm, isolated forest algorithm and decision tree model, the problem that traditional methods are difficult to deal with massive financial data is solved, and the rapid classification and extraction of financial data is realized, and data utilization efficiency is improved. E-commerce financial decision-making support.
Patent Information
- Application Number
- CN202411460704.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2024-12-27
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional data management methods are difficult to effectively process massive, high-dimensional and complex financial data, especially in e-commerce companies, which are difficult to quickly identify abnormal, duplicate, error and high-value data, resulting in inefficient data utilization.
Adopting a financial data management method based on artificial intelligence, collecting financial big data and user behavior data through data terminals, performing feature extraction and clustering, using Apriori algorithm and isolated forest algorithm for correlation analysis and abnormal detection, and constructing a decision tree classification model , optimize the classification model to achieve rapid classification and extraction of data.
It has improved the management and analysis efficiency and data mining capabilities of financial data, achieved rapid and accurate abnormal data detection and classification of financial big data, and supported e-commerce financial risk control and decision-making.
Smart Images

Figure CN119004339B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of financial data analysis, and more specifically, to a financial data management method and system based on artificial intelligence. Background Art
[0002] With the rapid development of financial technology, financial institutions are facing the challenge of processing massive, high-dimensional and complex financial data. Traditional data processing methods can no longer meet the current business needs of efficiency, accuracy and real-time. Especially in e-commerce companies, a large amount of financial data has been accumulated, which contains rich business information. However, traditional data management methods are difficult to cope with large data volumes, diverse types and frequently updated data, and the detection, analysis and classification of data anomalies are inefficient, resulting in low data utilization efficiency. Especially in financial decision-making, how to quickly identify anomalies, duplications, errors and high-value data from massive data has become a problem that needs to be solved urgently. Summary of the invention
[0003] The present invention overcomes the defects of the prior art and proposes a financial data management method and system based on artificial intelligence.
[0004] The first aspect of the present invention provides a financial data management method based on artificial intelligence, comprising:
[0005] Collect financial big data and user behavior data of e-commerce companies in a preset historical time period through data terminals;
[0006] Feature extraction is performed through user behavior data and imported into a preset clustering model to perform user clustering and grouping. The clustering process is performed based on the feature similarity of user behavior data and multiple user groups are formed.
[0007] For multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets. The user behavior data in each user group is analyzed for correlation with the financial data set based on the Apriori algorithm, and the associated data is extracted to form the financial correlation feature data of each user group.
[0008] Through the isolation forest algorithm, outlier detection is performed on financial big data and user behavior data to form anomaly detection results;
[0009] Construct a classification model based on a decision tree, import financial-related feature data and anomaly detection results into the classification model as training data, perform feature selection for the decision tree by calculating the information gain of the features, construct node features in the classification model by using the CART algorithm and form a classification model based on a complete decision tree, perform classification training on the classification model for abnormal data using the training data, optimize the classification model based on the accuracy of the classification results, and generate different classification models based on different user groups;
[0010] In one analysis cycle, real-time financial data and real-time user behavior data are collected through data terminals, and real-time users are grouped by combining real-time user behavior data with clustering models. The real-time financial data and real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted. The real-time financial related data are imported into the classification model for abnormal data classification. Based on the abnormal data classification results, the real-time financial data and real-time user behavior data are divided to obtain the target data set and the abnormal data set.
[0011] In this solution, the financial big data and user behavior data of e-commerce enterprises in a preset historical period are collected through data terminals, specifically:
[0012] Set a preset historical time period;
[0013] During the preset historical period, data of the target e-commerce enterprises is collected through the data terminal of the cloud platform to obtain financial big data and user behavior data;
[0014] Financial big data includes transaction records, balance information, payment data, credit and risk assessment data;
[0015] User behavior data includes browsing history, search tags, order data, and user portrait data.
[0016] In this solution, the user behavior data is used to extract features and import the preset clustering model to perform user clustering grouping. The clustering process is based on the feature similarity of the user behavior data and forms multiple user groups, specifically:
[0017] Get user behavior data for each user;
[0018] Perform data standardization preprocessing on user behavior data, extract features based on time, behavior, content, and social dimensions, and form user feature data;
[0019] Construct a k-means-based clustering model and initialize the clustering model. Use user feature data as clustering sample data, randomly set K cluster centers, and use the clustering model to cluster the clustering sample data. The clustering process uses the standard Euclidean distance to calculate the feature similarity between sample data, and finally obtain the clustering result.
[0020] Based on the clustering results, users are divided into multiple user groups.
[0021] In this solution, for multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets, and the user behavior data in each user group is subjected to correlation analysis with the financial data set based on the Apriori algorithm, and the associated data is extracted to form the financial correlation feature data of each user group, specifically:
[0022] For multiple user groups, the financial big data is grouped to obtain multiple financial data sets;
[0023] Each financial dataset corresponds to a user group;
[0024] Analyze a user group and integrate the corresponding user behavior data with the financial data set into a fused data set;
[0025] Based on the Apriori algorithm, set the support threshold and confidence threshold;
[0026] Generate multiple data item sets based on the fused data set, calculate the support of each data item set, and filter out frequent item sets based on the support threshold;
[0027] Based on frequent item sets and confidence thresholds, filter and record the item sets with strong associations to obtain strong association rules;
[0028] In the strong association rule, two associated data item sets are randomly selected and the two data item sets are judged;
[0029] Based on the strong association rule, determine whether the two associated data item sets exist in the user behavior data and the financial data set respectively. If so, mark the two data item sets to obtain the associated feature data;
[0030] Based on strong association rules, all associated feature data based on a user group are screened out, and the data is integrated to form financial association features;
[0031] A correlation analysis is performed on each user group to obtain the financial correlation characteristics of each user group.
[0032] In this solution, the isolation forest algorithm is used to detect outliers on financial big data and user behavior data to form anomaly detection results, specifically:
[0033] Construct an isolation forest model and initialize preset parameters, including the number of trees, maximum number of samples, and contamination rate;
[0034] Train and optimize the isolation forest model through existing sample data;
[0035] The financial big data and user behavior data are standardized and imported into the isolation forest model for anomaly detection. Based on the imported data, the anomaly score of each data point is predicted and calculated, and the abnormal data is screened out for marking. After the anomaly detection is completed, the anomaly detection result is obtained.
[0036] In this solution, the construction constructs a classification model based on a decision tree, imports financial related feature data and anomaly detection results as training data into the classification model, performs feature selection of the decision tree by calculating the information gain of the features, constructs node features in the classification model by the CART algorithm and forms a classification model based on a complete decision tree, performs classification training of abnormal data on the classification model by the training data, optimizes the classification model based on the accuracy of the classification results, and generates different classification models based on different user groups, specifically:
[0037] Build a decision tree-based classification model;
[0038] Select features from the financial-related feature data of a user group, calculate the information gain for each selected feature, select node features based on the information gain, and build a complete decision tree from the selected node features in a top-down order based on the CART algorithm. Start building nodes from the root node and recursively generate child nodes until the maximum depth is reached, and generate the final leaf node to form a classification model based on the complete decision tree;
[0039] The financial related feature data and anomaly detection results are used as training data, and the anomaly detection results are used as the real anomaly determination results in the training data. The classification model is subjected to cyclic optimization classification training, and the accuracy of the classification results is compared with the actual verification data. The training is carried out until all the verification data are passed to obtain the trained classification model.
[0040] Different decision tree models are trained based on different user groups, and different classification models are constructed.
[0041] In this solution, within one analysis cycle, real-time financial data and real-time user behavior data are collected through a data terminal, real-time users are grouped by combining real-time user behavior data with a clustering model, and real-time financial data and real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted, and the real-time financial related data are imported into a classification model for abnormal data classification. Based on the abnormal data classification results, real-time financial data and real-time user behavior data are divided to obtain a target data set and an abnormal data set, specifically:
[0042] Set an analysis cycle;
[0043] In one analysis cycle, real-time financial data and real-time user behavior data are collected through data terminals;
[0044] The behavior features are extracted from real-time user behavior data and imported into the clustering model to analyze the groups to which the real-time users belong, and the current user group is obtained;
[0045] Perform correlation analysis between real-time financial data and real-time user behavior data based on the Apriori algorithm, and extract real-time financial correlation data;
[0046] The real-time financial related data is imported into the classification model corresponding to the current user group for abnormal data detection and classification. Based on the abnormal data classification results, the real-time financial data and real-time user behavior data are divided to obtain the target data set and the abnormal data set.
[0047] The second aspect of the present invention further provides a financial data management system based on artificial intelligence, the system comprising: a memory, a processor, the memory comprising a financial data management program based on artificial intelligence, the financial data management program based on artificial intelligence when executed by the processor implements the following steps:
[0048] Collect financial big data and user behavior data of e-commerce companies in a preset historical time period through data terminals;
[0049] Feature extraction is performed through user behavior data and imported into a preset clustering model to perform user clustering and grouping. The clustering process is performed based on the feature similarity of user behavior data and multiple user groups are formed.
[0050] For multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets. The user behavior data in each user group is analyzed for correlation with the financial data set based on the Apriori algorithm, and the associated data is extracted to form the financial correlation feature data of each user group.
[0051] Through the isolation forest algorithm, outlier detection is performed on financial big data and user behavior data to form anomaly detection results;
[0052] Construct a classification model based on a decision tree, import financial-related feature data and anomaly detection results into the classification model as training data, perform feature selection for the decision tree by calculating the information gain of the features, construct node features in the classification model by using the CART algorithm and form a classification model based on a complete decision tree, perform classification training on the classification model for abnormal data using the training data, optimize the classification model based on the accuracy of the classification results, and generate different classification models based on different user groups;
[0053] In one analysis cycle, real-time financial data and real-time user behavior data are collected through data terminals, and real-time users are grouped by combining real-time user behavior data with clustering models. The real-time financial data and real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted. The real-time financial related data are imported into the classification model for abnormal data classification. Based on the abnormal data classification results, the real-time financial data and real-time user behavior data are divided to obtain the target data set and the abnormal data set.
[0054] The third aspect of the present invention also provides a computer-readable storage medium, which includes an artificial intelligence-based financial data management program. When the artificial intelligence-based financial data management program is executed by a processor, it implements the steps of the artificial intelligence-based financial data management method as described in any of the above items.
[0055] The present invention discloses a financial data management method and system based on artificial intelligence. By collecting historical financial big data and user behavior data, cluster analysis is used to group users, and the correlation between financial and behavioral characteristics is mined using an association algorithm. Subsequently, the abnormal results are detected in combination with the isolation forest algorithm and used as training data to construct a multi-user group classification model based on a decision tree, and a complete classification model is constructed through the CART algorithm. In real-time analysis, the system performs association analysis on real-time financial data, and uses a classification model to classify abnormal data to generate a target data set and an abnormal data set. Through the present invention, in e-commerce financial analysis, the correlation between financial data and user behavior can be combined to detect and classify abnormal data, realize effective and rapid data classification and extraction of financial big data, improve the management and analysis efficiency of financial data and data mining capabilities, and provide accurate data support for e-commerce financial risk control and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 A flowchart of a financial data management method based on artificial intelligence of the present invention is shown;
[0057] Figure 2 A block diagram of a financial data management system based on artificial intelligence of the present invention is shown. DETAILED DESCRIPTION
[0058] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.
[0060] Figure 1 A flow chart of a financial data management method based on artificial intelligence of the present invention is shown.
[0061] like Figure 1 As shown, the first aspect of the present invention provides a financial data management method based on artificial intelligence, comprising:
[0062] S102, collecting financial big data and user behavior data of e-commerce enterprises in a preset historical time period through a data terminal;
[0063] S104, extracting features from the user behavior data and importing them into a preset clustering model to perform user clustering and grouping, the clustering process is performed based on the feature similarity of the user behavior data, and a plurality of user groups are formed;
[0064] S106, for multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets, and the user behavior data in each user group is subjected to correlation analysis with the financial data set based on the Apriori algorithm, and the correlated data is extracted to form financial correlation feature data for each user group;
[0065] S108, performing outlier detection on the financial big data and the user behavior data by using an isolation forest algorithm to generate an anomaly detection result;
[0066] S110, constructing a classification model based on a decision tree, importing the financial related feature data and the anomaly detection results as training data into the classification model, performing feature selection of the decision tree by calculating the information gain of the features, constructing node features in the classification model by using the CART algorithm and forming a classification model based on a complete decision tree, performing classification training of the classification model on the anomaly data by using the training data, optimizing the classification model based on the accuracy of the classification results, and generating different classification models based on different user groups;
[0067] S112, within one analysis cycle, real-time financial data and real-time user behavior data are collected through the data terminal, real-time users are grouped by combining the real-time user behavior data with the clustering model, the real-time financial data and the real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted, the real-time financial related data is imported into the classification model for abnormal data classification, and based on the abnormal data classification result, the real-time financial data and the real-time user behavior data are divided to obtain the target data set and the abnormal data set.
[0068] It should be noted that in the classification model based on decision tree, decision tree is a machine learning algorithm, and the constructed classification model belongs to an artificial intelligence analysis model, which can be used for classification and regression problems. In the present invention, it is used to solve classification problems.
[0069] According to an embodiment of the present invention, the financial big data and user behavior data of the e-commerce enterprise in a preset historical time period are collected through the data terminal, specifically:
[0070] Set a preset historical time period;
[0071] During the preset historical period, data of the target e-commerce enterprises is collected through the data terminal of the cloud platform to obtain financial big data and user behavior data;
[0072] Financial big data includes transaction records, balance information, payment data, credit and risk assessment data;
[0073] User behavior data includes browsing history, search tags, order data, and user portrait data.
[0074] It should be noted that the user portrait data includes age, gender, consumption habit characteristics, etc. In the subsequent correlation analysis, the present invention can perform correlation analysis and extraction on various types of financial and behavioral characteristics. For example, the user's payment amount, credit risk and other information have a certain correlation with the price of the items purchased by the user, the purchase frequency, and the type of goods browsed. Users with the same behavioral characteristics also have similar financial data collected.
[0075] According to an embodiment of the present invention, the user behavior data is used to extract features and is imported into a preset clustering model to perform user clustering grouping. The clustering process is performed based on the feature similarity of the user behavior data, and multiple user groups are formed, specifically:
[0076] Get user behavior data for each user;
[0077] Perform data standardization preprocessing on user behavior data, extract features based on time, behavior, content, and social dimensions, and form user feature data;
[0078] Construct a k-means-based clustering model and initialize the clustering model. Use user feature data as clustering sample data, randomly set K cluster centers, and use the clustering model to cluster the clustering sample data. The clustering process uses the standard Euclidean distance to calculate the feature similarity between sample data, and finally obtain the clustering result.
[0079] Based on the clustering results, users are divided into multiple user groups.
[0080] It should be noted that in the feature extraction based on time, behavior, content, and social dimensions, time refers to the time node when the behavior occurs, behavior refers to the specific behavior record, such as purchase, payment, content refers to the type of browsed goods, and social dimensions refer to behavioral features such as comments and likes. The preset clustering model can be a k-means model, which clusters data in the form of center points to divide users.
[0081] According to an embodiment of the present invention, for multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets, and the user behavior data in each user group is subjected to correlation analysis with the financial data set based on the Apriori algorithm, and the associated data is extracted to form financial correlation feature data for each user group, specifically:
[0082] For multiple user groups, the financial big data is grouped to obtain multiple financial data sets;
[0083] Each financial dataset corresponds to a user group;
[0084] Analyze a user group and integrate the corresponding user behavior data with the financial data set into a fused data set;
[0085] Based on the Apriori algorithm, set the support threshold and confidence threshold;
[0086] Generate multiple data item sets based on the fused data set, calculate the support of each data item set, and filter out frequent item sets based on the support threshold;
[0087] Based on frequent item sets and confidence thresholds, filter and record the item sets with strong associations to obtain strong association rules;
[0088] In the strong association rule, two associated data item sets are randomly selected and the two data item sets are judged;
[0089] Based on the strong association rule, determine whether the two associated data item sets exist in the user behavior data and the financial data set respectively. If so, mark the two data item sets to obtain the associated feature data;
[0090] Based on strong association rules, all associated feature data based on a user group are screened out, and the data is integrated to form financial association features;
[0091] A correlation analysis is performed on each user group to obtain the financial correlation characteristics of each user group.
[0092] It should be noted that the strong association rule records associated data groups, and each data group includes at least two data item sets.
[0093] The association algorithm is an algorithm for performing internal association data mining based on an overall data set. In the present invention, the user behavior data and the financial data set are integrated into a fused data set, and the fused data set is used as a whole to perform association data mining. In the association results, there is some associated data belonging to the same data type or different data types, such as both may belong to financial data. The present invention needs to mine the association data of different data types, that is, to screen out two data item sets that are associated with the user behavior data and the financial data, so as to construct a decision tree in the subsequent process.
[0094] According to an embodiment of the present invention, the isolation forest algorithm is used to perform outlier detection on financial big data and user behavior data to form an anomaly detection result, which is specifically:
[0095] Construct an isolation forest model and initialize preset parameters, including the number of trees, maximum number of samples, and contamination rate;
[0096] Train and optimize the isolation forest model through existing sample data;
[0097] The financial big data and user behavior data are standardized and imported into the isolation forest model for anomaly detection. Based on the imported data, the anomaly score of each data point is predicted and calculated, and the abnormal data is screened out for marking. After the anomaly detection is completed, the anomaly detection result is obtained.
[0098] It should be noted that the existing sample data is the existing data set by the user, which is stored in the system database and includes normal and abnormal data sets for training models. In the present invention, the theoretical detection of outliers adopts the isolation forest algorithm, which has low efficiency and high accuracy in detecting big data. Therefore, the present invention performs a one-time theoretical detection through the model, and uses the abnormal detection results as the actual abnormal condition results of subsequent data to train and optimize the decision tree, thereby forming a classification model that can achieve accurate and rapid detection, and can be applied to the abnormal data detection model of e-commerce finance and user behavior data.
[0099] According to an embodiment of the present invention, the construction constructs a classification model based on a decision tree, imports financial related feature data and anomaly detection results as training data into the classification model, performs feature selection of the decision tree by calculating the information gain of the features, constructs node features in the classification model by the CART algorithm and forms a classification model based on a complete decision tree, performs classification training of abnormal data on the classification model by the training data, optimizes the classification model based on the accuracy of the classification results, and generates different classification models based on different user groups, specifically:
[0100] Build a decision tree-based classification model;
[0101] Select features from the financial-related feature data of a user group, calculate the information gain for each selected feature, select node features based on the information gain, and build a complete decision tree from the selected node features in a top-down order based on the CART algorithm. Start building nodes from the root node and recursively generate child nodes until the maximum depth is reached, and generate the final leaf node to form a classification model based on the complete decision tree;
[0102] The financial related feature data and anomaly detection results are used as training data, and the anomaly detection results are used as the real anomaly determination results in the training data. The classification model is subjected to cyclic optimization classification training, and the accuracy of the classification results is compared with the actual verification data. The training is carried out until all the verification data are passed to obtain the trained classification model.
[0103] Different decision tree models are trained based on different user groups, and different classification models are constructed.
[0104] It should be noted that the verification data can be obtained from the training data. Since the financial data characteristics and user e-commerce behavior characteristics in different user groups are different, different classification models can be constructed and applied based on different user groups in the construction of the classification model. One classification model corresponds to one user group. In the selection of node features based on information gain, the greater the information gain, the better the feature classification effect, and based on this, the preferred features are selected as feature nodes;
[0105] The information gain is calculated using the following formula:
[0106] gain(A)=H(D)-H(D|A)
[0107] Where gain is information gain, A is the selected feature, H(D) represents the information entropy of set D, and H(D|A) represents the information entropy of set D under conditional feature A.
[0108] The calculation formula of the feature information entropy H(D) is:
[0109] ;
[0110] Where n is the number of possible values of feature D, and p(xi) is the probability that D takes the value xi. In addition, set D represents the financial-related feature data in this embodiment.
[0111] In an embodiment of the present invention, by analyzing the correlation relationship between user behavior data and financial data sets, feature selection of abnormal data is performed from the associated data sets, and node features are established based on the feature selection, and finally a complete decision tree is formed. In the decision tree classification model, the classification of two dimensions of financial features and user behavior features is integrated, and data with abnormal, unreasonable, missing values, etc. can be quickly classified and screened, and financial data with a correlation relationship with user behavior can be quickly detected whether there are abnormal conditions, and the decision tree is trained based on the correlation features, and can subsequently perform rapid detection and classification of abnormal data through financial associated data in real time, thereby improving the classification and processing effect of financial data and accurately extracting effective data.
[0112] According to an embodiment of the present invention, in one analysis cycle, real-time financial data and real-time user behavior data are collected through a data terminal, real-time users are grouped by combining the real-time user behavior data with a clustering model, and the real-time financial data and the real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted, and the real-time financial related data are imported into a classification model for abnormal data classification, and based on the abnormal data classification result, the real-time financial data and the real-time user behavior data are divided to obtain a target data set and an abnormal data set, specifically:
[0113] Set an analysis cycle;
[0114] In one analysis cycle, real-time financial data and real-time user behavior data are collected through data terminals;
[0115] The behavior features are extracted from real-time user behavior data and imported into the clustering model to analyze the groups to which the real-time users belong, and the current user group is obtained;
[0116] Perform correlation analysis between real-time financial data and real-time user behavior data based on the Apriori algorithm, and extract real-time financial correlation data;
[0117] The real-time financial related data is imported into the classification model corresponding to the current user group for abnormal data detection and classification. Based on the abnormal data classification results, the real-time financial data and real-time user behavior data are divided to obtain the target data set and the abnormal data set.
[0118] It should be noted that the clustering model saves the data classification status. By performing distance analysis on the clustering model through the behavioral features extracted in real time, it is possible to analyze which user group’s feature data is closest (i.e., most similar), thereby selecting the user group to which it belongs and further selecting the classification model.
[0119] It is worth mentioning that in traditional e-commerce data analysis and financial data mining, outliers, error values, missing values, etc. have a greater impact on financial analysis, which can easily lead to errors in financial decision-making analysis results and low data mining efficiency, affecting the analysis and decision-making of e-commerce companies. Traditional anomaly detection often only focuses on one-sided data detection, such as whether a certain behavior data exceeds the normal value, whether the amount in the financial data exceeds the normal range, and the detection effect is poor and inefficient. It does not combine the correlation between financial data and user behavior for anomaly analysis, and the detection effect on important data in financial data is not good, which affects corporate decision-making analysis.
[0120] It should be noted that, since data with associated features is the main data and core data for financial decision analysis, the embodiment of the present invention can effectively mine the associated data set of financial features and user behavior features by performing corresponding association analysis on historical collected data and extracting data with associated features. In abnormal data detection, the anomalies in the associated data set are important abnormal analysis data for financial decision analysis. Furthermore, the present invention uses the associated data set and related anomaly detection training data as training data for a decision tree classification model and selects corresponding feature nodes to construct a decision tree that can effectively classify abnormal data for core associated data. In addition, before constructing the classification model, by clustering and grouping the user's behavioral characteristics, different classification models are constructed for users with different e-commerce behavior characteristics, thereby achieving effective anomaly detection for data collected from different users.
[0121] Furthermore, through the present invention, a model can be constructed for accurately and quickly classifying abnormal data in financial big data, providing data support for subsequent financial decision-making analysis, and realizing efficient data mining of large-scale and complex financial big data, screening out more valuable financial data.
[0122] Figure 2 A block diagram of a financial data management system based on artificial intelligence of the present invention is shown.
[0123] The second aspect of the present invention further provides a financial data management system 2 based on artificial intelligence, the system comprising: a memory 21, a processor 22, the memory 21 comprising an artificial intelligence-based financial data management program, the artificial intelligence-based financial data management program when executed by the processor 22 implements the following steps:
[0124] Collect financial big data and user behavior data of e-commerce companies in a preset historical time period through data terminals;
[0125] Feature extraction is performed through user behavior data and imported into a preset clustering model to perform user clustering and grouping. The clustering process is performed based on the feature similarity of user behavior data and multiple user groups are formed.
[0126] For multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets. The user behavior data in each user group is analyzed for correlation with the financial data set based on the Apriori algorithm, and the associated data is extracted to form the financial correlation feature data of each user group.
[0127] Through the isolation forest algorithm, outlier detection is performed on financial big data and user behavior data to form anomaly detection results;
[0128] Construct a classification model based on a decision tree, import financial-related feature data and anomaly detection results into the classification model as training data, perform feature selection for the decision tree by calculating the information gain of the features, construct node features in the classification model by using the CART algorithm and form a classification model based on a complete decision tree, perform classification training on the classification model for abnormal data using the training data, optimize the classification model based on the accuracy of the classification results, and generate different classification models based on different user groups;
[0129] In one analysis cycle, real-time financial data and real-time user behavior data are collected through data terminals, and real-time users are grouped by combining real-time user behavior data with clustering models. The real-time financial data and real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted. The real-time financial related data are imported into the classification model for abnormal data classification. Based on the abnormal data classification results, the real-time financial data and real-time user behavior data are divided to obtain the target data set and the abnormal data set.
[0130] The third aspect of the present invention also provides a computer-readable storage medium, which includes an artificial intelligence-based financial data management program. When the artificial intelligence-based financial data management program is executed by a processor, it implements the steps of the artificial intelligence-based financial data management method as described in any of the above items.
[0131] The present invention discloses a financial data management method and system based on artificial intelligence. By collecting historical financial big data and user behavior data, cluster analysis is used to group users, and the correlation between financial and behavioral characteristics is mined using an association algorithm. Subsequently, the abnormal results are detected in combination with the isolation forest algorithm and used as training data to construct a multi-user group classification model based on a decision tree, and a complete classification model is constructed through the CART algorithm. In real-time analysis, the system performs association analysis on real-time financial data, and uses a classification model to classify abnormal data to generate a target data set and an abnormal data set. Through the present invention, in e-commerce financial analysis, the correlation between financial data and user behavior can be combined to detect and classify abnormal data, realize effective and rapid data classification and extraction of financial big data, improve the management and analysis efficiency of financial data and data mining capabilities, and provide accurate data support for e-commerce financial risk control and decision-making.
[0132] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0133] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0134] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0135] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0136] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0137] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A financial data management method based on artificial intelligence, characterized in that: include: Collect financial big data and user behavior data of e-commerce companies in a preset historical time period through data terminals; Feature extraction is performed through user behavior data and imported into a preset clustering model to perform user clustering and grouping. The clustering process is performed based on the feature similarity of user behavior data and multiple user groups are formed. For multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets. The user behavior data in each user group is analyzed for correlation with the financial data set based on the Apriori algorithm, and the associated data is extracted to form the financial correlation feature data of each user group. Through the isolation forest algorithm, outlier detection is performed on financial big data and user behavior data to form anomaly detection results; Construct a classification model based on a decision tree, import financial-related feature data and anomaly detection results into the classification model as training data, perform feature selection for the decision tree by calculating the information gain of the features, construct node features in the classification model by using the CART algorithm and form a classification model based on a complete decision tree, perform classification training on the classification model for abnormal data using the training data, optimize the classification model based on the accuracy of the classification results, and generate different classification models based on different user groups; In one analysis cycle, real-time financial data and real-time user behavior data are collected through data terminals, and real-time users are grouped by combining real-time user behavior data with clustering models. The real-time financial data and real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted. The real-time financial related data are imported into the classification model for abnormal data classification. Based on the abnormal data classification results, the real-time financial data and real-time user behavior data are divided to obtain the target data set and the abnormal data set; The financial big data and user behavior data of the e-commerce enterprise in the preset historical time period are collected through the data terminal, specifically: Set a preset historical time period; During the preset historical period, data of the target e-commerce enterprises is collected through the data terminal of the cloud platform to obtain financial big data and user behavior data; Financial big data includes transaction records, balance information, payment data, credit and risk assessment data; User behavior data includes browsing history, search tags, order data, and user portrait data; Among them, the feature extraction is performed through the user behavior data and the preset clustering model is imported to perform user clustering grouping. The clustering process is performed based on the feature similarity of the user behavior data, and multiple user groups are formed, specifically: Get user behavior data for each user; Perform data standardization preprocessing on user behavior data, extract features based on time, behavior, content, and social dimensions, and form user feature data; Construct a k-means-based clustering model and initialize the clustering model. Use user feature data as clustering sample data, randomly set K cluster centers, and use the clustering model to cluster the clustering sample data. The clustering process uses the standard Euclidean distance to calculate the feature similarity between sample data, and finally obtain the clustering result. Based on the clustering results, users are divided into multiple user groups; Among them, for multiple user groups, the financial big data is grouped accordingly to obtain multiple financial data sets, and the user behavior data in each user group is subjected to correlation analysis with the financial data set based on the Apriori algorithm, and the associated data is extracted to form the financial correlation feature data of each user group, specifically: For multiple user groups, the financial big data is grouped to obtain multiple financial data sets; Each financial dataset corresponds to a user group; Analyze a user group and integrate the corresponding user behavior data with the financial data set into a fused data set; Based on the Apriori algorithm, set the support threshold and confidence threshold; Generate multiple data item sets based on the fused data set, calculate the support of each data item set, and filter out frequent item sets based on the support threshold; Based on frequent item sets and confidence thresholds, filter and record the item sets with strong associations to obtain strong association rules; In the strong association rule, two associated data item sets are randomly selected and the two data item sets are judged; Based on the strong association rule, determine whether the two associated data item sets exist in the user behavior data and the financial data set respectively. If so, mark the two data item sets to obtain the associated feature data; Based on strong association rules, all associated feature data based on a user group are screened out, and the data is integrated to form financial association features; Conduct correlation analysis on each user group to obtain the financial correlation characteristics of each user group; Among them, the isolation forest algorithm is used to perform outlier detection on financial big data and user behavior data to form anomaly detection results, specifically: Construct an isolation forest model and initialize preset parameters, including the number of trees, maximum number of samples, and contamination rate; Train and optimize the isolation forest model through existing sample data; The financial big data and user behavior data are standardized and imported into the isolation forest model for anomaly detection. Based on the imported data, the anomaly score of each data point is predicted and calculated, and the abnormal data is screened out for marking. After the anomaly detection is completed, the anomaly detection result is obtained; The construction of a classification model based on a decision tree includes importing financial related feature data and anomaly detection results as training data into the classification model, performing feature selection of the decision tree by calculating the information gain of the features, constructing node features in the classification model by the CART algorithm and forming a classification model based on a complete decision tree, performing classification training of abnormal data on the classification model by the training data, optimizing the classification model based on the accuracy of the classification results, and generating different classification models based on different user groups, specifically: Build a decision tree-based classification model; Select features from the financial-related feature data of a user group, calculate the information gain for each selected feature, select node features based on the information gain, and build a complete decision tree from the selected node features in a top-down order based on the CART algorithm. Start building nodes from the root node and recursively generate child nodes until the maximum depth is reached, and generate the final leaf node to form a classification model based on the complete decision tree; The financial related feature data and anomaly detection results are used as training data, and the anomaly detection results are used as the real anomaly determination results in the training data. The classification model is subjected to cyclic optimization classification training, and the accuracy of the classification results is compared with the actual verification data. The training is carried out until all the verification data are passed to obtain the trained classification model. Different decision tree models are trained based on different user groups, and different classification models are constructed; Among them, in one analysis cycle, real-time financial data and real-time user behavior data are collected through the data terminal, real-time users are grouped by combining the real-time user behavior data with the clustering model, the real-time financial data and the real-time user behavior data are subjected to correlation analysis based on the Apriori algorithm, and real-time financial related data are extracted, and the real-time financial related data are imported into the classification model for abnormal data classification, and based on the abnormal data classification result, the real-time financial data and the real-time user behavior data are divided to obtain the target data set and the abnormal data set, which are specifically: Set an analysis cycle; In one analysis cycle, real-time financial data and real-time user behavior data are collected through data terminals; The behavior features are extracted from real-time user behavior data and imported into the clustering model to analyze the groups to which the real-time users belong, and the current user group is obtained; Perform correlation analysis between real-time financial data and real-time user behavior data based on the Apriori algorithm, and extract real-time financial correlation data; The real-time financial related data is imported into the classification model corresponding to the current user group for abnormal data detection and classification. Based on the abnormal data classification results, the real-time financial data and real-time user behavior data are divided to obtain the target data set and the abnormal data set.
2. A financial data management system based on artificial intelligence, characterized in that: The system includes: a memory and a processor, wherein the memory includes an artificial intelligence-based financial data management program, and when the artificial intelligence-based financial data management program is executed by the processor, the steps of the artificial intelligence-based financial data management method as described in claim 1 are implemented.
3. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes an artificial intelligence-based financial data management program, and when the artificial intelligence-based financial data management program is executed by a processor, the steps of the artificial intelligence-based financial data management method according to claim 1 are implemented.
Citation Information
Cited By
Investor emotion divergence quantitative index construction system
CN120765384A