An improved collaborative filtering algorithm model for financial big data analysis

Through an improved collaborative filtering algorithm to identify abnormal data in financial big data analysis, the problem of failure to effectively identify abnormal data in the existing technology is solved, and the accuracy of financial data and analysis efficiency is improved.

CN111768283BActive Publication Date: 2024-11-12BEIJING KAIXUAN CHUANGZHI TECHNOLOGY CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202010623292.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-01
Publication Date
2024-11-12
Estimated Expiration
2040-07-01

AI Technical Summary

Technical Problem

The prior art has failed to effectively identify and process abnormal data in financial big data analysis, focusing only on data mining and ignoring the data accuracy and identification issues of abnormal data.

Method used

采用改进型协同过滤算法,通过多数据通道并发接收财务数据流,划分为第一和第二数据集,并将其发送至相应的数据链进行协同匹配分析,利用时序更新、信任度计算和预测评分优化步骤识别异常数据。

Benefits of technology

Quickly identify abnormal data in financial payment data, improve data accuracy and analysis efficiency, and ensure the quality of financial data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111768283B_ABST
    Figure CN111768283B_ABST
Patent Text Reader

Abstract

The present invention proposes a financial big data analysis method, which is implemented based on an improved collaborative filtering algorithm. The method includes the steps of concurrently receiving financial data streams through multiple data channels, performing attribute analysis on the financial data streams, dividing them into at least a first data set and a second data set, sending the first data set and the second data set to a first data link and a second data link respectively, and performing collaborative matching analysis on the financial data stream using the first data link and the second data link. The first data link or the second data link includes at least one data node, each of which corresponds to a payer information or a payee information, the payer information includes a first identity ID for uniquely identifying the payer, and the payee information includes a second identity ID for uniquely identifying the payee. The method proposed in the present invention can combine big data technology and collaborative filtering technology to quickly identify abnormal financial payment data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of big data processing, and in particular relates to a financial big data analysis method of an improved collaborative filtering algorithm model. Background Art

[0002] Big data is originally a term in the IT industry. It refers to a collection of data that cannot be captured, managed and processed by conventional software tools within a certain time frame. It is a massive, high-growth and diversified information asset that requires new processing models to have stronger decision-making power, insight discovery and process optimization capabilities.

[0003] With the evolution of big data technology, the concept of industry big data has also emerged in specific industries, such as medical big data, engineering big data, financial big data, etc. Financial big data is big data generated based on financial activities, of which the most important are various payment events and abnormal financial data. The big data era created by big data processing technology and big data management has not only changed the concept of traditional financial management, but also changed the positioning of financial management functions in enterprises, making financial control paths and models more diversified, which means that there will be a major breakthrough in the reshaping of the professional ability structure and career development bottlenecks of financial workers.

[0004] The foundation of big data is that the data must be accurate. If the data is not updated in a timely manner, or is false or erroneous, then there is no quality at all. This is especially true for financial big data.

[0005] In the prior art, a Chinese invention patent application with application number CN201810382074.4 has been retrieved, which proposes a financial analysis management system and method based on big data. The system includes: a financial statement acquisition module, used to obtain the original forms of various financial statements of the target enterprise; a financial item extraction module, used to extract predetermined types of financial items and corresponding financial amounts from the original forms of the various financial statements; a financial vector generation module, used to generate financial vectors; a financial vector recognition module, used to match the financial vector with a pre-trained financial vector recognition model; a financial type output module, used to output the financial type corresponding to the target enterprise after determining the type of the financial vector. The financial analysis management system based on big data of this application can comprehensively extract financial indicators in financial statements and perform comprehensive analysis based on machine learning, fully explore the regular characteristics of the analysis object in terms of financial information, make the analysis results more comprehensive, and save labor costs at the same time;

[0006] The Chinese invention patent application with application number CN201810553262.9 proposes a big data intelligent analysis service system for industry applications, including a cluster management subsystem, a data acquisition subsystem, an AI analysis modeling subsystem, an AI service configuration tool, an analysis display tool, and a data service subsystem. The big data intelligent analysis service system for industry applications of the present invention can be combined with industry-specific solutions to quickly build a big data intelligent analysis application system for different industry applications, which can effectively improve the industry's big data analysis and processing capabilities. As a basic platform for big data intelligent analysis, it can quickly expand various types of industry-specific big data analysis application products.

[0007] However, the inventors found that existing solutions only focus on data mining itself, but do not involve the accuracy of the data itself or the identification of abnormal data. Summary of the invention

[0008] In order to solve the above technical problems, the present invention proposes a financial big data analysis method, which is implemented based on an improved collaborative filtering algorithm. The method includes the steps of concurrently receiving financial data streams through multiple data channels, performing attribute analysis on the financial data streams, dividing them into at least a first data set and a second data set, sending the first data set and the second data set to the first data link and the second data link respectively, and using the first data link and the second data link to perform collaborative matching analysis on the financial data stream. The first data link or the second data link includes at least one data node, each of which corresponds to a payer information or a payee information, the payer information includes a first identity ID for uniquely identifying the payer, and the payee information includes a second identity ID for uniquely identifying the payee. The method proposed in the present invention can combine big data technology and collaborative filtering technology to quickly identify abnormal financial payment data.

[0009] Specifically, in a first aspect of the present invention, a financial big data analysis method is provided, the method is implemented based on an improved collaborative filtering algorithm, and the method comprises the following steps:

[0010] Step S100: concurrently receiving financial data streams through multiple data channels, wherein the financial data streams are sent in units of payment events;

[0011] Step S200: performing attribute analysis on the financial data stream to divide it into at least a first data set and a second data set;

[0012] Step S300: sending the first data set and the second data set to a first data link and a second data link respectively;

[0013] Step S400: updating the first data link and the second data link based on the improved collaborative filtering algorithm;

[0014] Step S500: Use the first data link and the second data link to perform collaborative matching analysis on the financial data flow.

[0015] In the present invention, the payment event includes payer information and payee information; the payer information includes payer ID number, payment method, payment medium and payment request time, and the payee information includes payee ID number, payment method, payment medium and payment response time;

[0016] The first data set includes payer information, payment terminal hardware information, payment method, and payment request time;

[0017] The second data set includes payee information, hardware information of the payment terminal, payment result, and payment response time;

[0018] The first data link is established based on a first data set, and the second data link is established based on a second data set.

[0019] The first data link or the second data link includes at least one data node, each of the data nodes corresponds to a payer information or a payee information, the payer information includes a first identity ID for uniquely identifying the payer, and the payee information includes a second identity ID for uniquely identifying the payee.

[0020] The improved collaborative filtering algorithm includes a time series update step, a trust calculation step, and a prediction score optimization step;

[0021] The timing update step calculates the similarity between the existing data nodes in the first data link or the second data link and the first data stream or the second data stream based on the payment request time or payment response time corresponding to the existing data nodes in the first data link or the second data link;

[0022] The trust degree calculation step calculates the trust degree between the sending network of the first data stream or the second data stream and the data connection network of the first data link or the second data link;

[0023] The prediction score optimization step calculates the prediction scores of the existing data nodes in the first data chain or the second data chain and the first data stream or the second data stream based on the similarity and trust, and recommends data nodes with prediction scores exceeding a predetermined value as nodes to be added.

[0024] If there is no data node corresponding to the second identity ID in the second data chain, determining whether there is a data node recommended for joining from the second data chain based on the improved collaborative filtering algorithm, specifically including:

[0025] If it is determined that there is no data node recommended to be added in the second data chain, the second data set is marked as an abnormal data set.

[0026] Further advantages of the present invention will be further reflected in detail in the specific embodiments section in conjunction with the drawings of the specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 This is an overall flow chart of a financial big data analysis method according to an embodiment of the present invention.

[0029] Figure 2 yes Figure 1 Specific analysis data flow chart of the financial big data analysis method

[0030] Figure 3 yes Figure 1 Specific embodiment 1 of the financial big data analysis method

[0031] Figure 4 yes Figure 1 Specific embodiment 2 of the financial big data analysis method DETAILED DESCRIPTION

[0032] The invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0033] Reference Figure 1 It is an overall flow chart of a financial big data analysis method according to an embodiment of the present invention.

[0034] Figure 1 In the method, the method consists of steps S100-S500, and each step is specifically implemented as follows:

[0035] Step S100: concurrently receiving financial data streams through multiple data channels, wherein the financial data streams are sent in units of payment events;

[0036] Step S200: performing attribute analysis on the financial data stream to divide it into at least a first data set and a second data set;

[0037] Step S300: sending the first data set and the second data set to a first data link and a second data link respectively;

[0038] Step S400: updating the first data link and the second data link based on the improved collaborative filtering algorithm;

[0039] Step S500: Use the first data link and the second data link to perform collaborative matching analysis on the financial data flow.

[0040] exist Figure 1 In an embodiment, a payment event includes payer information and payee information; the payer information includes payer ID number, payment method, payment medium and payment request time, and the payee information includes payee ID number, payment method, payment medium and payment response time.

[0041] As an example, the payer ID number can be a payment card number, payer ID number, payer mobile phone number, etc., which can uniquely identify the payer; the payment method / collection method includes various payment methods such as card payment, flash payment, code scanning payment, biometric payment (such as fingerprint payment, code scanning payment, iris payment, etc.), Bluetooth payment, cardless payment, NFC payment, etc.;

[0042] Payment media include bank cards, wearable devices, and portable mobile payment devices;

[0043] Payment media may include various payment devices, such as POS machines, barcode scanners, etc.

[0044] The hardware information of the payment terminal includes the model of the mobile payment device, IMEI code, CPU model and other fixed and unchangeable hardware parameters; the hardware information of the payment terminal is similar.

[0045] The payment method or collection method includes one or a combination of card payment, flash payment, code scanning payment, biometric payment, Bluetooth payment, cardless payment, and NFC payment.

[0046] In this embodiment, the first data link is established based on a first data set, and the second data link is established based on a second data set.

[0047] exist Figure 1 Based on Figure 2 . Performing attribute analysis on the financial data stream to divide it into at least a first data set and a second data set, that is, grouping and dividing a payment event (also called a payment event);

[0048] More specifically, the first data link or the second data link includes at least one data node, each of the data nodes corresponds to a payer information or a payee information, the payer information includes a first identity ID for uniquely identifying the payer, and the payee information includes a second identity ID for uniquely identifying the payee.

[0049] The first data set includes payer information, payment terminal hardware information, payment method, and payment request time;

[0050] The second data set includes payee information, hardware information of the payment terminal, payment result and payment response time.

[0051] See next Figure 3-4 .

[0052] In step S400, based on the improved collaborative filtering algorithm, updating the first data link specifically includes:

[0053] The first data link currently receives a request for joining the first data set, wherein the request for joining includes payer information;

[0054] Generate a first identity ID of the payer according to the payer information;

[0055] Determine whether there is a data node corresponding to the first identity ID in the first data link;

[0056] If it exists, adding the first data set to the data node;

[0057] If it does not exist, at least one data node recommended for joining is given from the first data chain based on an improved collaborative filtering algorithm.

[0058] In step S400, updating the second data link based on the improved collaborative filtering algorithm specifically includes:

[0059] The current second data link receives a request for joining the second data set, wherein the request for joining includes payee information;

[0060] Generate a second identity ID of the payee according to the payee information;

[0061] Determine whether there is a data node corresponding to the second identity ID in the second data link;

[0062] If it exists, adding the second data set to the data node;

[0063] If not, based on the improved collaborative filtering algorithm, it is determined from the second data chain whether there is a data node recommended for joining.

[0064] If there is no data node corresponding to the second identity ID in the second data chain, determining whether there is a data node recommended for joining from the second data chain based on the improved collaborative filtering algorithm, specifically including:

[0065] If it is determined that there is no data node recommended to be added in the second data chain, the second data set is marked as an abnormal data set.

[0066] The step S500 uses the first data link and the second data link to perform collaborative matching analysis on the financial data flow, specifically including:

[0067] The payer information and the payee information are matched with the corresponding first data set for all second data sets marked as abnormal data sets to obtain a matching result.

[0068] Next, the improved collaborative filtering algorithm used in various embodiments of the present invention is introduced in detail.

[0069] In the present invention, the improved collaborative filtering algorithm introduces the ideas of time series update, trust, prediction score optimization and structuring on the basis of the traditional collaborative filtering algorithm.

[0070] The traditional collaborative filtering algorithm was first seen in the collaborative filtering technology used in the email system Tapestry in the 1990s, and was mainly used for personalized recommendations.

[0071] On this basis, various literatures have further improved the recommendation technology of collaborative filtering to improve accuracy. Relevant technical literatures include:

[0072] L.Xiang, Q.Yuan, S.Zhao, et al.Temporal recommendation on graphs Vialong-and short-term preference fusion[C].ACM international conference on Knowledge discovery and data mining, Washington, 2010, 723-731.

[0073] Y.Ding, X.Li.Time weight collaborative filtering[C].ACM international conference on Information and knowledge management, Bremen, 2005, 485-492.

[0074] Y.Koren.Collaborative filtering with temporal dynamics[J].Communications of the ACM, 2010.

[0075] However, the prior art has not yet retrieved any literature that specifically applies collaborative filtering technology to financial big data analysis, which is the first contribution of the present invention.

[0076] Specifically, in the present invention, the improved collaborative filtering algorithm includes a time series update step, a trust calculation step, and a prediction score optimization step;

[0077] The timing update step calculates the similarity between the data nodes existing in the first data link or the second data link and the first data stream or the second data stream based on the payment request time or payment response time corresponding to the data nodes existing in the first data link or the second data link;

[0078] The trust degree calculation step calculates the trust degree between the sending network of the first data stream or the second data stream and the data connection network of the first data link or the second data link;

[0079] The prediction score optimization step calculates the prediction scores of the existing data nodes in the first data chain or the second data chain and the first data stream or the second data stream based on the similarity and trust, and recommends data nodes with prediction scores exceeding a predetermined value as nodes to be added.

[0080] It is worth pointing out that in the above model, the predetermined value is set according to the attributes of financial big data, and its accuracy is sufficient to distinguish normal data from abnormal data.

[0081] If in a certain prediction score optimization step, no prediction score exceeds the predetermined value, it is determined that there is no data node recommended for addition in the second data chain, and the second data set is marked as an abnormal data set.

[0082] On this basis, all second data sets marked as abnormal data sets and the corresponding first data sets are matched with the payer information and the payee information to obtain a matching result, thereby realizing the abnormal identification of financial big data.

[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A financial big data analysis method, the method being implemented based on an improved collaborative filtering algorithm, comprising the following steps: Step S100: Concurrently receiving financial data streams through multiple data channels, wherein the financial data streams are sent in units of payment events; A unit's payment event includes payer information and payee information; The payee information includes the payee ID number, payment method, payment medium and payment request time, and the payee information includes the payee ID number, payment method, payment medium and payment response time; Step S200: Analyze the attributes of the financial data stream and divide it into at least a first data set and a second data set; the first data set includes payer information, payment terminal hardware information, payment method, and payment request time; The second data set includes payee information, hardware information of the payment terminal, payment result, and payment response time; Step S300: sending the first data set and the second data set to the first data link and the second data link respectively; The first data link is established based on the first data set, and the second data link is established based on the second data set; Step S400: updating the first data link and the second data link based on the improved collaborative filtering algorithm; Step S500: performing collaborative matching analysis on the financial data stream using the first data link and the second data link; Its characteristics are: In step S400, updating the first data link based on the improved collaborative filtering algorithm specifically includes: The first data link currently receives a request for joining the first data set, where the request includes payer information; Generate a first identity ID of the payer according to the payer information; Determine whether there is a data node corresponding to the first identity ID in the first data chain; If it exists, adding the first data set to the data node; If not, then based on the improved collaborative filtering algorithm, give at least one data node recommended for joining from the first data chain; In step S400, updating the second data link based on the improved collaborative filtering algorithm specifically includes: The second data link currently receives a request to join the second data set, the request including payee information; and generates a second identity ID of the payee based on the payee information. Determine whether there is a data node corresponding to the second identity ID in the second data link; If it exists, adding the second data set to the data node; If not, determining whether there is a data node recommended for joining from the second data chain based on an improved collaborative filtering algorithm; If it is determined that there is no recommended data node in the second data chain, marking the second data set as an abnormal data set; The step S500 performs collaborative matching analysis on the financial data stream using the first data link and the second data link, specifically including: Matching the payer information and payee information of all second data sets marked as abnormal data sets with the corresponding first data sets to obtain a matching result; The improved collaborative filtering algorithm includes a time series update step, a trust calculation step, and a prediction score optimization step; The timing updating step calculates the similarity between the data nodes existing in the first data link or the second data link and the first data stream or the second data stream based on the payment request time or payment response time corresponding to the data nodes existing in the first data link or the second data link; The trust calculation step calculates the trust between the sending network of the first data stream or the second data stream and the data connection network of the first data link or the second data link; The prediction score optimization step calculates the prediction scores of the existing data nodes in the first data chain or the second data chain and the first data stream or the second data stream based on the similarity and trust, and recommends the data nodes whose prediction scores exceed a predetermined value as nodes to be added.

2. The financial big data analysis method according to claim 1, characterized in that: The first data chain or the second data chain includes at least one data node, each of the data nodes corresponds to a payer information or a payee information, the payer information includes a first identity ID for uniquely identifying the payer, and the payee information includes a second identity ID for uniquely identifying the payee.

3. The financial big data analysis method according to claim 1, characterized in that: The payment media include: bank cards and wearable payment devices.

4. A computer-readable storage medium having computer-executable program instructions stored thereon, wherein the executable program instructions are executed by a processor and a memory to implement the method according to any one of claims 1 to 3.

Citation Information

Patent Citations

  • Industrial application-oriented big data intelligent analysis service system

    CN108804630A

  • Intelligent predetermined plan system based on collaborative filtering

    CN102915484A

  • Finance analysis management system and method based on big data

    CN108563783A

  • Data processing method and device in online payment process

    CN108647944A

  • Improved collaborative filtering algorithm-based service recommendation model

    CN108694234A