Multi-dimensional financial data fusion analysis system

Abstract

The invention provides a multi-dimensional financial data fusion analysis system, and relates to the technical field of financial data analysis systems, and the system comprises a login platform, a data platform and a storage platform, and the login platform, the data platform and the storage platform are disposed in three groups of different servers. The login platform comprises a user authentication module, an authority management module and a platform interaction module, the data platform comprises a multi-dimensional data acquisition module and a data processing module, the data processing module performs data processing by adopting a data stream processing framework, and the storage platform comprises a data storage module and an intelligent model database. By adopting a multi-platform combination mode, deploying each platform on different servers and adopting a distributed mode, the platforms are independently arranged, and the calculation pressure of a single server is reduced, so that the response speed of the platform is guaranteed when a large amount of interaction requirements are met, a multi-dimensional data acquisition module is further utilized, and the interaction efficiency is improved. And data can be obtained from various different angles, and the involved range is wide.

Description

Technical Field

[0001] The present invention relates to the technical field of financial data analysis systems, and in particular to a multi-dimensional financial data fusion analysis system. Background Art

[0002] The main purpose of financial data analysis systems is to help financial institutions, investors, and analysts process and analyze large amounts of financial data. As financial markets become more and more complex and the amount of data grows explosively, traditional manual analysis methods can no longer meet the needs. Therefore, financial data analysis systems have emerged, which use advanced computing technology and algorithms to provide in-depth insights and decision support. At present, artificial intelligence technology is developing rapidly, so in the further research of financial data analysis systems, artificial intelligence technology will be combined with financial data analysis systems to bring better experience to users.

[0003] The development of artificial intelligence technology is inseparable from the support of a large amount of data. However, in the prior art, the data source of the financial data analysis system is single and simple, which makes it inconvenient to effectively integrate data from different sources and systems. In addition, due to the single data source, the frequency of obtaining data is also reduced, which will affect the real-time nature of the data, and then lead to the phenomenon of data islands, thereby affecting the comprehensiveness and accuracy of artificial intelligence data analysis. At the same time, it is usually deployed on a server, which increases the computing pressure of the server. When the amount of interaction is large, it will lead to a failure to respond in time when communicating with users. Therefore, the present invention proposes a multi-dimensional financial data fusion analysis system to solve the problems existing in the prior art. Summary of the invention

[0004] In response to the above problems, the purpose of the present invention is to propose a multi-dimensional financial data fusion analysis system, which has the function of acquiring multi-dimensional data and integrating and processing it, thereby providing data guarantee for artificial intelligence data analysis and solving the problems existing in the prior art.

[0005] To achieve the purpose of the present invention, the present invention is implemented through the following technical solutions: a multi-dimensional financial data fusion analysis system, including a login platform for interacting with users, a data platform for data collection and processing, and a storage platform for data storage, the login platform, the data platform and the storage platform are deployed in three different groups of servers, and the login platform, the data platform and the storage platform are connected through the Internet, the login platform is based on a microservice architecture, and the login platform includes a user authentication module, a permission management module and a platform interaction module, the user authentication module, the permission management module and the platform interaction module interact through a lightweight communication mechanism, the data platform includes a multi-dimensional data acquisition module and a data processing module, the multi-dimensional data acquisition module is provided with several groups of collection point units with a distributed design, and the data of the multi-dimensional data acquisition module is transmitted to the data processing module, the data processing model adopts a data stream processing framework for data processing, the storage platform includes a data storage module and an intelligent model database, the data storage module adopts a distributed structure for data storage, and the intelligent model database is provided with an artificial intelligence model for calling the platform interaction module.

[0006] A further improvement is that the data sources of the multidimensional data collection module include social data sources, Internet data sources, exchange data sources, bank data sources and regulatory data sources, the number of several groups of collection point units corresponds to the number of data sources, and a group of collection point units corresponds to a group of data sources.

[0007] A further improvement is that: the data processing module is provided with a data preprocessing unit, a data filtering unit, a data aggregation unit and a data conversion unit.

[0008] A further improvement is that the specific steps of data processing performed by the data stream processing framework are: S1: preprocessing the input data using a data preprocessing unit; S2: using the data filtering unit and the data aggregation unit to process the pre-processed data in sequence; S3: Use the data conversion unit to convert the data processed in S2.

[0009] A further improvement is that: the data platform is also provided with a local database for temporarily storing data locally and a data upload module for uploading data, and the data in the local database is uploaded to the data storage module through the data upload module.

[0010] A further improvement is that the platform interaction module is provided with a data query unit for users to query data, a data download unit for users to download data, an artificial intelligence interaction unit for artificial intelligence analysis interaction and a visualization data unit for displaying data.

[0011] A further improvement is that: an index data module is also provided in the storage platform, and the index data module is connected to the platform interaction module.

[0012] A further improvement is that a security log recording module is also provided in the login platform, and the security log recording module obtains the user's login location, login IP and login time.

[0013] The beneficial effects of the present invention are as follows: the multi-dimensional financial data fusion analysis system adopts a multi-platform combination method, and deploys each platform on a different server, and adopts a distributed method to make the multiple platforms independently set up, thereby reducing the computing pressure of a single server, thereby ensuring the response speed of the platform when facing a large number of interactive demands, and further utilizing the multi-dimensional data acquisition module to obtain data from a variety of different angles, covering a wide range, and the data platform is only used for data acquisition and processing, without other computing pressure, so it can maintain its ability to obtain data, thereby better ensuring the real-time nature of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the system structure framework of the present invention.

[0015] Figure 2 It is a schematic diagram of data structure acquired by the multi-dimensional data acquisition module of the present invention. DETAILED DESCRIPTION

[0016] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0017] according to Figure 1-Figure 2 As shown, this embodiment proposes a multi-dimensional financial data fusion analysis system, including a login platform for interacting with users, a data platform for data collection and processing, and a storage platform for data storage. The login platform, the data platform, and the storage platform are deployed in three different groups of servers, and the login platform, the data platform, and the storage platform are connected through the Internet. In this system, a distributed design is adopted, so that the login platform is used for user login and interaction, the data platform is used for data collection and processing, and the storage platform is used for data storage. Therefore, each platform exists independently. When a problem occurs on one of the platforms, or when it is upgraded and maintained, it will not affect other platforms. On this basis, new technologies and experimental new ideas can be quickly introduced without affecting the entire system. Accordingly, the corresponding server only needs to perform corresponding calculations without requiring other loads.

[0018] Specifically, the login platform is based on the microservice architecture, which is a software development style that decomposes the application into a set of small, independent services. Each service is built around a specific business function and can be independently deployed, run and expanded. Therefore, in this system, the login platform includes a user authentication module, a permission management module and a platform interaction module. The user authentication module is used to verify the user when logging in. After the verification is passed, the user enters the permission management module. The permission management module is equipped with permissions related to user information, and then the corresponding permissions are granted based on the verified user information, thereby allowing the user to enter the platform interaction module. Accordingly, the user authentication module and the permission management module have their own databases, which store the corresponding user information and permission information.

[0019] Furthermore, the user authentication module, the authority management module and the platform interaction module interact with each other through a lightweight communication mechanism, which is the HTTP communication protocol. Accordingly, a security log recording module is also provided in the login platform. The security log recording module obtains the user's login location, login IP and login time, that is, whenever a user visits, the relevant information is recorded (user consent is required. In order to ensure the security of the data and the authenticity of the user, users who do not accept information records will not have access rights to this system) for subsequent tracing.

[0020] Furthermore, the platform interaction module is provided with a data query unit for users to perform data queries, a data download unit for users to download data, an artificial intelligence interaction unit for artificial intelligence analysis interaction, and a visualization data unit for displaying data. That is, the system provides data query and data download functions, and also has the function of visualizing data, which can visualize the data related to user consultation in the form of charts, etc., to facilitate user queries.

[0021] The data platform includes a multidimensional data acquisition module and a data processing module. The multidimensional data acquisition module is equipped with several groups of collection point units with distributed design, and the data of the multidimensional data acquisition module is transmitted to the data processing module. The setting of the multidimensional data acquisition module is the focus of this system. First of all, the data sources of the multidimensional data acquisition module include social data sources, Internet data sources, exchange data sources, bank data sources and regulatory data sources. The number of several groups of collection point units corresponds to the number of data sources, and one group of collection point units corresponds to one group of data sources. That is, in this system, there are five groups of data sources, so there are also five groups of collection point units. When the data source has other sources, the collection point unit can also be set up accordingly. The collection point unit provides an interface for a data source, and obtains the data in the data source through web crawlers (suitable for grabbing unstructured data from websites, such as web page content, forum posts, news articles, etc.), API calls (many websites and services provide APIs (application programming interfaces) that allow developers to request and obtain data programmatically. This method is usually used to obtain structured data, such as stock quotes, etc.), database exports (when the data source is a database, data can be directly exported from the database) and other methods.

[0022] In the present invention, the social data source corresponds to the publicly available socio-economic data, enterprise information and other data. The socio-economic data includes demographic information, employment data, consumer index, housing price index, etc. These data reflect the socio-economic situation and have an important impact on the financial market. Enterprise information includes the business registration information, patent information, recruitment information, etc. of the enterprise. These data are helpful in evaluating the business conditions and development potential of the enterprise. The Internet data source corresponds to the publicly available financial market data on the Internet, such as stock prices, foreign exchange rates, commodity prices, etc. These data are the basis for investors to trade and analyze, and also include news reports, blog articles, forum discussions, etc. These contents can provide market sentiment and event-driven information, which have a certain impact on investors' decision-making. The exchange data source corresponds to the public market data provided by major stock exchanges, including stock prices, trading volumes, indices, futures, options and other information. The bank data source corresponds to the data provided by banks and other financial institutions, such as loan interest rates, deposit interest rates, exchange rates, credit scores, etc. The regulatory data source corresponds to the economic indicators, policy announcements, regulatory changes and other data released by financial regulatory agencies. Therefore, this system provides support for the analysis of subsequent artificial intelligence models by obtaining these data.

[0023] The data processing model uses a data stream processing framework for data processing. The data stream processing framework is a type of software framework specifically used to process real-time data streams. They can receive continuous data streams from different data sources and perform real-time analysis and processing on them. In this embodiment, the Apache Flink open source stream processing framework is used. Specifically, the data processing module is provided with a data preprocessing unit, a data filtering unit, a data aggregation unit, and a data conversion unit.

[0024] The specific steps for its processing process are as follows: S1: Use the data preprocessing unit to preprocess the input data. The preprocessing method includes data cleaning, which aims to detect and delete duplicate records to avoid bias in analysis, and then perform data standardization, that is, converting the data into a unified format, such as scaling all numerical features to the same range, and then perform data integration, merging data from different sources together, involving operations such as data connection and splicing to ensure that the merged data has a consistent format; S2: Use the data filtering unit and the data aggregation unit to process the pre-processed data in sequence. The data filtering unit has preset filtering conditions, which are set by the administrator to determine which data is relevant and which should be excluded. This may be based on a certain attribute value, timestamp, event type, etc. of the data, and filter the data after S1, and pass the filtered data to the data aggregation unit. The data aggregation unit is to integrate the data again for the purpose of classification and aggregation. According to different categories or types, it is classified and processed, that is, it is responsible for aggregating the data in the data stream; S3: The data processed in S2 is converted by the data conversion unit, and after being summarized, it is converted into a recognition format of the system through the data conversion unit and stored in the data storage module.

[0025] In this system, the data platform is also provided with a local database for local temporary storage of data and a data upload module for uploading data. The data in the local database is uploaded to the data storage module through the data upload module. For the data platform, it has the functions of data collection and processing, but since the platforms are independently set up, the data needs to be transmitted. In order to avoid data transmission interruption, data loss, etc. due to network problems, the local database is used for temporary storage, and then uploaded through the data upload module to avoid data loss, etc.

[0026] The storage platform includes a data storage module and an intelligent model database. The data storage module adopts a distributed structure for data storage. The intelligent model database is equipped with an artificial intelligence model for calling the platform interaction module. The artificial intelligence model is used to interact with users and provide corresponding financial analysis based on the data stored in the data storage module. Specifically, the storage platform is also equipped with an index data module, which is connected to the platform interaction module. The index data module is responsible for creating and maintaining indexes in this system. The index can quickly locate specific information in the database table, thereby reducing the time required for query. At the same time, the index can pre-sort the data, making the sorting operation more efficient. Correspondingly, the index creation rules are set by the administrator.

[0027] The present invention adopts a multi-platform combination method and deploys each platform on a different server in a distributed manner, so that the multiple platforms are independently set up, reducing the computing pressure of a single server, thereby ensuring the response speed of the platform when facing a large number of interactive demands, and further utilizing a multi-dimensional data acquisition module to obtain data from a variety of different angles, covering a wide range. The data platform is only used for data acquisition and processing, without other computing pressure, so it can maintain its ability to obtain data, thereby better ensuring the real-time nature of the data.

[0028] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the framework and scope of application of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A multi-dimensional financial data fusion analysis system, characterized by: It includes a login platform for interacting with users, a data platform for data collection and processing, and a storage platform for data storage. The login platform, data platform and storage platform are deployed in three different groups of servers, and the login platform, data platform and storage platform are connected through the Internet. The login platform is based on a microservice architecture, and the login platform includes a user authentication module, a permission management module and a platform interaction module. The user authentication module, the permission management module and the platform interaction module interact with each other through a lightweight communication mechanism. The data platform includes a multidimensional data acquisition module and a data processing module. The multidimensional data acquisition module is provided with several groups of collection point units with a distributed design, and the data of the multidimensional data acquisition module is transmitted to the data processing module. The data processing model adopts a data stream processing framework for data processing. The storage platform includes a data storage module and an intelligent model database. The data storage module adopts a distributed structure for data storage, and the intelligent model database is provided with an artificial intelligence model for calling the platform interaction module.

2. A multi-dimensional financial data fusion analysis system according to claim 1, characterized in that: The data sources of the multidimensional data collection module include social data sources, Internet data sources, exchange data sources, bank data sources and regulatory data sources. The number of the several groups of collection point units corresponds to the number of data sources, and a group of the collection point units corresponds to a group of data sources.

3. A multi-dimensional financial data fusion analysis system according to claim 1, characterized in that: The data processing module is provided with a data preprocessing unit, a data filtering unit, a data aggregation unit and a data conversion unit.

4. A multi-dimensional financial data fusion analysis system according to claim 3, characterized in that: The specific steps of data processing by the data stream processing framework are as follows: S1: preprocessing the input data using a data preprocessing unit; S2: using the data filtering unit and the data aggregation unit to process the pre-processed data in sequence; S3: Use the data conversion unit to convert the data processed in S2.

5. The multi-dimensional financial data fusion analysis system according to claim 1, characterized in that: The data platform is also provided with a local database for temporarily storing data locally and a data upload module for uploading data. The data in the local database is uploaded to the data storage module through the data upload module.

6. A multi-dimensional financial data fusion analysis system according to claim 1, characterized in that: The platform interaction module is provided with a data query unit for users to query data, a data download unit for users to download data, an artificial intelligence interaction unit for artificial intelligence analysis interaction, and a visualization data unit for displaying data.

7. The multi-dimensional financial data fusion analysis system according to claim 1, characterized in that: The storage platform is also provided with an index data module, and the index data module is connected to the platform interaction module.

8. The multi-dimensional financial data fusion analysis system according to claim 1, characterized in that: The login platform is also provided with a security log recording module, which obtains the user's login location, login IP and login time.

Images