SaaS-based cloud platform system and data storage method thereof

Abstract

The invention relates to the technical field of data storage, and discloses a SaaS-based cloud platform system and a data storage method thereof, the SaaS-based cloud platform system comprises an acquisition layer, a calculation layer, a storage layer, a data driving layer, a data application layer and a management and control layer; a tenant generates data by using an APP or SYS, and transmits the generated data to the calculation layer; the calculation layer is used for integrating and calculating the transmitted data; the storage layer is used for storing the data processed by the calculation layer; the data driving layer provides platform capability for the data application layer; the data application layer is used for creating a data type application based on the provided platform capability; and the management and control layer provides data management capability, monitoring capability, operation and maintenance capability and the like. According to the method and the device, the problem of waste of data information resources caused by difficulty in fully utilizing potential information values in the data is effectively avoided.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to a SaaS-based cloud platform system and a data storage method thereof. Background Art

[0002] SaaS is a model where software is deployed on cloud servers and provided to users via the internet. Users typically pay for the service on an on-demand basis through a subscription model, eliminating the need to purchase, install, or maintain the software and related hardware. SaaS has a wide range of applications, encompassing everything from office automation, customer relationship management (CRM), and enterprise resource planning (ERP) to project management and collaboration tools.

[0003] Most existing SaaS cloud platform systems simply store data. Since it is difficult to calculate, drive and apply data, it is difficult to fully utilize the potential information value in the data, resulting in a waste of data information resources. To this end, we provide a SaaS-based cloud platform system and its data storage method. Summary of the Invention

[0004] In order to solve the problem in the above background technology that it is difficult to fully utilize the potential information value in the data, thereby leading to waste of data information resources, the present invention provides a SaaS-based cloud platform system and a data storage method thereof.

[0005] The present invention is implemented by the following technical solution: A SaaS-based cloud platform system, characterized by comprising:

[0006] Collection layer, computing layer, storage layer, data drive layer, data application layer and management and control layer;

[0007] At the collection layer, tenants generate data by using APP or SYS and transmit the generated data to the computing layer;

[0008] The computing layer integrates and calculates the transmitted data;

[0009] The storage layer stores the data processed by the computing layer;

[0010] The data-driven layer provides platform capabilities for the data application layer;

[0011] Data application layer, creating data-based applications based on the provided platform capabilities;

[0012] The management and control layer provides data management, monitoring, and operation and maintenance capabilities.

[0013] As a further improvement to the above solution, the data includes database data and log data. The database data is synchronized to Hadoop offline in a T+1 manner. The log data includes application logs and system logs and is collected through Flume and synchronized to the Storm cluster and Hadoop in real time in the form of Kafka messages.

[0014] As a further improvement of the above solution, the computing layer includes real-time computing and offline computing. The real-time computing uses Storm technology to provide computing power in seconds, and the offline computing uses Hadoop technology to provide computing power in days.

[0015] As a further improvement of the above solution, the storage layer includes a relational DB, an analytical DB, a time series DB, messages, a KV database and cached data. The relational DB is used to store business data and metadata, the analytical DB is used to store reports and multi-dimensional query data, the time series DB is used to store status data and time series data, the messages are used to push data to the upper layer, the KV database is convenient for expansion, and the cached data accelerates data queries through distributed caching.

[0016] As a further improvement of the above solution, the platform capabilities include data services, data decision-making and AI algorithm platform. The data service provides data query and push capabilities, the data decision provides data decision-making capabilities, and the AI ​​algorithm platform is used for feature extraction, model selection, parameter verification and model training.

[0017] As a further improvement of the above solution, the management and control layer includes effect evaluation, decision configuration, data service configuration, data tracking, AB testing, data governance, metadata management, operation and maintenance, and monitoring.

[0018] As a further improvement of the above solution, the data service processes and aggregates heterogeneous data sources by accessing various data sources of the data storage layer and based on the heterogeneous distributed data access layer DAG.

[0019] As a further improvement of the above solution, the data decision provides decision-making capabilities based on the rule engine, actions, algorithm capabilities of the algorithm platform and decision trees.

[0020] As a further improvement of the above solution, the AI ​​algorithm platform includes statistical algorithms, machine learning, deep learning and time series algorithms.

[0021] A data storage method for a SaaS-based cloud platform system includes the following steps:

[0022] S1. Tenants generate data using APP or SYS and transmit the generated data to the computing layer;

[0023] S2, integrating and calculating the transmitted data;

[0024] S3, storing the data processed by the computing layer;

[0025] S4, provides platform capabilities for the data application layer;

[0026] S5. Create data-based applications based on the provided platform capabilities;

[0027] S6. Provide data management, monitoring, and operation and maintenance capabilities.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. The present invention integrates and calculates the data collected by the collection layer through the computing layer, so that it can be used to gain insights into business rules and mine potential information, thereby fully utilizing big data and achieving the purpose of empowering business and creating value.

[0030] 2. The present invention creates data-driven applications through the data application layer, which can improve efficiency and accuracy, simplify the development process, ensure cross-platform compatibility, enhance data management capabilities, and support concurrency control and multi-user environments.

[0031] 3. The present invention can build a unified, standardized and shareable global data system through the tool platform in the computing layer, thereby avoiding data redundancy and repeated calculations, and circumventing data silos and inconsistencies. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is the cloud platform system architecture diagram of the SaaS of the present invention;

[0033] Figure 2 This is a data driven architecture diagram of the present invention;

[0034] Figure 3 This is a flowchart of the data storage method of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0036] Example 1:

[0037] Please combine Figure 1-3, a SaaS-based cloud platform system of this embodiment includes an acquisition layer, a computing layer, a storage layer, a data drive layer, a data application layer, and a management and control layer;

[0038] At the collection layer, tenants generate data by using APP or SYS and transmit the generated data to the computing layer;

[0039] The computing layer integrates and calculates the transmitted data;

[0040] The storage layer stores the data processed by the computing layer;

[0041] The data-driven layer provides platform capabilities for the data application layer;

[0042] Data application layer, creating data-based applications based on the provided platform capabilities;

[0043] For various types of data applications, we abstract out the execution components and state management parts. Based on these two parts, we can build applications in the travel, operation, risk control, and credit fields.

[0044] The management and control layer provides data management, monitoring, and operation and maintenance capabilities.

[0045] The data includes database data and log data. The database data is synchronized offline to Hadoop in a T+1 manner. The log data includes application logs and system logs and is collected through Flume and synchronized to the Storm cluster and Hadoop in real time in the form of Kafka messages.

[0046] The computing layer includes both real-time and offline computing. Real-time computing uses Storm technology to provide second-level computing capabilities, while offline computing uses Hadoop technology to provide daily computing capabilities. For example, T-1 processes the last piece of data at dawn each day. It receives Kafka message data and outputs the computational results to the storage layer or directly transmits them to the data driver layer. The computing layer includes metadata such as Pipeline, DSL, Transform, and UDF.

[0047] The storage layer includes relational DB, analytical DB, time series DB, messages, KV database and cached data. The relational DB is used to store business data and metadata, the analytical DB is used to store reports and multi-dimensional query data, the time series DB is used to store status data and time series data, and the messages are used to push data to the upper layer. Real-time calculation results generally flow out to Hbase. Hbase has a distributed architecture and is easy to expand, thereby achieving large capacity, high QPS and low RT. The KV database is easy to expand, and the cached data accelerates data queries through distributed caching.

[0048] The platform capabilities include data services, data decision-making and AI algorithm platform. Data services provide data query and push capabilities, data decision-making provides data decision-making capabilities, and the AI ​​algorithm platform is used for feature extraction, model selection, parameter verification and model training.

[0049] The management and control layer includes effect evaluation, decision configuration, data service configuration, data tracking, AB testing, data governance, metadata management, operation and maintenance, and monitoring.

[0050] The data service processes and aggregates heterogeneous data sources by accessing various data sources in the data storage layer and based on the heterogeneous distributed data access layer DAG, mainly providing simple data query, compound and complex data query services and real-time data push services.

[0051] The data decision provides decision-making capabilities based on the rule engine, actions, algorithm capabilities of the algorithm platform and decision trees.

[0052] The AI ​​algorithm platform includes statistical algorithms, machine learning, deep learning and time series algorithms. This module receives data from the storage layer for offline training to obtain the required algorithm model and then makes predictions on new data.

[0053] A data storage method for a SaaS-based cloud platform system includes the following steps:

[0054] The first step is that tenants generate data by using APP or SYS and transmit the generated data to the computing layer;

[0055] The second step is to integrate and calculate the transmitted data;

[0056] The third step is to store the data processed by the computing layer;

[0057] Step 4: Provide platform capabilities for the data application layer;

[0058] Step 5: Create data-based applications based on the provided platform capabilities;

[0059] Step 6: Provide data management, monitoring, and operation and maintenance capabilities.

[0060] The implementation principle of a SaaS-based cloud platform system and its data storage method in the embodiment of the present application is as follows: tenants generate data by using APP or SYS and transmit the generated data to the computing layer. The computing layer integrates and calculates the transmitted data so that it can be used to gain insights into business rules and mine potential information, thereby fully utilizing big data and achieving the purpose of empowering business and creating value. The storage layer will store the data processed by the computing layer, the data-driven layer will provide platform capabilities to the data application layer, the data application layer will create data-type applications based on the provided platform capabilities, and the management and control layer will provide data management capabilities, monitoring capabilities, and operation and maintenance capabilities.

[0061] Example 2:

[0062] This embodiment is a further improvement on the first embodiment in that the offline computing uses Hadoop technology to provide computing capabilities in units of hours, weeks, and months.

[0063] The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and replacements made by technicians in this field on the basis of the present invention fall within the scope of protection required by the present invention.

Claims

1. A SaaS-based cloud platform system, characterized in that: include: Collection layer, computing layer, storage layer, data drive layer, data application layer and management and control layer; At the collection layer, tenants generate data by using APP or SYS and transmit the generated data to the computing layer; The computing layer integrates and calculates the transmitted data; The storage layer stores the data processed by the computing layer; The data-driven layer provides platform capabilities for the data application layer; Data application layer, creating data-based applications based on the provided platform capabilities; The management and control layer provides data management, monitoring, and operation and maintenance capabilities.

2. A SaaS-based cloud platform system according to claim 1, characterized in that: The data includes database data and log data. The database data is synchronized offline to Hadoop in a T+1 manner. The log data includes application logs and system logs and is collected through Flume and synchronized to the Storm cluster and Hadoop in real time in the form of Kafka messages.

3. A SaaS-based cloud platform system according to claim 2, characterized in that: The computing layer includes real-time computing and offline computing. The real-time computing uses Storm technology to provide computing power in seconds, and the offline computing uses Hadoop technology to provide computing power in days.

4. A SaaS-based cloud platform system according to claim 3, characterized in that: The storage layer includes relational DB, analytical DB, time series DB, messages, KV database and cached data. The relational DB is used to store business data and metadata, the analytical DB is used to store reports and multi-dimensional query data, the time series DB is used to store status data and time series data, the messages are used to push data to the upper layer, the KV database is convenient for expansion, and the cached data accelerates data queries through distributed cache.

5. A SaaS-based cloud platform system according to claim 4, characterized in that: The platform capabilities include data services, data decision-making and AI algorithm platform. The data services provide data query and push capabilities, the data decision-making provides data decision-making capabilities, and the AI ​​algorithm platform is used for feature extraction, model selection, parameter verification and model training.

6. A SaaS-based cloud platform system according to claim 5, characterized in that: The management and control layer includes effect evaluation, decision configuration, data service configuration, data tracking, AB testing, data governance, metadata management, operation and maintenance, and monitoring.

7. A SaaS-based cloud platform system according to claim 6, characterized in that: The data service processes and aggregates heterogeneous data sources by accessing various data sources in the data storage layer and based on the heterogeneous distributed data access layer DAG.

8. The SaaS-based cloud platform system according to claim 7, wherein: The data decision provides decision-making capabilities based on the rule engine, actions, algorithm capabilities of the algorithm platform and decision trees.

9. The SaaS-based cloud platform system according to claim 8, wherein: The AI ​​algorithm platform includes statistical algorithms, machine learning, deep learning and time series algorithms.

10. A data storage method for a SaaS-based cloud platform system according to claims 1-9, characterized in that: The following steps are involved: S1. Tenants generate data using APP or SYS and transmit the generated data to the computing layer; S2, integrating and calculating the transmitted data; S3, storing the data processed by the computing layer; S4, provides platform capabilities for the data application layer; S5. Create data-based applications based on the provided platform capabilities; S6. Provide data management, monitoring, and operation and maintenance capabilities.

Images