Data management system for integrated service management platform

Abstract

The invention discloses a data management system for an integrated service management platform. The data management system comprises a data acquisition module, a data integration module, a data analysis module, a data governance module, a data modeling module, a data exchange module, a data security module, a system management module and a business application module. The data acquisition module is used for acquiring data from various data sources, and the various data sources comprise a relational database, a non-relational database and a file system; the data integration module is used for performing unified management and integration on the acquired data and supporting dynamic access of a data source; and the data analysis module is used for carrying out statistics, analysis and mining on the integrated data through a data analysis algorithm, and providing data visualization display through a data visualization tool. The integrated design is adopted, the functions of data acquisition, data storage, data processing, data analysis, data display and the like are integrated, and the data management requirements of multiple scenes and multiple service lines can be met.

Description

Technical Field

[0001] The present invention belongs to the technical field of data management, and in particular relates to a data management system for an integrated service management platform. Background Art

[0002] With the continuous development of information technology, enterprises have an increasing demand for data management. Traditional data management systems often have the following problems: 1) They cannot meet the data management needs of multiple scenarios and multiple business lines; 2) Data silos are serious, making it difficult to share and circulate data; 3) Data security is low and prone to the risk of data leakage; 4) The system is highly complex and difficult to maintain and upgrade.

[0003] In order to solve the above problems, there is an urgent need for a data management system for an integrated service management platform that can meet the data management needs of multiple scenarios and multiple business lines, realize data sharing and circulation, improve data security, reduce system complexity, and facilitate maintenance and upgrades. To this end, we propose a data management system for an integrated service management platform. Summary of the invention

[0004] The purpose of the present invention is to provide a data management system for an integrated service management platform to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a data management system for an integrated service management platform, comprising a data acquisition module, a data integration module, a data analysis module, a data governance module, a data modeling module, a data exchange module, a data security module, a system management module, and a business application module;

[0006] The data collection module is responsible for collecting data from a variety of data sources, including relational databases, non-relational databases, and file systems;

[0007] The data integration module is used to uniformly manage and integrate the collected data and support dynamic access to data sources;

[0008] The data analysis module is used to perform statistics, analysis and mining on the integrated data through data analysis algorithms, and provide data visualization through data visualization tools;

[0009] The data governance module is used to perform quality inspection, cleaning and desensitization on the integrated data to ensure the authenticity, accuracy and security of the data;

[0010] The data modeling module is used to build a data model, realize data abstraction and encapsulation, and reduce the complexity of data management; the data modeling module includes an entity relationship model and a data model conversion tool, the entity relationship model is used to represent the relationship between data; the data model conversion tool is used to convert the entity relationship model into a data storage structure;

[0011] The data exchange module is used to realize data sharing and circulation through a variety of data exchange methods;

[0012] The data security module is used to protect data security and prevent data leakage by using encryption storage and access control technology;

[0013] The system management module is responsible for the configuration, monitoring, maintenance and upgrade operations of the data management system;

[0014] The business application module is used to develop corresponding data management applications according to business needs, such as customer relationship management and supply chain management.

[0015] Preferably, the data acquisition module includes a data source connector, a data extractor, a data converter and a data loader, wherein the data source connector is used to establish connections with multiple data sources; the data extractor is used to extract data from multiple data sources; the data converter is used to convert the extracted data into a unified format; and the data loader is used to load the converted data into the data integration module.

[0016] Preferably, the data integration module includes a data warehouse unit, a data cleaning engine unit and a data quality management tool unit; the data warehouse is used to store the integrated data; the data cleaning engine is used to identify and correct data errors; and the data quality management tool unit is used to monitor data quality and provide data quality reports.

[0017] Preferably, the data analysis module includes the following formula in the process of statistics, analysis and mining of the integrated data by using the data analysis algorithm:

[0018] Mean formula: The mean is obtained by dividing the sum of all values ​​in the data set by the number of data. The formula is:

[0019] Mean = (x1+x2+…+xn) / n;

[0020] Among them, x1, x2, ..., xn represent each value in the data set, and n represents the number of data;

[0021] Median formula: After sorting the data set by size, the value in the middle is the median. If the number of data is an even number, the median is the average of the two middle values. The formula is:

[0022] Median=x(n / 2);

[0023] Among them, x(n / 2) represents the value in the middle position after the data set is sorted by size.

[0024] Preferably, in the process of statistics, analysis and mining of the integrated data by the data analysis algorithm in the data analysis module, the following formula is also included:

[0025] Standard deviation formula: The average of the squares of the differences between each value in the data set and the mean is the standard deviation, and the formula is:

[0026] Standard deviation = sqrt(1 / n*∑(xi-μ)2);

[0027] Among them, xi represents each value in the data set, μ represents the mean, n represents the number of data, and ∑ represents the sum of all data;

[0028] Correlation coefficient formula: An indicator used to measure the strength and direction of the linear relationship between two variables, that is, the correlation coefficient. The value range of the correlation coefficient is -1 to 1, where 1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no correlation. The formula is:

[0029] Correlation coefficient = cov(X,Y) / (stddev(X)*stddev(Y));

[0030] Among them, cov(X,Y) represents the covariance of variables X and Y, stddev(X) represents the standard deviation of variable X, and stddev(Y) represents the standard deviation of variable Y.

[0031] Preferably, the data governance module includes a data dictionary unit, a data access control unit, and a data security policy unit; the data dictionary unit is used to define the meaning and structure of the data; the data access control unit is used to limit access rights to the data; and the data security policy unit is used to implement data encryption and secure storage measures.

[0032] Preferably, the data exchange module comprises a data synchronization unit and a data sharing unit, wherein the data synchronization unit is used to synchronize data between different data storages, and the data sharing unit is used to exchange data with other systems or platforms.

[0033] Preferably, the business application module includes a business logic processor and a business rule engine unit, wherein the business logic processor is used to process business-related data operations; and the business rule engine unit is used to implement business rules and policies.

[0034] Preferably, the data management system also includes a mobile application module, which is used to provide a mobile data management application and supports iOS and Android platforms; the mobile application module includes a user interface unit and a mobile data processing engine unit, the user interface unit is used to perform human-computer interaction with a mobile device user, and the mobile data processing engine unit is used to perform data processing tasks on the mobile device.

[0035] Preferably, the management method of the data management system comprises the following steps:

[0036] S1. Collect data from different data sources;

[0037] S2. Integrate the collected data so that the data is stored and accessed in a unified data warehouse;

[0038] S3. Govern the integrated data to ensure the authenticity, accuracy and security of the data;

[0039] S4. Use data modeling technology to build data models and simplify data management complexity;

[0040] S5. Realize data sharing and circulation among different systems through data exchange mechanism;

[0041] S6. Apply data security measures to protect data from unauthorized access or disclosure;

[0042] S7. Maintain and upgrade the entire data management system through the system management module.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) The present invention adopts an integrated design, integrating functions such as data collection, data storage, data processing, data analysis and data display, and can meet the data management needs of multiple scenarios and multiple business lines;

[0045] (2) The present invention supports access to multiple data sources, including relational databases, non-relational databases, file systems, etc., by setting up a data acquisition module, thereby achieving unified management and integration of data;

[0046] (3) The present invention can provide data governance functions by setting up a data governance module, including data quality detection, data cleaning, and data desensitization, thereby ensuring the authenticity, accuracy, and security of the data;

[0047] (4) The present invention can support data modeling by setting up a data modeling module, and realize data abstraction and encapsulation by building a data model, thereby reducing the complexity of data management;

[0048] (5) The present invention can provide data exchange functions by setting up a data exchange module, support multiple data exchange modes, such as real-time synchronization, timed synchronization, etc., to achieve data sharing and circulation;

[0049] (6) The present invention improves data security and prevents data leakage by setting up a data security module and adopting encryption storage and access control technology;

[0050] (7) The present invention adopts a modular design by setting up a system management module, has good scalability, and is easy to maintain and upgrade. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a structural block diagram of the present invention;

[0052] Figure 2 It is a structural block diagram of the data acquisition module of the present invention;

[0053] Figure 3 It is a structural block diagram of the data integration module of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] See also Figure 1-Figure 3 ,The present invention provides a technical solution: a data management system for an integrated service management platform, including a data acquisition module, a data integration module, a data analysis module, a data governance module, a data modeling module, a data exchange module, a data security module, a system management module, and a business application module;

[0056] The data collection module is responsible for collecting data from a variety of data sources, including relational databases, non-relational databases, and file systems;

[0057] The data integration module is used to uniformly manage and integrate the collected data and support dynamic access to data sources;

[0058] The data analysis module is used to perform statistics, analysis and mining on the integrated data through data analysis algorithms, and provide data visualization through data visualization tools;

[0059] The data governance module is used to perform quality inspection, cleaning and desensitization on the integrated data to ensure the authenticity, accuracy and security of the data;

[0060] The data modeling module is used to build a data model, realize data abstraction and encapsulation, and reduce the complexity of data management; the data modeling module includes an entity relationship model and a data model conversion tool, the entity relationship model is used to represent the relationship between data; the data model conversion tool is used to convert the entity relationship model into a data storage structure;

[0061] The data exchange module is used to realize data sharing and circulation through a variety of data exchange methods;

[0062] The data security module is used to protect data security and prevent data leakage by using encryption storage and access control technology;

[0063] The system management module is responsible for the configuration, monitoring, maintenance and upgrade operations of the data management system;

[0064] The business application module is used to develop corresponding data management applications according to business needs, such as customer relationship management and supply chain management.

[0065] In this embodiment, preferably, the data acquisition module includes a data source connector, a data extractor, a data converter and a data loader, wherein the data source connector is used to establish connections with multiple data sources; the data extractor is used to extract data from multiple data sources; the data converter is used to convert the extracted data into a unified format; and the data loader is used to load the converted data into the data integration module.

[0066] In this embodiment, preferably, the data integration module includes a data warehouse unit, a data cleaning engine unit and a data quality management tool unit; the data warehouse is used to store the integrated data; the data cleaning engine is used to identify and correct data errors; and the data quality management tool unit is used to monitor data quality and provide data quality reports.

[0067] In this embodiment, preferably, the data analysis module includes the following formula in the process of statistics, analysis and mining of the integrated data by using the data analysis algorithm:

[0068] Mean formula: The mean is obtained by dividing the sum of all values ​​in the data set by the number of data. The formula is:

[0069] Mean = (x1+x2+…+xn) / n;

[0070] Among them, x1, x2, ..., xn represent each value in the data set, and n represents the number of data;

[0071] Median formula: After sorting the data set by size, the value in the middle is the median. If the number of data is an even number, the median is the average of the two middle values. The formula is:

[0072] Median=x(n / 2);

[0073] Among them, x(n / 2) represents the value in the middle position after the data set is sorted by size.

[0074] In this embodiment, preferably, in the process of statistics, analysis and mining of the integrated data by the data analysis algorithm in the data analysis module, the following formula is also included:

[0075] Standard deviation formula: The average of the squares of the differences between each value in the data set and the mean is the standard deviation, and the formula is:

[0076] Standard deviation = sqrt(1 / n*∑(xi-μ)2);

[0077] Among them, xi represents each value in the data set, μ represents the mean, n represents the number of data, and ∑ represents the sum of all data;

[0078] Correlation coefficient formula: An indicator used to measure the strength and direction of the linear relationship between two variables, that is, the correlation coefficient. The value range of the correlation coefficient is -1 to 1, where 1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no correlation. The formula is:

[0079] Correlation coefficient = cov(X,Y) / (stddev(X)*stddev(Y));

[0080] Among them, cov(X,Y) represents the covariance of variables X and Y, stddev(X) represents the standard deviation of variable X, and stddev(Y) represents the standard deviation of variable Y.

[0081] In this embodiment, preferably, the data governance module includes a data dictionary unit, a data access control unit, and a data security policy unit; the data dictionary unit is used to define the meaning and structure of the data; the data access control unit is used to limit access rights to the data; and the data security policy unit is used to implement data encryption and secure storage measures.

[0082] In this embodiment, preferably, the data exchange module includes a data synchronization unit and a data sharing unit, the data synchronization unit is used to synchronize data between different data storages, and the data sharing unit is used to exchange data with other systems or platforms.

[0083] In this embodiment, preferably, the business application module includes a business logic processor and a business rule engine unit, wherein the business logic processor is used to process business-related data operations; and the business rule engine unit is used to implement business rules and policies.

[0084] In this embodiment, preferably, the data management system also includes a mobile application module, which is used to provide a mobile data management application and supports iOS and Android platforms; the mobile application module includes a user interface unit and a mobile data processing engine unit, the user interface unit is used to perform human-computer interaction with a mobile device user, and the mobile data processing engine unit is used to perform data processing tasks on the mobile device.

[0085] In this embodiment, preferably, the management method of the data management system comprises the following steps:

[0086] S1. Collect data from different data sources;

[0087] S2. Integrate the collected data so that the data is stored and accessed in a unified data warehouse;

[0088] S3. Govern the integrated data to ensure the authenticity, accuracy and security of the data;

[0089] S4. Use data modeling technology to build data models and simplify data management complexity;

[0090] S5. Realize data sharing and circulation among different systems through data exchange mechanism;

[0091] S6. Apply data security measures to protect data from unauthorized access or disclosure;

[0092] S7. Maintain and upgrade the entire data management system through the system management module.

[0093] Principles and advantages of the present invention:

[0094] The present invention adopts an integrated design, integrating functions such as data acquisition, data storage, data processing, data analysis and data display, and can meet the data management needs of multiple scenarios and multiple business lines; by setting up a data acquisition module, it supports the access of multiple data sources, including relational databases, non-relational databases, file systems, etc., and realizes unified management and integration of data; by setting up a data governance module, it can provide data governance functions, including data quality detection, data cleaning, and data desensitization, so as to ensure the authenticity, accuracy and security of the data; by setting up a data modeling module, it can support data modeling, and by building a data model, it can realize data abstraction and encapsulation, and reduce the complexity of data management; by setting up a data exchange module, it can provide data exchange functions, support multiple data exchange methods, such as real-time synchronization, timed synchronization, etc., to realize data sharing and circulation; by setting up a data security module, encrypted storage and access control technology are adopted to improve data security and prevent data leakage; by setting up a system management module, a modular design is adopted, which has good scalability and is easy to maintain and upgrade.

[0095] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A data management system for an integrated service management platform, characterized in that: It includes data acquisition module, data integration module, data analysis module, data governance module, data modeling module, data exchange module, data security module, system management module and business application module; The data collection module is responsible for collecting data from a variety of data sources, including relational databases, non-relational databases, and file systems; The data integration module is used to uniformly manage and integrate the collected data and support dynamic access to data sources; The data analysis module is used to perform statistics, analysis and mining on the integrated data through data analysis algorithms, and provide data visualization through data visualization tools; The data governance module is used to perform quality inspection, cleaning and desensitization on the integrated data to ensure the authenticity, accuracy and security of the data; The data modeling module is used to build a data model, realize data abstraction and encapsulation, and reduce the complexity of data management; the data modeling module includes an entity relationship model and a data model conversion tool, the entity relationship model is used to represent the relationship between data; the data model conversion tool is used to convert the entity relationship model into a data storage structure; The data exchange module is used to realize data sharing and circulation through a variety of data exchange methods; The data security module is used to protect data security and prevent data leakage by using encryption storage and access control technology; The system management module is responsible for the configuration, monitoring, maintenance and upgrade operations of the data management system; The business application module is used to develop corresponding data management applications according to business needs, such as customer relationship management and supply chain management.

2. The data management system for an integrated service management platform according to claim 1, characterized in that: The data acquisition module includes a data source connector, a data extractor, a data converter and a data loader. The data source connector is used to establish connections with multiple data sources; the data extractor is used to extract data from multiple data sources; the data converter is used to convert the extracted data into a unified format; and the data loader is used to load the converted data into the data integration module.

3. The data management system for an integrated service management platform according to claim 1, characterized in that: The data integration module includes a data warehouse unit, a data cleaning engine unit and a data quality management tool unit; the data warehouse is used to store the integrated data; the data cleaning engine is used to identify and correct data errors; the data quality management tool unit is used to monitor data quality and provide data quality reports.

4. The data management system for an integrated service management platform according to claim 1, characterized in that: The data analysis module uses the data analysis algorithm to perform statistics, analysis and mining on the integrated data, including the following formula: Mean formula: The mean is obtained by dividing the sum of all values ​​in the data set by the number of data. The formula is: Mean = (x1+x2+…+xn) / n; Among them, x1, x2, ..., xn represent each value in the data set, and n represents the number of data; Median formula: After sorting the data set by size, the value in the middle is the median. If the number of data is an even number, the median is the average of the two middle values. The formula is: Median=x(n / 2); Among them, x(n / 2) represents the value in the middle position after the data set is sorted by size.

5. The data management system for an integrated service management platform according to claim 4, characterized in that: The data analysis module also includes the following formula in the process of statistics, analysis and mining of the integrated data through the data analysis algorithm: Standard deviation formula: The average of the squares of the differences between each value in the data set and the mean is the standard deviation, and the formula is: Standard deviation = sqrt(1 / n*∑(xi-μ)2); Among them, xi represents each value in the data set, μ represents the mean, n represents the number of data, and ∑ represents the sum of all data; Correlation coefficient formula: An indicator used to measure the strength and direction of the linear relationship between two variables, that is, the correlation coefficient. The value range of the correlation coefficient is -1 to 1, where 1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no correlation. The formula is: Correlation coefficient = cov(X,Y) / (stddev(X)*stddev(Y)); Among them, cov(X,Y) represents the covariance of variables X and Y, stddev(X) represents the standard deviation of variable X, and stddev(Y) represents the standard deviation of variable Y.

6. The data management system for an integrated service management platform according to claim 1, characterized in that: The data governance module includes a data dictionary unit, a data access control unit, and a data security policy unit; the data dictionary unit is used to define the meaning and structure of the data; the data access control unit is used to limit access rights to the data; and the data security policy unit is used to implement data encryption and secure storage measures.

7. The data management system for an integrated service management platform according to claim 1, characterized in that: The data exchange module includes a data synchronization unit and a data sharing unit. The data synchronization unit is used to synchronize data between different data storages, and the data sharing unit is used to exchange data with other systems or platforms.

8. The data management system for an integrated service management platform according to claim 1, characterized in that: The business application module includes a business logic processor and a business rule engine unit. The business logic processor is used to process business-related data operations; the business rule engine unit is used to implement business rules and policies.

9. The data management system for an integrated service management platform according to claim 1, characterized in that: The data management system also includes a mobile application module, which is used to provide mobile data management applications and supports iOS and Android platforms; the mobile application module includes a user interface unit and a mobile data processing engine unit, the user interface unit is used to perform human-computer interaction with mobile device users, and the mobile data processing engine unit is used to perform data processing tasks on the mobile device.

10. The data management system for an integrated service management platform according to claim 1, characterized in that: The management method of the data management system comprises the following steps: S1. Collect data from different data sources; S2. Integrate the collected data so that the data is stored and accessed in a unified data warehouse; S3. Govern the integrated data to ensure the authenticity, accuracy and security of the data; S4. Use data modeling technology to build data models and simplify data management complexity; S5. Realize data sharing and circulation among different systems through data exchange mechanism; S6. Apply data security measures to protect data from unauthorized access or disclosure; S7. Maintain and upgrade the entire data management system through the system management module.

Images