Data aggregation and sharing management system based on big data

Abstract

The invention discloses a data aggregation sharing management system based on big data, and particularly relates to the technical field of data sharing, which comprises a time division module, a shared data acquisition module, a shared data analysis module, a comprehensive analysis module and an early warning module. The system effectively solves the defects that a traditional system is not comprehensive in data collection, not deep in analysis, not timely in early warning and the like, the accuracy and the real-time performance of data management are remarkably improved, enterprises can be helped to comprehensively master the condition of data aggregation and sharing, potential problems can be found in time, early warning can be given out, and the system is suitable for popularization and application. The method provides powerful data support for decision and operation of an enterprise, on the whole, the efficiency and the quality of data management are remarkably improved, and the competitiveness and the anti-risk capability of the enterprise are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of data sharing, and more specifically, to a data aggregation and sharing management system based on big data. Background Art

[0002] In today's era of rapid development of informatization, big data has become an indispensable and important resource for corporate decision-making and operations. However, with the rapid growth of data volume and the diversification of data sources, how to efficiently aggregate, manage and share this data has become an urgent problem to be solved. Traditional data aggregation and sharing management systems often have defects such as incomplete data collection, in-depth analysis, and untimely warnings, which make it difficult to meet the company's requirements for accuracy and real-time data management.

[0003] First, traditional systems have limitations in data collection. They can often only collect data on some key indicators, but ignore other equally important data dimensions, such as user behavior data, network security data, etc. The incompleteness of data collection leads to one-sided analysis results, which makes it difficult to fully reflect the actual situation of data aggregation and sharing management.

[0004] Secondly, the lack of depth and breadth of data analysis is also a significant problem of traditional systems. They usually only stay at the surface analysis of data and lack the ability to dig deep and conduct quantitative evaluation. This makes it difficult for managers to quickly find the root cause of the problem when faced with complex data, and they are unable to develop effective solutions.

[0005] In addition, traditional systems also have obvious deficiencies in early warning functions. They can often only handle problems after they occur, but cannot provide early warnings and interventions before problems occur. This delayed processing method not only increases the operational risks of the enterprise, but also reduces the efficiency and quality of data management. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a data aggregation and sharing management system based on big data, which solves the problems raised in the above-mentioned background technology through the following scheme.

[0007] To achieve the above object, the present invention provides the following technical solution: a data aggregation and sharing management system based on big data, comprising:

[0008] Time zone division module: used to determine the target enterprise's data aggregation and shared management data as the target data zone, divide the target data zone into sub-data zones by equal time division, and mark them as 1, 2...n in sequence;

[0009] Shared data collection module: used to collect user behavior data, system performance data, data sharing efficiency data, and network security data of each sub-data area, and transmit the collected data to the shared data analysis module;

[0010] Shared data analysis module: including user behavior data analysis unit, system performance data analysis unit, data sharing efficiency data analysis unit, and network security data analysis unit, used to analyze the data transmitted by the shared data acquisition module and transmit the analysis results to the comprehensive analysis module;

[0011] Comprehensive analysis module: used to establish a comprehensive analysis model, import the data transmitted by the shared data analysis module into the comprehensive analysis model, calculate the comprehensive abnormal index of the target data area, and transmit it to the early warning module;

[0012] Early warning module: used to establish a preset value of the comprehensive abnormality index, judge the comprehensive abnormality index of the target data area through the preset value of the comprehensive abnormality index, and issue an early warning signal based on the judgment result.

[0013] Preferably, the user behavior data include page dwell time, click frequency, page jump rate, and user churn rate, which are marked as Ut, Uf, Uj, and Ul, respectively; the system performance data include response time, maximum number of parallel requests, resource utilization, and error rate, which are marked as Pt, Pb, Pe, and Pw, respectively; the data sharing efficiency data include data transmission rate, data integrity rate, data delay, and data access frequency, which are marked as Ev, Ec, El, and Ef, respectively; the network security data include intrusion detection rate, malware interception rate, encrypted data ratio, and vulnerability repair time, which are marked as Cd, Cm, Ce, and Ct, respectively.

[0014] Preferably, the user behavior data analysis unit is used to establish a user behavior data analysis model, import the user behavior data transmitted by the shared data acquisition module into the user behavior data analysis model, and calculate the user behavior feature value of each sub-data area, which is specifically expressed as: SU i Indicates the user behavior characteristic value of the ith sub-data area, Ut i Indicates the page residence time of the i-th sub-data area, Uf i Indicates the click frequency of the ith sub-data area, Uj i represents the page jump rate of the ith sub-data area, Ul i represents the user churn rate of the i-th sub-data area, and μ1 represents other influencing factors of the customer behavior characteristic value.

[0015] Preferably, the system performance data analysis unit is used to establish a system performance data analysis model, import the system performance data transmitted by the shared data acquisition module into the system performance data analysis model, and calculate the system performance characteristic value of each sub-data area, which is specifically expressed as: SP i represents the system performance characteristic value of the ith sub-data area, Pt i represents the response time of the ith sub-data area, Pb i represents the maximum number of parallel requests for the ith sub-data area, Pe i represents the resource utilization of the ith sub-data area, Pw i represents the error rate of the ith sub-data region, Pb max represents the maximum number of parallel requests for the target data area, and μ2 represents other influencing factors of the system performance characteristic value.

[0016] Preferably, the data sharing efficiency data analysis unit is used to establish a data sharing efficiency data analysis model, import the data sharing efficiency data transmitted by the shared data acquisition module into the data sharing efficiency data analysis model, and calculate the data sharing efficiency value of each sub-data area, which is specifically expressed as: SE i represents the data sharing efficiency value of the i-th sub-data area, Ev i represents the data transmission rate of the ith sub-data area, Ec i represents the data integrity rate of the ith sub-data area, El i represents the data delay of the ith sub-data area, Ef i represents the data access frequency of the i-th sub-data area, and μ3 represents other influencing factors of the data sharing efficiency value.

[0017] Preferably, the network security data analysis unit is used to establish a network security data analysis model, import the network security data transmitted by the shared data acquisition module into the network security data analysis model, and calculate the network security effectiveness value of each sub-data area, which is specifically expressed as: SC i represents the network security effectiveness value of the ith sub-data area, Cd i represents the intrusion detection rate of the ith sub-data area, Cm i represents the malware interception rate of the ith sub-data area, Ce i Indicates the proportion of encrypted data in the ith sub-data area, Ct i represents the vulnerability repair time of the i-th sub-data area, and μ4 represents other influencing factors of the network security effectiveness value.

[0018] Preferably, the comprehensive analysis model is specifically expressed as: η represents the comprehensive abnormality index of the target data area, SU i represents the user behavior feature value of the ith sub-data area, SP i represents the system performance characteristic value of the ith sub-data area, SE i represents the data sharing efficiency value of the i-th sub-data area, SC i represents the network security effectiveness value of the ith sub-data area, and λ represents other influencing factors of the comprehensive anomaly index.

[0019] Preferably, the preset value of the comprehensive abnormality index is marked as η Def , when η Def >η, it means that the comprehensive anomaly index of the target data area is less than the preset value of the comprehensive anomaly index, indicating that the data aggregation and sharing quality of the target enterprise is good. Def When <η, it means that the comprehensive abnormality index of the target data area is greater than the preset value of the comprehensive abnormality index, indicating that the data aggregation and sharing quality of the target enterprise is poor, and an early warning signal is sent to the management terminal.

[0020] Technical effects and advantages of the present invention:

[0021] The present invention divides the target enterprise's data aggregation and shared management data into sub-data areas of equal time periods through a time division module, which helps to more finely monitor and analyze data changes in different time periods, thereby more accurately identifying potential problems or trends and improving the timeliness and accuracy of data management; through a shared data acquisition module, user behavior data, system performance data, data sharing efficiency data and network security data of each sub-data area are comprehensively collected, ensuring the integrity and diversity of the data. These data provide a solid foundation for subsequent analysis and early warning, and help to discover potential problems or optimization points; through the shared data analysis module, user behavior data analysis, system performance data analysis, data sharing efficiency data analysis and network security data analysis models are established. This module can deeply mine the collected data and calculate the characteristic values ​​and efficiency values ​​of each sub-data area. These analysis results provide specific quantitative indicators for subsequent comprehensive analysis, improving the accuracy and pertinence of data analysis; through the comprehensive analysis module, the analysis results of each sub-data area are integrated, a comprehensive analysis model is established, and the comprehensive abnormality index of the target data area is calculated. This index fully reflects the overall situation of data aggregation and sharing management, provides managers with an intuitive and comprehensive data overview, and helps to quickly discover potential problems or risk points; through the early warning module, the preset value of the comprehensive abnormality index is set, and the comprehensive abnormality index of the target data area is monitored and judged in real time. Once an abnormality is found, an early warning signal is immediately sent to the manager terminal, so that the manager can respond quickly and take measures. This function effectively improves the security and reliability of data aggregation and sharing management, and reduces the probability of potential risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] refer to Figure 1 A data aggregation and sharing management system based on big data is shown, which includes a time division module, a shared data acquisition module, a shared data analysis module, a comprehensive analysis module, and an early warning module.

[0025] The time area division module is used to determine the target enterprise's data aggregation shared management data as the target data area, divide the target data area into various sub-data areas by equal time division, and mark them as 1, 2...n in sequence.

[0026] The shared data collection module is used to collect user behavior data, system performance data, data sharing efficiency data, and network security data of each sub-data area, and transmit the collected data to the shared data analysis module.

[0027] The user behavior data include page dwell time, click frequency, page jump rate, and user churn rate, which are marked as Ut, Uf, Uj, and Ul respectively; the system performance data include response time, maximum number of parallel requests, resource utilization, and error rate, which are marked as Pt, Pb, Pe, and Pw respectively; the data sharing efficiency data include data transmission rate, data integrity rate, data delay, and data access frequency, which are marked as Ev, Ec, El, and Ef respectively; the network security data include intrusion detection rate, malware interception rate, encrypted data ratio, and vulnerability repair time, which are marked as Cd, Cm, Ce, and Ct respectively.

[0028] The shared data analysis module includes a user behavior data analysis unit, a system performance data analysis unit, a data sharing efficiency data analysis unit, and a network security data analysis unit, which are used to analyze the data transmitted by the shared data acquisition module and transmit the analysis results to the comprehensive analysis module.

[0029] The user behavior data analysis unit is used to establish a user behavior data analysis model, import the user behavior data transmitted by the shared data acquisition module into the user behavior data analysis model, and calculate the user behavior feature value of each sub-data area, which is specifically expressed as: SU i Indicates the user behavior characteristic value of the ith sub-data area, Ut i Indicates the page residence time of the i-th sub-data area, Uf i Indicates the click frequency of the ith sub-data area, Uj i represents the page jump rate of the ith sub-data area, Ul i represents the user churn rate of the i-th sub-data area, and μ1 represents other influencing factors of the customer behavior characteristic value.

[0030] The system performance data analysis unit is used to establish a system performance data analysis model, import the system performance data transmitted by the shared data acquisition module into the system performance data analysis model, and calculate the system performance characteristic value of each sub-data area, which is specifically expressed as: SP i represents the system performance characteristic value of the ith sub-data area, Pt i represents the response time of the ith sub-data area, Pb i represents the maximum number of parallel requests for the ith sub-data area, Pe i represents the resource utilization of the ith sub-data area, Pw i represents the error rate of the ith sub-data region, Pb max represents the maximum number of parallel requests for the target data area, and μ2 represents other influencing factors of the system performance characteristic value.

[0031] The data sharing efficiency data analysis unit is used to establish a data sharing efficiency data analysis model, import the data sharing efficiency data transmitted by the shared data acquisition module into the data sharing efficiency data analysis model, and calculate the data sharing efficiency value of each sub-data area, which is specifically expressed as: SE i represents the data sharing efficiency value of the i-th sub-data area, Ev i represents the data transmission rate of the ith sub-data area, Ec i represents the data integrity rate of the ith sub-data area, El i represents the data delay of the ith sub-data area, Ef i represents the data access frequency of the i-th sub-data area, and μ3 represents other influencing factors of the data sharing efficiency value.

[0032] The network security data analysis unit is used to establish a network security data analysis model, import the network security data transmitted by the shared data acquisition module into the network security data analysis model, and calculate the network security efficiency value of each sub-data area, which is specifically expressed as: SC i represents the network security effectiveness value of the ith sub-data area, Cd i represents the intrusion detection rate of the ith sub-data area, Cm i represents the malware interception rate of the ith sub-data area, Ce i Indicates the proportion of encrypted data in the ith sub-data area, Ct i represents the vulnerability repair time of the i-th sub-data area, and μ4 represents other influencing factors of the network security effectiveness value.

[0033] The comprehensive analysis module is used to establish a comprehensive analysis model, import the data transmitted by the shared data analysis module into the comprehensive analysis model, calculate the comprehensive abnormality index of the target data area, and transmit it to the early warning module.

[0034] The comprehensive analysis model is specifically expressed as: η represents the comprehensive abnormality index of the target data area, SU i represents the user behavior feature value of the ith sub-data area, SP i represents the system performance characteristic value of the ith sub-data area, SE i represents the data sharing efficiency value of the i-th sub-data area, SC i represents the network security effectiveness value of the ith sub-data area, and λ represents other influencing factors of the comprehensive anomaly index.

[0035] The early warning module is used to establish a preset value of the comprehensive abnormality index, judge the comprehensive abnormality index of the target data area through the preset value of the comprehensive abnormality index, and issue an early warning signal according to the judgment result.

[0036] The preset value of the comprehensive abnormality index is marked as η Def , when η Def >η, it means that the comprehensive anomaly index of the target data area is less than the preset value of the comprehensive anomaly index, indicating that the data aggregation and sharing quality of the target enterprise is good. Def When <η, it means that the comprehensive abnormality index of the target data area is greater than the preset value of the comprehensive abnormality index, indicating that the data aggregation and sharing quality of the target enterprise is poor, and an early warning signal is sent to the management terminal.

[0037] The present invention divides the target enterprise's data aggregation and shared management data into sub-data areas of equal time periods through a time division module, which helps to more finely monitor and analyze data changes in different time periods, thereby more accurately identifying potential problems or trends and improving the timeliness and accuracy of data management; through a shared data acquisition module, user behavior data, system performance data, data sharing efficiency data and network security data of each sub-data area are comprehensively collected, ensuring the integrity and diversity of the data. These data provide a solid foundation for subsequent analysis and early warning, and help to discover potential problems or optimization points; through the shared data analysis module, user behavior data analysis, system performance data analysis, data sharing efficiency data analysis and network security data analysis models are established. This module can deeply mine the collected data and calculate the characteristic values ​​and efficiency values ​​of each sub-data area. These analysis results provide specific quantitative indicators for subsequent comprehensive analysis, improving the accuracy and pertinence of data analysis; through the comprehensive analysis module, the analysis results of each sub-data area are integrated, a comprehensive analysis model is established, and the comprehensive abnormality index of the target data area is calculated. This index fully reflects the overall situation of data aggregation and sharing management, provides managers with an intuitive and comprehensive data overview, and helps to quickly discover potential problems or risk points; through the early warning module, the preset value of the comprehensive abnormality index is set, and the comprehensive abnormality index of the target data area is monitored and judged in real time. Once an abnormality is found, an early warning signal is immediately sent to the manager terminal, so that the manager can respond quickly and take measures. This function effectively improves the security and reliability of data aggregation and sharing management, and reduces the probability of potential risks.

[0038] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other;

[0039] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data aggregation and sharing management system based on big data, characterized in that: include: Time zone division module: used to determine the target enterprise's data aggregation and shared management data as the target data zone, divide the target data zone into sub-data zones by equal time division, and mark them as 1, 2...n in sequence; Shared data collection module: used to collect user behavior data, system performance data, data sharing efficiency data, and network security data of each sub-data area, and transmit the collected data to the shared data analysis module; Shared data analysis module: including user behavior data analysis unit, system performance data analysis unit, data sharing efficiency data analysis unit, and network security data analysis unit, used to analyze the data transmitted by the shared data acquisition module and transmit the analysis results to the comprehensive analysis module; Comprehensive analysis module: used to establish a comprehensive analysis model, import the data transmitted by the shared data analysis module into the comprehensive analysis model, calculate the comprehensive abnormal index of the target data area, and transmit it to the early warning module; Early warning module: used to establish a preset value of the comprehensive abnormality index, judge the comprehensive abnormality index of the target data area through the preset value of the comprehensive abnormality index, and issue an early warning signal based on the judgment result.

2. According to the big data-based data aggregation and sharing management system of claim 1, it is characterized by: The user behavior data include page dwell time, click frequency, page jump rate, and user churn rate, which are marked as Ut, Uf, Uj, and Ul respectively; the system performance data include response time, maximum number of parallel requests, resource utilization, and error rate, which are marked as Pt, Pb, Pe, and Pw respectively; the data sharing efficiency data include data transmission rate, data integrity rate, data delay, and data access frequency, which are marked as Ev, Ec, El, and Ef respectively; the network security data include intrusion detection rate, malware interception rate, encrypted data ratio, and vulnerability repair time, which are marked as Cd, Cm, Ce, and Ct respectively.

3. The data aggregation and sharing management system based on big data according to claim 1 is characterized by: The user behavior data analysis unit is used to establish a user behavior data analysis model, import the user behavior data transmitted by the shared data acquisition module into the user behavior data analysis model, and calculate the user behavior feature value of each sub-data area, which is specifically expressed as: SU i Indicates the user behavior characteristic value of the ith sub-data area, Ut i Indicates the page residence time of the i-th sub-data area, Uf i Indicates the click frequency of the ith sub-data area, Uj i represents the page jump rate of the ith sub-data area, Ul i represents the user churn rate of the i-th sub-data area, and μ1 represents other influencing factors of the user behavior feature value.

4. The data aggregation and sharing management system based on big data according to claim 1 is characterized by: The system performance data analysis unit is used to establish a system performance data analysis model, import the system performance data transmitted by the shared data acquisition module into the system performance data analysis model, and calculate the system performance characteristic value of each sub-data area, which is specifically expressed as: SP i represents the system performance characteristic value of the ith sub-data area, Pt i represents the response time of the ith sub-data area, Pb i represents the maximum number of parallel requests for the ith sub-data area, Pe i represents the resource utilization of the ith sub-data area, Pw i represents the error rate of the ith sub-data region, Pb max represents the maximum number of parallel requests for the target data area, and μ2 represents other influencing factors of the system performance characteristic value.

5. The data aggregation and sharing management system based on big data according to claim 1 is characterized by: The data sharing efficiency data analysis unit is used to establish a data sharing efficiency data analysis model, import the data sharing efficiency data transmitted by the shared data acquisition module into the data sharing efficiency data analysis model, and calculate the data sharing efficiency value of each sub-data area, which is specifically expressed as: SE i represents the data sharing efficiency value of the i-th sub-data area, Ev i represents the data transmission rate of the ith sub-data area, Ec i represents the data integrity rate of the ith sub-data area, El i represents the data delay of the ith sub-data area, Ef i represents the data access frequency of the i-th sub-data area, and μ3 represents other influencing factors of the data sharing efficiency value.

6. The data aggregation and sharing management system based on big data according to claim 1 is characterized by: The network security data analysis unit is used to establish a network security data analysis model, import the network security data transmitted by the shared data acquisition module into the network security data analysis model, and calculate the network security efficiency value of each sub-data area, which is specifically expressed as: SC i represents the network security effectiveness value of the ith sub-data area, Cd i represents the intrusion detection rate of the ith sub-data area, Cm i represents the malware interception rate of the ith sub-data area, Ce i Indicates the proportion of encrypted data in the ith sub-data area, Ct i represents the vulnerability repair time of the i-th sub-data area, and μ4 represents other influencing factors of the network security effectiveness value.

7. The data aggregation and sharing management system based on big data according to claim 1 is characterized by: The comprehensive analysis model is specifically expressed as: η represents the comprehensive abnormality index of the target data area, SU i represents the user behavior feature value of the ith sub-data area, SP i represents the system performance characteristic value of the ith sub-data area, SE i represents the data sharing efficiency value of the i-th sub-data area, SC i represents the network security effectiveness value of the ith sub-data area, and λ represents other influencing factors of the comprehensive anomaly index.

8. The data aggregation and sharing management system based on big data according to claim 1 is characterized by: The preset value of the comprehensive abnormality index is marked as η Def , when η Def >η, it means that the comprehensive anomaly index of the target data area is less than the preset value of the comprehensive anomaly index, indicating that the data aggregation and sharing quality of the target enterprise is good. Def When <η, it means that the comprehensive abnormality index of the target data area is greater than the preset value of the comprehensive abnormality index, indicating that the data aggregation and sharing quality of the target enterprise is poor, and an early warning signal is sent to the management terminal.

Images