Shared data governance platform

Abstract

The present invention belongs to the field of data governance technology. The present invention provides a shared data governance platform. By analyzing the data after governance by the governance platform, it is helpful to identify the data governance capability of the governance platform, analyze the degree of difference between the current actual shared data and the historical governance data, and evaluate the difference between the actual shared data information and the historical governance data, thereby improving the accuracy of evaluating the difference between the actual data and the historical governance data, which is conducive to adjusting the data governance plan of the governance platform, and then classifying and processing the memory overlimit group and the feature non-overlapping group, solving the problem of low efficiency of the governance platform in governing diverse shared data, and then improving the governance efficiency of the governance platform in governing diverse data by adjusting the priority of the data governance platform.

Description

Technical Field

[0001] The present invention belongs to the technical field of data management, and specifically relates to a shared data management platform. Background Art

[0002] Data governance platforms usually use data middle-office technology and microservice architecture to provide data services for data resource centers and external data systems. At the same time, data governance platforms are also data quality inspection tools that support data comparison, duplicate similarity checks and other operations.

[0003] In the prior art, a data governance platform is usually used to govern shared data from different sources. However, there is a lack of systematic evaluation of the ability and characteristics of the governance platform to process data within the historical receiving cycle. For example, the diversity of receiving features in historical data, memory usage, etc. are not effectively analyzed, making it difficult to judge the limitations or diversity of the governance platform when processing data. Moreover, in the actual data governance process, the matching evaluation between the actual shared data and the governance type of the governance platform is also ignored, which may cause the governance platform to be inefficient when processing certain types of data, or even unable to work properly. Therefore, this application helps to identify the data governance capabilities of the governance platform by analyzing the data after governance by the governance platform, and analyzes the degree of difference between the current actual shared data and the historical governance data, and evaluates the difference between the actual shared data information and the historical governance data, thereby improving the accuracy of evaluating the difference between the actual data and the historical governance data, and is conducive to adjusting the data governance plan of the governance platform. Then, the memory overlimit group and the feature non-overlapping group are classified and processed to solve the problem of inefficiency of the governance platform in governing diverse shared data, and then by adjusting the priority of the data governance platform to govern the data, the governance efficiency of the governance platform in governing diverse data is improved.

[0004] To this end, the present invention provides a shared data governance platform. Summary of the invention

[0005] In order to make up for the deficiencies of the prior art, at least one technical problem raised in the background technology is solved.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a shared data governance platform, including:

[0007] Historical data collection module: obtains the historical reception information of the governance platform within the historical reception cycle, analyzes and processes the historical reception information, obtains the reception characteristic value, and obtains the historical governance table based on the reception characteristic value;

[0008] Historical data analysis module: Based on the historical governance table, multiple analysis data groups in the historical governance table are analyzed to obtain historical governance data, where the historical governance data includes type proportion and memory proportion. The historical governance data is analyzed and processed to obtain the platform governance value, and the platform governance value is compared with the platform governance threshold. If the platform governance value is greater than the platform governance threshold, a governance limitation signal is generated;

[0009] Actual data analysis module: Based on the governance limitation signal, the actual shared data information is extracted through the actual data log, where the actual shared data information includes the memory difference value and the type difference value. The actual shared data information is analyzed and processed with the historical governance data to obtain the data difference value, which is compared with the data difference threshold. If the data difference value is less than the data difference threshold, a governance optimization signal is generated;

[0010] Governance plan optimization module: Based on the governance optimization signal, the governance plan is adjusted to complete the governance of the actual shared data.

[0011] As a further solution of the present invention, the method of obtaining the historical management table is:

[0012] During the historical receiving cycle, the shared data on the data governance platform is extracted and classified and summarized. The process is as follows:

[0013] Summarizing the same reception characteristic data into a group of analysis data sets to obtain a plurality of analysis data sets, wherein a group of analysis data sets is composed of at least two or more same reception characteristic data sets;

[0014] Obtain the number of identical reception features in the analysis data group, calculate the ratio of the number of identical reception features in the analysis data group to the total number of shared data in the target analysis period, and obtain the reception features;

[0015] The receiving characteristic values ​​corresponding to the multiple analysis data groups are compared in size and sorted from large to small to obtain a historical governance table.

[0016] As a further solution of the present invention, the platform governance value is obtained in the following manner:

[0017] The proportion of types and memory usage Substituting into the formula: , calculate the platform governance value ,in, , Expressed as a preset scaling factor.

[0018] As a further solution of the present invention, the method of obtaining historical governance data is:

[0019] In the historical governance table, obtain the number of identical reception characteristics in the analysis data group, and calculate the ratio with the total number of reception characteristics in the historical reception information to obtain the unit group quantity ratio;

[0020] Add the number of unit groups corresponding to the analysis data groups in the historical governance table and take the average to get the type ratio, which is marked as ;

[0021] Obtain the memory of the shared data corresponding to the same receiving characteristics in the analysis data group, and compare the size of the memory of the shared data corresponding to the same receiving characteristics to obtain the maximum memory and the minimum memory in the historical group, and make a difference between the maximum memory and the minimum memory in the historical group, and calculate the ratio with the memory of the shared data governance platform to obtain the unit memory range difference;

[0022] Add the unit memory range differences corresponding to the analysis data groups in the historical management table and take the average to get the memory share, which is marked as .

[0023] As a further solution of the present invention, the data difference value is obtained in the following manner:

[0024] Type difference value Difference value from memory Substituting into the formula: , calculate the data difference value ,in, , Expressed as a preset scaling factor.

[0025] As a further solution of the present invention, the memory difference value is extracted in the following manner:

[0026] The actual shared data is classified and summarized according to data features, and the actual shared data with the same data features is used as a comparison data group to obtain multiple comparison data groups;

[0027] Randomly extract a comparison data group from multiple comparison data groups for analysis. The process is as follows:

[0028] Obtain the memory corresponding to the actual shared data in the comparison data group, and compare the size of the memory corresponding to the actual shared data to obtain the maximum memory in the group;

[0029] The process of analyzing historical governance data is as follows:

[0030] Obtain the maximum memory and minimum memory in the historical group corresponding to the analysis data group, compare the maximum memory in the historical groups corresponding to multiple analysis data groups, extract the maximum memory in the largest historical group, and mark it as the historical maximum memory;

[0031] Compare the maximum memory in the group with the historical maximum memory. If the maximum memory in the group is greater than the historical maximum memory, mark the comparison data group with actual shared data that has memory exceeding the historical maximum memory as a memory limit exceeded group.

[0032] Count the number of memory overlimit groups, and calculate the ratio of the number of memory overlimit groups to the total number of comparison data groups to obtain the memory overlimit number;

[0033] Extract the actual shared data that exceeds the historical maximum memory in the memory overlimit group, obtain the memory of the actual shared data that exceeds the historical maximum memory, subtract it from the historical maximum memory, and calculate the ratio with the historical maximum memory to obtain the memory excess value;

[0034] The memory excess values ​​corresponding to all memory excess groups are added together and averaged to obtain the excess degree value;

[0035] The memory difference value is obtained by multiplying the memory limit amount and the degree of excess value, and is marked as .

[0036] As a further solution of the present invention, the type difference value is obtained in the following manner:

[0037] Integrate the data features corresponding to multiple comparison data groups to obtain a comparison feature set ,in, It is represented as the data feature corresponding to the nth comparison data set, and n is represented as the total number of comparison data sets;

[0038] Integrate the corresponding receiving features of multiple analysis data groups to obtain the analysis feature set ,in, It is represented as the receiving feature corresponding to the mth analysis data group, and m represents the total number of analysis data groups;

[0039] Perform feature overlap comparison between the elements in the comparison feature set R and the analysis feature set T. If the data features of the elements in the comparison feature set R do not overlap with the received features of the elements in the analysis feature set T, mark the data features of the elements in the comparison feature set R as non-overlapping features, and mark the comparison data group corresponding to the non-overlapping features as a feature non-overlapping group again;

[0040] Count the number of non-overlapping feature groups, and compare the total number of data groups to calculate the ratio, get the type difference value, and mark it as .

[0041] As a further solution of the present invention, the determination process of classification discussion is as follows:

[0042] Compare the memory overrun group with the feature overlap group. If the memory overrun group and the feature overlap group are the same comparison data group, they are marked as the same overrun overlap group.

[0043] If the memory over-limit group and the feature overlap group are not the same comparison data group, they are marked as different over-limit overlap groups;

[0044] The memory overrun group and the feature non-overlap group are compared for overlap. If the memory overrun group and the feature non-overlap group are the same comparison data group, they are marked as the same non-feature overrun group;

[0045] If the memory limit exceeding group and the feature non-overlapping group are not the same comparison analysis group, they are marked as different non-feature limit exceeding groups.

[0046] As a further solution of the present invention, the treatment scheme for the same over-limit overlap group and the different over-limit overlap groups is adjusted, and the process is as follows:

[0047] The actual shared data in the same over-limit overlap group with memory exceeding the historical maximum memory is compressed, and governance operations are performed after it is compressed to less than or equal to the historical maximum memory;

[0048] Prioritize governance operations on the data within the feature overlap group, and then perform memory compression on the actual shared data within the memory limit excess group whose memory exceeds the historical maximum memory. Governance operations are performed after compression to a value less than or equal to the historical maximum memory.

[0049] As a further solution of the present invention, the treatment scheme for the same non-feature over-limit group and the different non-feature over-limit groups is adjusted, and the process is as follows:

[0050] Compare the memory excess values ​​in the non-feature excess limit group and sort them in ascending order to obtain a priority management table;

[0051] According to the order in the priority management table, the actual shared data in the feature overlimit group that exceeds the historical maximum memory is compressed one by one to be less than or equal to the historical maximum memory, and the compressed actual shared data is converted into the receiving features corresponding to the analysis data group in the historical management table;

[0052] Prioritize the memory overlimit group, perform memory compression on the actual shared data in the memory overlimit group that exceeds the historical maximum memory, compress it to a level less than or equal to the historical maximum memory, perform data governance operations, and then convert the feature non-overlapping group into the receiving features corresponding to the analysis data group in the historical governance table, and perform data governance operations.

[0053] The beneficial effects of the present invention are as follows:

[0054] (1) The present invention analyzes the receiving characteristics corresponding to the historical data managed by the data governance platform, summarizes and integrates the historical data with the same receiving characteristics, obtains the historical governance table, analyzes the data in the historical governance table, and obtains the platform governance value. The platform governance value reflects the data type processed by the governance platform in the historical period and the memory size of the data, which helps to identify the data governance capabilities of the governance platform;

[0055] (2) Based on the governance limitation signal, the present invention extracts the actual shared data information through the actual data log, and compares and analyzes it with the historical governance data to obtain the data difference value. The data difference value comprehensively reflects the degree of difference between the actual shared data and the historical governance data, and evaluates the difference between the actual shared data information and the historical governance data, thereby improving the accuracy of evaluating the difference between the actual data and the historical governance data, which is conducive to adjusting the data governance plan of the governance platform;

[0056] (3) The present invention analyzes the data in the memory limit excess group, feature overlap group and feature non-overlap group through classified discussion, and improves the governance strategy of the analysis results in a targeted manner, thereby solving the problem of low efficiency of the governance platform in managing diverse shared data, and then improving the governance efficiency of the governance platform in managing diverse data by adjusting the priority of data governance on the data governance platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention will be further described below in conjunction with the accompanying drawings.

[0058] Figure 1 It is a module composition diagram of the shared data governance platform of the present invention;

[0059] Figure 2 It is a flow chart of the steps in the shared data governance platform of the present invention. DETAILED DESCRIPTION

[0060] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0061] Example 1

[0062] like Figure 1 - Figure 2 As shown, the shared data governance platform described in the embodiment of the present invention includes:

[0063] Historical data collection module: obtains historical receiving information of the governance platform within the historical receiving cycle, wherein the historical receiving information includes receiving characteristics, analyzes and processes the receiving characteristics to obtain receiving characteristic values, and obtains historical governance tables based on the receiving characteristic values;

[0064] Among them, receiving characteristics include but are not limited to receiving format, receiving memory, etc.;

[0065] Specifically, during the historical receiving cycle, the shared data on the data governance platform is extracted and classified and summarized. The process is as follows:

[0066] It should be noted that the same type of induction is performed by inductively aggregating data with the same receiving characteristics;

[0067] Summarizing the same reception characteristic data into a group of analysis data sets to obtain a plurality of analysis data sets, wherein a group of analysis data sets is composed of at least two or more same reception characteristic data sets;

[0068] Obtain the number of identical reception features in the analysis data group, calculate the ratio of the number of identical reception features in the analysis data group to the total number of shared data in the target analysis period, and obtain the reception feature value;

[0069] Compare the corresponding receiving characteristic values ​​of multiple analysis data groups and sort them from large to small to obtain a historical governance table;

[0070] Historical data analysis module: Based on the historical governance table, multiple analysis data groups in the historical governance table are analyzed to obtain historical governance data, where the historical governance data includes type proportion and memory proportion. The historical governance data is analyzed and processed to obtain the platform governance value, and the platform governance value is compared with the platform governance threshold to generate a governance type signal;

[0071] Among them, governance type signals include governance diversity signals or governance limitation signals;

[0072] It should be noted that in the historical governance table, the reception characteristics in different analysis data groups are different, but the target reception characteristics in the analysis data groups are the same characteristics;

[0073] In the historical governance table, obtain the number of identical reception characteristics in the analysis data group, and calculate the ratio with the total number of reception characteristics in the historical reception information to obtain the unit group quantity ratio;

[0074] Add the number of unit groups corresponding to the analysis data groups in the historical governance table and take the average to get the type ratio, which is marked as ;

[0075] Obtain the memory of the shared data corresponding to the same receiving characteristics in the analysis data group, and compare the size of the memory of the shared data corresponding to the same receiving characteristics to obtain the maximum memory and the minimum memory in the historical group, and make a difference between the maximum memory and the minimum memory in the historical group, and calculate the ratio with the memory of the shared data governance platform to obtain the unit memory range difference;

[0076] Add the unit memory range differences corresponding to the analysis data groups in the historical management table and take the average to get the memory share, which is marked as ;

[0077] The proportion of types and memory usage Substituting into the formula: , calculate the platform governance value ,in, , is represented by a preset scaling factor, and The value is 2.347. The value is 6.516;

[0078] It can be understood that the meaning of the platform governance value is: reflecting the data type processed by the governance platform in the historical period and the memory size of the data. Specifically, if the platform governance value is larger, it means that the data type processed by the governance platform in the historical period is single, and the memory range of the processed data is smaller. On the contrary, if this value is smaller, it means that the data type processed by the governance platform in the historical period is diversified, and the memory size of the corresponding data is diversified.

[0079] Compare the platform governance value to the platform governance threshold as follows:

[0080] If the platform governance value is greater than the platform governance threshold, it means that the data type processed by the governance platform in the historical period is uniform, and the memory range of the processed data is small, generating a governance limitation signal;

[0081] If the platform governance value is less than or equal to the platform governance threshold, it means that the data types processed by the governance platform in the historical period are diversified, and the memory size of the processed data is diversified, generating a governance diversity signal;

[0082] The specific implementation method of the embodiment of the present invention is: by analyzing the receiving characteristics corresponding to the historical data governed by the data governance platform, and summarizing and integrating the historical data with the same receiving characteristics to obtain a historical governance table, analyzing the data in the historical governance table to obtain a platform governance value, so that the platform governance value reflects the type of data processed by the governance platform in the historical period and the memory size of the data, which helps to identify the data governance capabilities of the governance platform, and improve the governance strategy in a targeted manner based on the identified data governance capabilities.

[0083] Example 2

[0084] like Figure 1 - Figure 2As shown, based on Example 1, the shared data governance platform described in this embodiment of the present invention includes:

[0085] Actual data analysis module: Based on the governance limitation signal, the actual shared data information is extracted through the actual data log, where the actual shared data information includes the memory difference value and the type difference value. The actual shared data information is analyzed and processed with the historical governance data to obtain the data difference value, which is compared with the data difference threshold. If the data difference value is less than the data difference threshold, a governance optimization signal is generated;

[0086] In some embodiments, the actual shared data is classified and summarized according to data features, and the actual shared data with the same data features is used as a comparison data group to obtain multiple comparison data groups;

[0087] It should be noted that the data characteristics of the actual shared data in each comparison data group are the same, and the comparison data group is composed of at least two or more actual shared data, but the data characteristics of different comparison data groups are different;

[0088] Randomly extract a comparison data group from multiple comparison data groups for analysis. The process is as follows:

[0089] Obtain the memory corresponding to the actual shared data in the comparison data group, and compare the size of the memory corresponding to the actual shared data to obtain the maximum memory in the group;

[0090] The process of analyzing historical governance data is as follows:

[0091] Obtain the maximum memory in the historical group corresponding to the analysis data group, compare the maximum memory in the historical groups corresponding to multiple analysis data groups, extract the maximum memory in the largest historical group, and mark it as the historical maximum memory;

[0092] Compare the maximum memory in the group with the historical maximum memory. The process is as follows:

[0093] If the maximum memory in the group is greater than the historical maximum memory, that is, there is actual shared data with memory exceeding the historical maximum memory in the comparison data group, the comparison data group with actual shared data with memory exceeding the historical maximum memory will be marked as a memory limit exceeded group;

[0094] If the maximum memory in the group is less than or equal to the historical maximum memory, that is, the actual shared data memory in the comparison data group is less than the historical maximum memory, and it is marked as the memory does not exceed the group;

[0095] Count the number of memory overlimit groups, and calculate the ratio of the number of memory overlimit groups to the total number of comparison data groups to obtain the memory overlimit number;

[0096] Extract the actual shared data that exceeds the historical memory range in the memory limit excess group, obtain the memory of the actual shared data that exceeds the historical memory range, subtract it from the historical maximum memory, and calculate the ratio with the historical maximum memory to obtain the memory excess value;

[0097] The memory excess values ​​corresponding to all memory excess groups are added together and averaged to obtain the excess degree value;

[0098] The memory difference value is obtained by multiplying the memory limit amount and the degree of excess value, and is marked as ;

[0099] Integrate the data features corresponding to multiple comparison data groups to obtain a comparison feature set ,in, It is represented as the data feature corresponding to the nth comparison data set, and n is represented as the total number of comparison data sets;

[0100] Integrate the corresponding receiving features of multiple analysis data groups to obtain the analysis feature set ,in, It is represented as the receiving feature corresponding to the mth analysis data group, and m represents the total number of analysis data groups;

[0101] The feature overlap comparison is performed from the elements in the comparison feature set R and the analysis feature set T. The process is as follows:

[0102] If the data features of the elements in the comparison feature set R coincide with the received features of the elements in the analysis feature set T, the data features of the elements in the comparison feature set R are marked as coincident features, and the comparison data group corresponding to the coincident features is marked again as a feature coincidence group;

[0103] If the data features of the elements in the comparison feature set R do not overlap with the received features of the elements in the analysis feature set T, the data features of the elements in the comparison feature set R are marked as non-overlapping features, and the comparison data group corresponding to the non-overlapping features is marked again as a feature non-overlapping group;

[0104] Count the number of non-overlapping feature groups, and compare the total number of data groups to calculate the ratio, get the type difference value, and mark it as ;

[0105] Type difference value Difference value from memory Substituting into the formula: , calculate the data difference value ,in, , is represented by a preset scaling factor, and The value is 5.647. The value is 2.741;

[0106] It can be understood that the meaning of the data difference value is: the type difference value reflects the degree of difference in type between the actual shared data and the historical governance data, and the memory difference value reflects the degree of difference in memory between the actual shared data and the historical governance data, which comprehensively reflects the degree of difference between the actual shared data and the historical governance data. Specifically, if this value is larger, it means that the degree of difference in type and the degree of difference in memory between the actual shared data and the historical governance data are smaller. Conversely, if this value is smaller, it means that the degree of difference in type and the degree of difference in memory between the actual shared data and the historical governance data are larger.

[0107] The data difference value is compared with the data difference threshold value, and the process is as follows:

[0108] If the data difference value is greater than or equal to the data difference threshold, it means that the difference between the actual shared data and the historical governance data in type and the smaller the difference in memory, the non-governance optimization signal is generated;

[0109] If the data difference value is less than the data difference threshold, it means that the difference between the actual shared data and the historical governance data in type and in memory is greater, and a governance optimization signal is generated;

[0110] The specific implementation method of the embodiment of the present invention is: based on the governance limitation signal, the actual shared data information is extracted through the actual data log, and compared and analyzed with the historical governance data to obtain the data difference value, and the data difference value is used to comprehensively reflect the degree of difference between the actual shared data and the historical governance data, and the difference between the actual shared data information and the historical governance data is evaluated, thereby improving the accuracy of evaluating the difference between the actual data and the historical governance data, which is conducive to adjusting the data governance plan of the governance platform.

[0111] Example 3

[0112] like Figure 1 - Figure 2 As shown, based on Example 1 and Example 2, the shared data governance platform described in the embodiment of the present invention includes:

[0113] Governance plan optimization module: Based on governance optimization signals, the module analyzes the data in the memory overlimit group, feature overlap group, and feature non-overlap group through classified discussions, and adjusts the governance plan to complete the governance of the actual shared data;

[0114] Specifically, the process of formulating a governance plan is as follows:

[0115] Compare the memory overrun group with the feature overlap group. The process is as follows:

[0116] If the memory-exceeding group and the feature-overlapping group are the same comparison data group, they will be marked as the same-exceeding-overlapping group, and the actual shared data in the same-exceeding-overlapping group whose memory exceeds the historical maximum memory will be compressed, and governance operations will be performed after the data is compressed to a value less than or equal to the historical maximum memory.

[0117] If the memory over-limit group and the feature overlap group are not the same comparison data group, they are marked as different over-limit overlap groups, and the data in the feature overlap group is first managed, and then the actual shared data in the memory over-limit group that exceeds the historical maximum memory is compressed, and the management operation is performed after it is compressed to less than or equal to the historical maximum memory;

[0118] The memory overrun group and the feature non-overlap group are compared for overlap. The process is as follows:

[0119] If the memory overrun group and the feature non-overlap group are the same comparison data group, they are marked as the same non-feature overrun group;

[0120] Based on the same or non-feature over-limit groups, the memory over-limit values ​​in the same or non-feature over-limit groups are compared and sorted in ascending order to obtain a priority management table;

[0121] According to the order in the priority management table, the actual shared data in the feature overlimit group that exceeds the historical maximum memory is compressed one by one to be less than or equal to the historical maximum memory, and the compressed actual shared data is converted into the receiving features corresponding to the analysis data group in the historical management table;

[0122] If the memory limit-exceeding group and the feature non-overlapping group are not the same comparison analysis group, they are marked as different non-feature limit-exceeding groups;

[0123] Prioritize the memory overlimit group, compress the actual shared data in the memory overlimit group that exceeds the historical maximum memory to a size less than or equal to the historical maximum memory, perform data governance operations, and then convert the feature non-overlapping group into the corresponding receiving feature of the analysis data group in the historical governance table, and perform data governance operations;

[0124] The specific implementation plan of the embodiment of the present invention is: analyze the data in the memory limit excess group, feature overlap group and feature non-overlap group through classified discussion, and improve the governance strategy of the analysis results in a targeted manner, so as to solve the problem of low efficiency of the governance platform in governing diverse shared data, and then improve the governance efficiency of the governance platform in governing diverse data by adjusting the priority of data governance on the data governance platform.

[0125] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. Shared data governance platform, characterized by: include: Historical data collection module: obtains the historical reception information of the governance platform within the historical reception cycle, analyzes and processes the historical reception information, obtains the reception characteristic value, and obtains the historical governance table based on the reception characteristic value; During the historical receiving cycle, the shared data on the data governance platform is extracted and classified and summarized. The process is as follows: Summarizing the same reception characteristic data into a group of analysis data sets to obtain a plurality of analysis data sets, wherein a group of analysis data sets is composed of at least two or more same reception characteristic data sets; Obtain the number of identical reception features in the analysis data group, calculate the ratio of the number of identical reception features in the analysis data group to the total number of shared data in the target analysis period, and obtain the reception features; Compare the corresponding receiving characteristic values ​​of multiple analysis data groups and sort them from large to small to obtain a historical governance table; Historical data analysis module: Based on the historical governance table, multiple analysis data groups in the historical governance table are analyzed to obtain historical governance data, where the historical governance data includes type proportion and memory proportion. The historical governance data is analyzed and processed to obtain the platform governance value, and the platform governance value is compared with the platform governance threshold. If the platform governance value is greater than the platform governance threshold, a governance limitation signal is generated; The proportion of types and memory usage Substituting into the formula: , calculate the platform governance value ,in, , Expressed as a preset scaling factor; Actual data analysis module: Based on the governance limitation signal, the actual shared data information is extracted through the actual data log, where the actual shared data information includes the memory difference value and the type difference value. The actual shared data information is analyzed and processed with the historical governance data to obtain the data difference value, which is compared with the data difference threshold. If the data difference value is less than the data difference threshold, a governance optimization signal is generated; Governance plan optimization module: Based on the governance optimization signal, the governance plan is adjusted to complete the governance of the actual shared data.

2. The shared data governance platform according to claim 1, characterized in that: The historical governance data is obtained as follows: In the historical governance table, obtain the number of identical reception characteristics in the analysis data group, and calculate the ratio with the total number of reception characteristics in the historical reception information to obtain the unit group quantity ratio; Add the number of unit groups corresponding to the analysis data groups in the historical governance table and take the average to get the type ratio, which is marked as ; Obtain the memory of the shared data corresponding to the same receiving characteristics in the analysis data group, and compare the size of the memory of the shared data corresponding to the same receiving characteristics to obtain the maximum memory and the minimum memory in the historical group, and make a difference between the maximum memory and the minimum memory in the historical group, and calculate the ratio with the memory of the shared data governance platform to obtain the unit memory range difference; Add the unit memory range differences corresponding to the analysis data groups in the historical management table and take the average to get the memory share, which is marked as .

3. The shared data governance platform according to claim 1, characterized in that: The data difference value is obtained as follows: Type difference value Difference value from memory Substituting into the formula: , calculate the data difference value ,in, , Expressed as a preset scaling factor.

4. The shared data governance platform according to claim 3, characterized in that: The memory difference value is extracted as follows: The actual shared data is classified and summarized according to data features, and the actual shared data with the same data features is used as a comparison data group to obtain multiple comparison data groups; Randomly extract a comparison data group from multiple comparison data groups for analysis. The process is as follows: Obtain the memory corresponding to the actual shared data in the comparison data group, and compare the size of the memory corresponding to the actual shared data to obtain the maximum memory in the group; The process of analyzing historical governance data is as follows: Obtain the maximum memory and minimum memory in the historical group corresponding to the analysis data group, compare the maximum memory in the historical groups corresponding to multiple analysis data groups, extract the maximum memory in the largest historical group, and mark it as the historical maximum memory; Compare the maximum memory in the group with the historical maximum memory. If the maximum memory in the group is greater than the historical maximum memory, mark the comparison data group with actual shared data that has memory exceeding the historical maximum memory as a memory limit exceeded group. Count the number of memory overlimit groups, and calculate the ratio of the number of memory overlimit groups to the total number of comparison data groups to obtain the memory overlimit number; Extract the actual shared data that exceeds the historical maximum memory in the memory overlimit group, obtain the memory of the actual shared data that exceeds the historical maximum memory, subtract it from the historical maximum memory, and calculate the ratio with the historical maximum memory to obtain the memory excess value; The memory excess values ​​corresponding to all memory excess groups are added together and averaged to obtain the excess degree value; The memory difference value is obtained by multiplying the memory limit amount and the degree of excess value, and is marked as .

5. The shared data governance platform according to claim 3, characterized in that: The type difference value is obtained as follows: Integrate the data features corresponding to multiple comparison data groups to obtain a comparison feature set ,in, It is represented as the data feature corresponding to the nth comparison data set, and n is represented as the total number of comparison data sets; Integrate the corresponding receiving features of multiple analysis data groups to obtain the analysis feature set ,in, It is represented as the receiving feature corresponding to the mth analysis data group, and m represents the total number of analysis data groups; Perform feature overlap comparison between the elements in the comparison feature set R and the analysis feature set T. If the data features of the elements in the comparison feature set R do not overlap with the received features of the elements in the analysis feature set T, mark the data features of the elements in the comparison feature set R as non-overlapping features, and mark the comparison data group corresponding to the non-overlapping features as a feature non-overlapping group again; Count the number of non-overlapping feature groups, and compare the total number of data groups to calculate the ratio, get the type difference value, and mark it as .

6. The shared data governance platform according to claim 1, characterized in that: The decision process for classification discussion is as follows: Compare the memory over-limit group with the feature overlap group. If the memory over-limit group and the feature overlap group are the same comparison data group, they are marked as the same over-limit overlap group. If the memory over-limit group and the feature overlap group are not the same comparison data group, they are marked as different over-limit overlap groups; The memory overrun group and the feature non-overlap group are compared for overlap. If the memory overrun group and the feature non-overlap group are the same comparison data group, they are marked as the same non-feature overrun group; If the memory limit exceeding group and the feature non-overlapping group are not the same comparison analysis group, they are marked as different non-feature limit exceeding groups.

7. The shared data governance platform according to claim 6, characterized in that: The process of adjusting the treatment plan for the same over-limit overlap group and different over-limit overlap groups is as follows: The actual shared data in the same over-limit overlap group with memory exceeding the historical maximum memory is compressed, and governance operations are performed after it is compressed to less than or equal to the historical maximum memory; Prioritize governance operations on the data within the feature overlap group, and then perform memory compression on the actual shared data within the memory limit excess group whose memory exceeds the historical maximum memory. Governance operations are performed after compression to a value less than or equal to the historical maximum memory.

8. The shared data governance platform according to claim 6, characterized in that: The process of adjusting the governance plan for the same non-feature over-limit group and different non-feature over-limit groups is as follows: Compare the memory excess values ​​in the non-feature excess limit group and sort them in ascending order to obtain a priority management table; According to the order in the priority management table, the actual shared data in the feature overlimit group that exceeds the historical maximum memory is compressed one by one to be less than or equal to the historical maximum memory, and the compressed actual shared data is converted into the receiving features corresponding to the analysis data group in the historical management table; Prioritize the memory overlimit group, perform memory compression on the actual shared data in the memory overlimit group that exceeds the historical maximum memory, compress it to a level less than or equal to the historical maximum memory, perform data governance operations, and then convert the feature non-overlapping group into the receiving features corresponding to the analysis data group in the historical governance table, and perform data governance operations.

Images