Service data integration and sharing method and system and medium

Abstract

The embodiment of the invention provides a business data integration and sharing method and system and a medium, and the method comprises the steps: obtaining multi-source business data in real time based on a multi-source sensor, carrying out the preprocessing of the multi-source business data, and obtaining the preprocessed business data; acquiring configuration information of the data integration task, and generating a data integration mode based on the configuration information of the data integration task to obtain integrated data; classifying and sharing the integrated data based on the data sharing mode to obtain different types of shared data, and performing encryption processing to obtain ciphertext data; calling the ciphertext data based on the data calling instruction to obtain calling data, and decrypting the calling data to obtain shared data; through configuration analysis of the data integration task, multiple integration modes of multi-source service data are realized, the integration effect is improved, different types of sharing is performed on the integrated data through different data sharing modes, the shared data are encrypted, and secure access of the shared data is realized.

Description

Technical Field

[0001] The present application relates to the technical field of data integration, and in particular, to a method, system, and medium for business data integration and sharing. Background Art

[0002] Currently, it mainly relies on professional data implementation personnel to achieve the integration and sharing of business data through tools, and there are problems in both response time and data security. How to enable business personnel to simply and quickly complete the tasks of data integration and sharing and get rid of the dependence on third-party data implementation personnel is an urgent problem to be solved. Under the background of fierce market competition, the competition among enterprises is intensifying, and the informatization process of enterprises is also accelerating. There is an increasing need for communication and coordination within enterprise groups to improve work efficiency and thus achieve the purpose of enhancing enterprise competitiveness. The way of communication often requires information sharing, and data sharing and exchange are becoming more and more frequent. Therefore, it is necessary to integrate various heterogeneous data of group enterprises. This kind of data sharing between different systems plays a crucial role in enterprise information management and decision-making. How to integrate data from different data sources, different formats, and different qualities through logical and physical methods to meet the requirements of different departments of group enterprises for data sharing, and achieve the purpose of providing data support for enterprise decision-making and enhancing the value of data assets has become a major problem facing the information management departments of group enterprises. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a method, system, and medium for business data integration and sharing. By configuring and analyzing data integration tasks, various integration methods for multi-source business data can be realized, the integration effect can be improved, and different categories of sharing can be performed on the integrated data through different data sharing modes, and the shared data is encrypted to achieve secure access to the shared data.

[0004] The embodiments of the present application also provide a method for business data integration and sharing, including:

[0005] Based on multi-source sensors, multi-source business data is obtained in real time, and the multi-source business data is preprocessed to obtain preprocessed business data;

[0006] The configuration information of the data integration task is obtained, and a data integration method is generated based on the configuration information of the data integration task;

[0007] Based on the data integration method, data integration processing is performed on the preprocessed business data to obtain integrated data;

[0008] Based on the data sharing mode, classified sharing is performed on the integrated data to obtain different categories of shared data, and the different categories of shared data are encrypted to obtain ciphertext data;

[0009] Generate a data retrieval instruction based on the access permission, retrieve the ciphertext data according to the data retrieval instruction to obtain the retrieved data, and decrypt the retrieved data to obtain the shared data.

[0010] Optionally, in the business data integration and sharing method described in the embodiments of the present application, multi-source business data is obtained in real time based on multi-source sensors, and the multi-source business data is preprocessed to obtain preprocessed business data, specifically including:

[0011] Obtain various types of sensors, and respectively obtain different types of multi-source business data based on the various types of sensors;

[0012] Analyze the missing values of the multi-source business data to obtain the missing data, correct the missing data to obtain the corrected data, and clean the data based on the corrected data to remove duplicate data to obtain the cleaned data;

[0013] Obtain the data format of the cleaned data, and convert the data format of the cleaned data based on the standard format information to obtain the converted data corresponding to different types of multi-source business data;

[0014] Extract the data features of the converted data corresponding to different types of multi-source business data, and perform a fusion process on the data features of the converted data to obtain the fused data;

[0015] Based on the repeated analysis of the fused data, eliminate duplicate data to obtain the preprocessed business data.

[0016] Optionally, in the business data integration and sharing method described in the embodiments of the present application, obtain the configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task, specifically including:

[0017] Obtain the configuration information of the data integration task, where the configuration information of the data task includes the data source, data type, data integration condition, and data integration target;

[0018] Generate configuration parameters based on the data source, data type, data integration condition, and data integration target;

[0019] Match the data integration method based on the configuration parameters and calculate the matching degree;

[0020] Judge whether the matching degree is greater than or equal to the set matching degree threshold;

[0021] If it is greater than or equal to, select the current configuration parameters; if it is less than, generate adjustment information and optimize the current configuration parameters based on the adjustment information.

[0022] Optionally, in the business data integration and sharing method described in the embodiments of the present application, data integration processing is performed on the preprocessed business data based on the data integration method to obtain integrated data, specifically including:

[0023] Obtain the data integration method, perform integration processing on the business data based on the data integration method, and analyze redundant data;

[0024] Analyze the data redundancy degree based on the redundant data;

[0025] Determine whether the data redundancy degree is greater than or equal to the set redundancy threshold;

[0026] If it is greater than or equal to the set redundancy threshold, eliminate the redundant data and update the integrated data;

[0027] If it is less, perform data integration processing on the preprocessed business data based on the data integration method to obtain integrated data.

[0028] Optionally, in the business data integration and sharing method described in the embodiments of the present application, classification sharing is performed on the integrated data based on the data sharing mode to obtain shared data of different categories, and the shared data of different categories is encrypted to obtain ciphertext data, specifically including:

[0029] Determine the sharing scope based on the data sharing mode, and perform classification sharing on the integrated data based on the sharing scope to obtain shared data of different categories;

[0030] Generate multiple random keys based on the key generation rule, and match the multiple random keys with the shared data of different categories;

[0031] Set the valid time of the random key, and perform different encryption processing on the shared data of different categories based on the multiple random keys respectively to obtain ciphertext data;

[0032] Obtain the duration of the random key, and determine whether the duration of the random key reaches the valid time;

[0033] If it reaches the valid time, update the random key;

[0034] If it does not reach the valid time, continue to calculate the valid time of the random key.

[0035] Optionally, in the business data integration and sharing method described in the embodiments of the present application, a data retrieval instruction is generated based on the access permission, the ciphertext data is retrieved based on the data retrieval instruction to obtain retrieved data, and the retrieved data is decrypted to obtain shared data, specifically including:

[0036] Generate a data retrieval instruction based on the access permission, parse the data retrieval instruction, and analyze the access intention;

[0037] Retrieve the ciphertext data based on the access intention to obtain a retrieval instruction;

[0038] Locate the ciphertext data based on the retrieval instruction to obtain the location information of the ciphertext data;

[0039] Obtain the ciphertext data at the corresponding position based on the location information of the ciphertext data, and decrypt and read the ciphertext data based on a secure reading mechanism to obtain the shared data.

[0040] In a second aspect, an embodiment of the present application provides a service data integration and sharing system, which includes: a memory and a processor. The memory includes a program of the service data integration and sharing method. When the program of the service data integration and sharing method is executed by the processor, the following steps are implemented:

[0041] Real-time obtain multi-source service data based on multi-source sensors, and preprocess the multi-source service data to obtain preprocessed service data;

[0042] Obtain the configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task;

[0043] Perform data integration processing on the preprocessed service data based on the data integration method to obtain integrated data;

[0044] Classify and share the integrated data based on the data sharing mode to obtain shared data of different categories, and encrypt the shared data of different categories to obtain ciphertext data;

[0045] Generate a data retrieval instruction based on the access permission, retrieve the ciphertext data based on the data retrieval instruction to obtain retrieved data, and decrypt the retrieved data to obtain the shared data.

[0046] Optionally, in the service data integration and sharing system described in the embodiment of the present application, real-time obtain multi-source service data based on multi-source sensors, and preprocess the multi-source service data to obtain preprocessed service data, which specifically includes:

[0047] Obtain multiple types of sensors, and respectively obtain different types of multi-source service data based on the multiple types of sensors;

[0048] Analyze the missing values of the multi-source service data to obtain missing data, correct the missing data to obtain corrected data, and perform cleaning on the corrected data to remove duplicate data to obtain cleaned data;

[0049] Obtain the data format of the cleaned data, and convert the data format of the cleaned data based on the standard format information to obtain conversion data corresponding to different types of multi-source service data;

[0050] Extract the data features of the conversion data corresponding to different types of multi-source service data, and perform fusion processing on the data features of the conversion data to obtain fused data;

[0051] Based on repeated analysis of the fused data, eliminate duplicate data to obtain preprocessed service data.

[0052] Optionally, in the service data integration and sharing system described in the embodiments of the present application, obtain the configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task, specifically including:

[0053] Obtain the configuration information of the data integration task, where the configuration information of the data task includes data source, data type, data integration conditions, and data integration target;

[0054] Generate configuration parameters based on the data source, data type, data integration conditions, and data integration target;

[0055] Match the data integration method based on the configuration parameters and calculate the matching degree;

[0056] Determine whether the matching degree is greater than or equal to the set matching degree threshold;

[0057] If it is greater than or equal to, select the current configuration parameters. If it is less than, generate adjustment information and optimize the current configuration parameters based on the adjustment information.

[0058] In a third aspect, the embodiments of the present application further provide a computer-readable storage medium, which includes a service data integration and sharing method program. When the service data integration and sharing method program is executed by a processor, the steps of the service data integration and sharing method described in any one of the above are implemented.

[0059] As can be seen from the above, a method, system, and medium for business data integration and sharing provided by an embodiment of the present application obtain multi-source business data in real time based on multi-source sensors, preprocess the multi-source business data to obtain preprocessed business data; obtain configuration information of a data integration task, and generate a data integration method based on the configuration information of the data integration task; perform data integration processing on the preprocessed business data based on the data integration method to obtain integrated data; classify and share the integrated data based on a data sharing mode to obtain shared data of different categories, encrypt the shared data of different categories to obtain ciphertext data; generate a data retrieval instruction based on an access right, retrieve the ciphertext data based on the data retrieval instruction to obtain retrieved data, and decrypt the retrieved data to obtain shared data; by performing configuration analysis on the data integration task, various integration methods of multi-source business data are realized, the integration effect is improved, different categories of sharing are performed on the integrated data through different data sharing modes, and the shared data is encrypted to achieve secure access to the shared data. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0061] Figure 1 It is a flowchart of the method for business data integration and sharing provided by an embodiment of the present application;

[0062] Figure 2 It is a flowchart of the preprocessing of multi-source business data in the method for business data integration and sharing provided by an embodiment of the present application;

[0063] Figure 3 It is a flowchart of the data integration method analysis method in the method for business data integration and sharing provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and illustrated herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but merely represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0065] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0066] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for business data integration and sharing in some embodiments of the present application. The method for business data integration and sharing is used in a terminal device. The method for business data integration and sharing includes the following steps:

[0067] S101, based on multi-source sensors, real-time obtain multi-source business data, preprocess the multi-source business data to obtain preprocessed business data;

[0068] S102, obtain the configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task;

[0069] S103, perform data integration processing on the preprocessed business data based on the data integration method to obtain integrated data;

[0070] S104, classify and share the integrated data based on the data sharing mode to obtain shared data of different categories, and encrypt the shared data of different categories to obtain ciphertext data;

[0071] S105, generate a data retrieval instruction based on the access permission, retrieve the ciphertext data based on the data retrieval instruction to obtain retrieved data, and decrypt the retrieved data to obtain shared data.

[0072] It should be noted that storing data in the form of files in a shared folder or file server and allowing users to obtain and use these files through network access permissions is suitable for situations where the amount of data is relatively small, the data structure is relatively simple, and the real-time requirements are not high. For example, some documents, reports and other data files are shared between internal teams of an enterprise. By setting database access permissions, different users or systems are allowed to access data in the same database, thereby achieving data sharing.

[0073] Please refer to Figure 2 , Figure 2 This is a multi-source business data preprocessing flow chart of a business data integration and sharing method in some embodiments of the present application. According to an embodiment of the present invention, multi-source business data is acquired in real time based on multi-source sensors, and the multi-source business data is preprocessed to obtain preprocessed business data, specifically including:

[0074] S201, acquiring multiple types of sensors, and acquiring different types of multi-source business data based on the multiple types of sensors;

[0075] S202, analyzing missing values ​​of multi-source business data to obtain missing data, correcting the missing data to obtain corrected data, and cleaning based on the corrected data to remove duplicate data to obtain cleaned data;

[0076] S203, obtaining the data format of the cleaned data, converting the data format of the cleaned data based on the standard format information, and obtaining converted data corresponding to different types of multi-source business data;

[0077] S204, extracting data features of the converted data corresponding to different types of multi-source business data, and fusing the data features of the converted data to obtain fused data;

[0078] S205, based on repeated analysis of the fused data, duplicate data is eliminated to obtain pre-processed business data.

[0079] It should be noted that different processing methods are used for missing values ​​of different types of sensor data. For numerical data, such as missing values ​​in temperature and humidity sensor data, the mean filling method can be used, that is, the average value of the historical data of the sensor is calculated to fill the missing values; if the data has time series characteristics, the linear interpolation method can be used to perform linear estimation filling based on the data of adjacent time points. For missing pixels in image data, image restoration algorithms, such as image restoration methods based on non-local means, can be used to fill missing areas based on local similarities of the image.

[0080] Please refer to Figure 3 , Figure 3It is a flowchart of a method for analyzing a data integration method in a business data integration and sharing method in some embodiments of the present application. According to an embodiment of the present invention, configuration information of a data integration task is obtained, and a data integration method is generated based on the configuration information of the data integration task, specifically including:

[0081] S301, obtain the configuration information of the data integration task, where the configuration information of the data task includes data source, data type, data integration condition, and data integration target;

[0082] S302, generate configuration parameters based on the data source, data type, data integration condition, and data integration target;

[0083] S303, match the data integration method based on the configuration parameters and calculate the matching degree;

[0084] S304, determine whether the matching degree is greater than or equal to a set matching degree threshold;

[0085] S305, if it is greater than or equal to, select the current configuration parameters, if it is less than, generate adjustment information, and optimize the current configuration parameters based on the adjustment information.

[0086] It should be noted that by analyzing the configuration information of the data integration task, corresponding configuration parameters are generated, and the configuration parameters are dynamically adjusted by analyzing the matching degree between the configuration parameters and the data integration method, so as to improve the matching degree of the configuration parameters.

[0087] According to an embodiment of the present invention, based on the data integration method, the preprocessed business data is subjected to data integration processing to obtain integrated data, specifically including:

[0088] Obtain the data integration method, perform integration processing on the business data based on the data integration method, and analyze redundant data;

[0089] Analyze the data redundancy degree based on the redundant data;

[0090] Determine whether the data redundancy degree is greater than or equal to a set redundancy degree threshold;

[0091] If it is greater than or equal to the set redundancy degree threshold, the redundant data is removed and the integrated data is updated;

[0092] If it is less than, perform data integration processing on the preprocessed business data based on the data integration method to obtain integrated data.

[0093] It should be noted that by analyzing the redundant data after data integration, and analyzing the redundancy degree, according to the set redundancy degree threshold, the redundant data is removed, so as to improve the data integration processing effect.

[0094] According to an embodiment of the present invention, the integrated data is classified and shared based on a data sharing mode to obtain shared data of different categories, and the shared data of different categories is encrypted to obtain ciphertext data, specifically including:

[0095] Determine the sharing scope based on the data sharing mode, and classify and share the integrated data based on the sharing scope to obtain shared data of different categories;

[0096] Generate multiple random keys based on the key generation rule, and match the multiple random keys with the shared data of different categories;

[0097] Set the valid time of the random key, and encrypt the shared data of different categories in different ways based on the multiple random keys to obtain ciphertext data;

[0098] Obtain the duration of the random key, and determine whether the duration of the random key reaches the valid time;

[0099] If the valid time is reached, update the random key;

[0100] If the valid time is not reached, continue to calculate the valid time of the random key.

[0101] It should be noted that in the encryption process, the management of keys is crucial. A Key Management System (KMS) is used to centrally manage keys, including key generation, storage, distribution, and update. For shared data of different categories, different keys can be used for encryption to improve data security. At the same time, the keys are updated regularly to reduce the risk of key cracking. For example, set the validity period of the key to one month, and automatically generate a new key and re-encrypt the relevant data after expiration. In the key distribution process, a secure transmission method is adopted, such as transmission through an SSL / TLS encrypted channel.

[0102] According to an embodiment of the present invention, a data retrieval instruction is generated based on the access permission, the ciphertext data is retrieved based on the data retrieval instruction to obtain the retrieved data, and the retrieved data is decrypted to obtain the shared data, specifically including:

[0103] Generate a data retrieval instruction based on the access permission, and parse the data retrieval instruction to analyze the access intention;

[0104] Retrieve the ciphertext data based on the access intention to obtain the retrieval instruction;

[0105] Locate the ciphertext data based on the retrieval instruction to obtain the location information of the ciphertext data;

[0106] Obtain the ciphertext data at the corresponding position based on the location information of the ciphertext data, and decrypt and read the ciphertext data based on a secure reading mechanism to obtain the shared data.

[0107] It should be noted that when decrypting the retrieved ciphertext data, the system matches the corresponding decryption key from the KMS according to the encryption method and storage location of the data. Then, through a secure key distribution mechanism, the decryption key is transmitted to the execution environment of the decryption operation. For example, adopting a role-based key distribution strategy, only the system components corresponding to the user roles with decryption permissions can obtain the decryption key. At the same time, during the key distribution process, an encrypted channel is used for transmission to ensure the confidentiality of the key.

[0108] According to an embodiment of the present invention, it further includes: adopting homomorphic encryption technology to allow specific calculation operations on encrypted data in data sharing without first decrypting the data. For example, in a financial data sharing scenario, a data analysis institution needs to perform a summation calculation on encrypted customer account balance data to count the total amount of funds. Homomorphic encryption technology enables the calculation to be directly performed on the encrypted data, and the calculation result is the same as that of performing the same calculation on the plaintext data after decryption, thereby realizing the analysis and processing of data while protecting data privacy.

[0109] In a second aspect, an embodiment of the present application provides a business data integration and sharing system, which includes: a memory and a processor. The memory includes a program of the business data integration and sharing method. When the program of the business data integration and sharing method is executed by the processor, the following steps are implemented:

[0110] Based on multi-source sensors, multi-source business data is obtained in real time, and the multi-source business data is preprocessed to obtain preprocessed business data;

[0111] Obtain the configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task;

[0112] Based on the data integration method, data integration processing is performed on the preprocessed business data to obtain integrated data;

[0113] Based on the data sharing mode, the integrated data is classified and shared to obtain shared data of different categories, and the shared data of different categories is encrypted to obtain ciphertext data;

[0114] Based on the access permission, a data retrieval instruction is generated, and based on the data retrieval instruction, the ciphertext data is retrieved to obtain retrieved data, and the retrieved data is decrypted to obtain shared data.

[0115] It should be noted that storing data in the form of files in a shared folder or file server, and users obtaining and using these files through network access rights, is applicable to situations where the data volume is relatively small, the data structure is relatively simple, and the real-time requirement is not high. For example, when sharing some data files such as documents and reports among internal teams of an enterprise, by setting the access rights of the database, different users or systems are allowed to access the data in the same database to achieve data sharing.

[0116] According to an embodiment of the present invention, multi-source service data is obtained in real time based on multi-source sensors, and the multi-source service data is preprocessed to obtain preprocessed service data, which specifically includes:

[0117] Obtain various types of sensors, and respectively obtain different types of multi-source service data based on the various types of sensors;

[0118] Analyze the missing values of the multi-source service data to obtain missing data, correct the missing data to obtain corrected data, and perform cleaning based on the corrected data to remove duplicate data to obtain cleaned data;

[0119] Obtain the data format of the cleaned data, and convert the data format of the cleaned data based on the standard format information to obtain conversion data corresponding to different types of multi-source service data;

[0120] Extract the data features of the conversion data corresponding to different types of multi-source service data, and perform fusion processing on the data features of the conversion data to obtain fusion data;

[0121] Based on repeated analysis of the fusion data, eliminate duplicate data to obtain preprocessed service data.

[0122] It should be noted that for the missing values of different types of sensor data, different processing methods are adopted. For numerical data, such as the missing values in temperature and humidity sensor data, the mean filling method can be used, that is, calculate the average value of the historical data of the sensor to fill the missing values; if the data has time series characteristics, linear interpolation can be used to perform linear estimation and filling based on the data at adjacent time points. For the missing pixels in image data, an image inpainting algorithm can be used, such as the non-local means based image inpainting method, to fill the missing area according to the local similarity of the image.

[0123] According to an embodiment of the present invention, obtain the configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task, which specifically includes:

[0124] Obtain the configuration information of the data integration task, and the configuration information of the data task includes data source, data type, data integration condition and data integration target;

[0125] Generate configuration parameters based on data sources, data types, data integration conditions, and data integration targets;

[0126] Match the data integration method based on the configuration parameters and calculate the matching degree;

[0127] Determine whether the matching degree is greater than or equal to the set matching degree threshold;

[0128] If it is greater than or equal to, the current configuration parameters are selected; if it is less than, adjustment information is generated and the current configuration parameters are optimized based on the adjustment information.

[0129] It should be noted that by analyzing the configuration information of the data integration task, the corresponding configuration parameters are generated, and by analyzing the matching degree between the configuration parameters and the data integration method, the configuration parameters are dynamically adjusted to improve the matching degree of the configuration parameters.

[0130] According to an embodiment of the present invention, data integration processing is performed on pre-processed business data based on a data integration method to obtain integrated data, specifically including:

[0131] Obtain data integration methods, integrate business data based on the data integration methods, and analyze redundant data;

[0132] Analyze data redundancy based on redundant data;

[0133] Determine whether the data redundancy is greater than or equal to the set redundancy threshold;

[0134] If it is greater than or equal to the set redundancy threshold, the redundant data will be eliminated and the integrated data will be updated;

[0135] If it is less than, the pre-processed business data is subjected to data integration processing based on the data integration method to obtain integrated data.

[0136] It should be noted that by analyzing the redundant data after data integration and analyzing the redundancy, the redundant data is eliminated according to the set redundancy threshold, thereby improving the data integration processing effect.

[0137] According to an embodiment of the present invention, the integrated data is classified and shared based on the data sharing mode to obtain different categories of shared data, and the different categories of shared data are encrypted to obtain ciphertext data, which specifically includes:

[0138] Determine the sharing scope based on the data sharing mode, and classify and share the integrated data based on the sharing scope to obtain different categories of shared data;

[0139] Generate multiple random keys based on the key generation rule, and match the multiple random keys with different categories of shared data;

[0140] Set the valid time of the random key, and perform different encryption processes on different categories of shared data based on multiple random keys to obtain ciphertext data;

[0141] Obtain the duration of the random key, and determine whether the duration of the random key reaches the valid time;

[0142] If the valid time is reached, update the random key;

[0143] If the valid time is not reached, continue to calculate the valid time of the random key.

[0144] It should be noted that in the encryption process, the management of keys is crucial. A Key Management System (KMS) is used to centrally manage keys, including key generation, storage, distribution, and update. For different categories of shared data, different keys can be used for encryption to improve data security. At the same time, keys are updated regularly to reduce the risk of key cracking. For example, set the validity period of the key to one month, and automatically generate a new key and re-encrypt the relevant data after expiration. In the process of key distribution, a secure transmission method is adopted, such as transmission through an SSL / TLS encrypted channel.

[0145] According to the embodiments of the present invention, generate a data retrieval instruction based on the access permission, retrieve the ciphertext data based on the data retrieval instruction to obtain the retrieved data, and decrypt the retrieved data to obtain the shared data, specifically including:

[0146] Generate a data retrieval instruction based on the access permission, parse the data retrieval instruction, and analyze the access intention;

[0147] Retrieve the ciphertext data based on the access intention to obtain the retrieval instruction;

[0148] Locate the ciphertext data based on the retrieval instruction to obtain the location information of the ciphertext data;

[0149] Obtain the ciphertext data at the corresponding position based on the location information of the ciphertext data, and decrypt and read the ciphertext data based on a secure reading mechanism to obtain the shared data.

[0150] It should be noted that when decrypting the retrieved ciphertext data, the system matches the corresponding decryption key from the KMS according to the encryption method and storage location of the data. Then, through a secure key distribution mechanism, the decryption key is transmitted to the execution environment of the decryption operation. For example, adopt a role-based key distribution strategy, and only the system components corresponding to the user role with decryption permission can obtain the decryption key. At the same time, in the process of key distribution, an encrypted channel is used for transmission to ensure the confidentiality of the key.

[0151] According to an embodiment of the present invention, it further includes: adopting homomorphic encryption technology, which allows specific calculation operations to be performed on encrypted data in data sharing without decrypting the data first. For example, in the scenario of financial data sharing, a data analysis institution needs to perform a summation calculation on encrypted customer account balance data to count the total amount of funds. The homomorphic encryption technology enables the calculation to be directly performed on the encrypted data, and the calculation result is the same as that of performing the same calculation on the plaintext data after decryption, thereby realizing the analysis and processing of data while protecting data privacy.

[0152] A third aspect of the present invention provides a computer-readable storage medium, which includes a program for the business data integration and sharing method. When the program for the business data integration and sharing method is executed by a processor, the steps of the business data integration and sharing method as described in any one of the above are implemented.

[0153] A method, system and medium for business data integration and sharing disclosed by the present invention, by obtaining multi-source business data in real time based on multi-source sensors, preprocessing the multi-source business data to obtain preprocessed business data; obtaining configuration information of a data integration task, and generating a data integration method based on the configuration information of the data integration task; performing data integration processing on the preprocessed business data based on the data integration method to obtain integrated data; classifying and sharing the integrated data based on a data sharing mode to obtain shared data of different categories, encrypting the shared data of different categories to obtain ciphertext data; generating a data retrieval instruction based on an access permission, retrieving the ciphertext data based on the data retrieval instruction to obtain retrieved data, and decrypting the retrieved data to obtain shared data; by configuring and analyzing the data integration task, various integration methods of multi-source business data are realized, the integration effect is improved, and different categories of sharing are performed on the integrated data through different data sharing modes, and the shared data is encrypted to realize secure access to the shared data.

[0154] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0155] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0156] In addition, each functional unit in the embodiments of the present invention can be all integrated in a processing unit, or each unit can be separately regarded as a unit alone, or two or more units can be integrated in one unit; the above-mentioned integrated unit can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0157] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: removable storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks and other various media that can store program codes.

[0158] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the foregoing storage medium includes: removable storage devices, ROM, RAM, magnetic disks or optical disks and other various media that can store program codes.

Claims

1. A method for business data integration and sharing, characterized in that, Including: Based on multi-source sensors, real-time multi-source service data is acquired, and the multi-source service data is preprocessed to obtain preprocessed service data; The configuration information of the data integration task is obtained, and a data integration method is generated based on the configuration information of the data integration task; Based on the data integration method, the preprocessed service data is subjected to data integration processing to obtain integrated data; Based on the data sharing mode, the integrated data is classified and shared to obtain shared data of different categories, and the shared data of different categories is encrypted to obtain ciphertext data; Based on the access authority, a data retrieval instruction is generated, and based on the data retrieval instruction, the ciphertext data is retrieved to obtain retrieved data, and the retrieved data is decrypted to obtain shared data.

2. The service data integration and sharing method according to claim 1, wherein Based on multi-source sensors, real-time multi-source service data is acquired, and the multi-source service data is preprocessed to obtain preprocessed service data, specifically including: Multiple types of sensors are obtained, and different types of multi-source service data are respectively obtained based on the multiple types of sensors; The missing values of the multi-source service data are analyzed to obtain missing data, the missing data is corrected to obtain corrected data, and based on the corrected data, duplicate data is removed through cleaning to obtain cleaned data; The data format of the cleaned data is obtained, and the data format of the cleaned data is converted based on the standard format information to obtain conversion data corresponding to different types of multi-source service data; The data features of the conversion data corresponding to different types of multi-source service data are extracted, and the data features of the conversion data are fused to obtain fused data; Based on repeated analysis of the fused data, duplicate data is removed to obtain preprocessed service data.

3. The business data integration and sharing method according to claim 2, wherein The configuration information of the data integration task is obtained, and a data integration method is generated based on the configuration information of the data integration task, specifically including: The configuration information of the data integration task is obtained, and the configuration information of the data task includes data source, data type, data integration condition, and data integration target; Based on the data source, data type, data integration condition, and data integration target, configuration parameters are generated; Based on the configuration parameters, a data integration method is matched, and the matching degree is calculated; It is judged whether the matching degree is greater than or equal to a set matching degree threshold; If it is greater than or equal to, the current configuration parameters are selected. If it is less than, adjustment information is generated, and based on the adjustment information, the current configuration parameters are optimized.

4. The service data integration and sharing method according to claim 3, characterized in that Based on the data integration method, the preprocessed service data is subjected to data integration processing to obtain integrated data, specifically including: The data integration method is obtained, and based on the data integration method, the service data is integrated and processed, and redundant data is analyzed; Based on the redundant data, the data redundancy degree is analyzed; It is judged whether the data redundancy degree is greater than or equal to a set redundancy degree threshold; If it is greater than or equal to the set redundancy degree threshold, the redundant data is removed, and the integrated data is updated; If it is less than, based on the data integration method, the preprocessed service data is subjected to data integration processing to obtain integrated data.

5. The service data integration and sharing method according to claim 4, characterized in that Based on the data sharing mode, the integrated data is classified and shared to obtain shared data of different categories, and the shared data of different categories is encrypted to obtain ciphertext data, specifically including: Determine the sharing scope based on the data sharing mode, and classify and share the integrated data based on the sharing scope to obtain different categories of shared data; Generate multiple random keys based on the key generation rule, and match the multiple random keys with different categories of shared data; The validity period of the random key is set, and different types of shared data are encrypted in different ways based on multiple random keys to obtain ciphertext data; Obtain the duration of the random key and determine whether the duration of the random key reaches the validity period; If the validity period is reached, the random key is updated; If the validity period has not been reached, continue calculating the validity period of the random key.

6. The business data integration and sharing method according to claim 5, wherein Generate a data retrieval instruction based on the access permission, retrieve the ciphertext data based on the data retrieval instruction to obtain the retrieved data, decrypt the retrieved data to obtain the shared data, specifically including: Generate data retrieval instructions based on access rights, parse the data retrieval instructions, and analyze access intentions; Retrieve the ciphertext data based on the access intention and obtain a retrieval instruction; Locate the ciphertext data based on the call instruction to obtain the location information of the ciphertext data; The ciphertext data at the corresponding position is obtained based on the positioning information of the ciphertext data, and the ciphertext data is decrypted and read based on a secure reading mechanism to obtain the shared data.

7. A business data integration and sharing system, characterized in that, The system includes: a memory and a processor, wherein the memory includes a program of a business data integration and sharing method, and when the program of the business data integration and sharing method is executed by the processor, the following steps are implemented: Acquire multi-source business data in real time based on multi-source sensors, pre-process the multi-source business data, and obtain pre-processed business data; Acquire configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task; Perform data integration processing on the pre-processed business data based on the data integration method to obtain integrated data; Based on the data sharing mode, the integrated data is classified and shared to obtain different categories of shared data, and the different categories of shared data are encrypted to obtain ciphertext data; A data retrieval instruction is generated based on the access permission, the ciphertext data is retrieved based on the data retrieval instruction to obtain the retrieved data, and the retrieved data is decrypted to obtain the shared data.

8. The business data integration and sharing system according to claim 7, characterized in that Based on multi-source sensors, multi-source business data is acquired in real time, and the multi-source business data is pre-processed to obtain pre-processed business data, which specifically includes: Acquire multiple types of sensors, and acquire different types of multi-source business data based on the multiple types of sensors; Analyze the missing values ​​of multi-source business data to obtain missing data, correct the missing data to obtain corrected data, clean the corrected data, remove duplicate data, and obtain cleaned data; Obtain the data format of the cleaned data, convert the data format of the cleaned data based on the standard format information, and obtain converted data corresponding to different types of multi-source business data; Extracting data features of conversion data corresponding to different types of multi-source business data, fusing the data features of the conversion data to obtain fused data; Based on repeated analysis of the fused data, duplicate data is eliminated to obtain pre-processed business data.

9. The business data integration and sharing system according to claim 8, wherein Obtain the configuration information of the data integration task, and generate a data integration method based on the configuration information of the data integration task, specifically including: Obtain the configuration information of the data integration task, where the configuration information of the data task includes data source, data type, data integration conditions, and data integration target; Generate configuration parameters based on the data source, data type, data integration conditions, and data integration target; Match the data integration method based on the configuration parameters and calculate the matching degree; Judge whether the matching degree is greater than or equal to the set matching degree threshold; If it is greater than or equal to, select the current configuration parameters. If it is less than, generate adjustment information and optimize the current configuration parameters based on the adjustment information.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a business data integration and sharing method program. When the business data integration and sharing method program is executed by a processor, the steps of the business data integration and sharing method described in any one of claims 1 to 6 are implemented.

Images