Multi-data center monitoring image data sharing method, system and equipment
Through federated learning and privacy computing technology, security and privacy protection issues in data sharing among data centers are solved, efficient data analysis and processing are achieved, and collaboration capabilities and resource utilization efficiency between data centers are improved.
Patent Information
- Application Number
- CN202411097428.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-12-06
AI Technical Summary
Existing data center data sharing technology faces data security and privacy protection, data quality and consistency, technical compatibility and integration, as well as legal and ethical issues, limiting the efficiency and accuracy of data sharing.
Using federated learning and privacy computing technology, we build a joint model through data preprocessing, encryption and model alignment to realize the secure sharing of image data in multi-data centers, ensure data privacy and improve data analysis capabilities.
It enhances data privacy and security, improves data analysis and processing capabilities, reduces data transmission costs, improves resource utilization efficiency and collaboration capabilities between data centers.
Smart Images

Figure CN119089489A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a data sharing method, system and device, and in particular to a multi-data center monitoring image data sharing method, system and device. Background Art
[0002] In today's digital age, data has become one of the most valuable assets of enterprises. However, with the surge in data volume, how to safely and efficiently manage and utilize data, especially data across multiple data centers, has become a major challenge facing enterprises. In large corporate groups, there are usually multiple data centers, which are distributed in different geographical locations and serve different business departments or subsidiaries. These data centers often have their own independent data, and due to business competition, they need to maintain data independence and confidentiality. The monitoring image data of the data center will contain privacy data of parameters such as relevant equipment nameplates, equipment models, specifications and configurations. Once this information is leaked, it will expose key business secrets, so it cannot be shared between data centers. Due to the competitive relationship between data centers, sharing this sensitive information will weaken their respective market competitiveness.
[0003] However, the head office often needs to extract valuable information from these scattered data and build a general model that can reflect the business characteristics of the entire enterprise group in order to optimize decision-making, improve service quality and enhance market competitiveness. In order to solve this problem, this patent proposes a new multi-data center data sharing technology, which combines the advantages of federated learning and privacy computing, and aims to achieve cross-institutional data sharing and analysis while protecting data privacy. This technology can realize collaborative learning among multiple data centers and jointly train an efficient model under the premise of protecting data privacy. Through federated learning, the data of each data center does not need to leave the local area, and only collaborates by exchanging model parameters, effectively avoiding the risk of data leakage. At the same time, combined with privacy computing technology, data security during model training is further guaranteed. The application of federated learning in the field of data center sharing can not only improve the accuracy and response speed of data sharing, but also promote data sharing and collaborative innovation within the industry, and promote the green development and energy efficiency optimization of the data center industry.
[0004] At present, there have been some studies on federated learning technology, privacy computing technology and data sharing technology, as follows:
[0005] Ship trajectory prediction based on federated learning: Aiming at the real-time prediction of maritime ship trajectories, a ship trajectory prediction algorithm (E-FVTP) based on the Fedves federated learning framework and the convolutional neural network-gated recurrent unit (CNN-GRU) model is proposed. According to the Fedves federated learning framework, by standardizing the client data set size and client regularization term, the impact of the data's non-independent and identically distributed characteristics on the global model is reduced while maintaining the original client data characteristics, thereby accelerating the convergence speed. Aiming at the scenario of shortage of marine communication resources, a CNN-GRU ship trajectory prediction model based on the ship automatic identification system (AIS) data is established to solve the problem of insufficient computing power of ship terminal equipment. Experimental results on the MarineCadastre open source and Zhoushan marine ship navigation AIS dataset show that E-FVTP can achieve 67% higher convergence speed and 76.32% lower communication cost while reducing the prediction error by 40% compared with centralized training, which can achieve accurate prediction of ship trajectories in complex marine environments and ensure maritime traffic safety.
[0006] Multi-line high-speed train bogie fault diagnosis based on federated learning: The lack of sufficient fault data features of high-speed train bogies on a single line leads to limited generalization ability of the fault diagnosis model. In order to diagnose bogie faults of high-speed trains on multiple lines, a global bogie fault diagnosis method based on federated learning is proposed. For the bogie vibration signals of each line, a multi-scale convolution fusion algorithm is used locally to extract fault features at different scales and fuse them, and a local bogie fault diagnosis model is established locally; without leaking data privacy, the fault diagnosis models of all lines are aggregated through a third party, the model parameter weights are adjusted, and the fault diagnosis model is optimized, and finally a multi-party joint training of the bogie global fault diagnosis model is realized. Experiments show that under the framework of federated learning, the global bogie fault diagnosis model not only has an accuracy rate of more than 93% for bogie fault diagnosis of lines participating in federated modeling, but also has an accuracy rate of more than 75% for bogie fault diagnosis of lines not participating in federated modeling, providing a practical solution to the "data island" problem in rail transit.
[0007] Railway data privacy computing: Aiming at the data privacy protection issues of multiple data subjects in railway data sharing, in order to achieve the purpose and value of data sharing, the railway data privacy computing system is studied and constructed. Based on the overview of the theory and technology of privacy computing and the introduction of the railway data privacy computing demand scenarios, the architecture and functions of the railway data privacy computing system are elaborated in detail, and the application scenarios of the system are introduced. The construction of the railway data privacy computing system aims to improve the utilization efficiency of railway industry data, enhance the security and privacy of data, and provide more support for data analysis and decision-making in the railway industry.
[0008] Privacy computing for financial applications: Financial institutions are in urgent need of using data resources to improve operational efficiency and carry out cross-industry and cross-institutional data integration. The emergence of privacy computing provides financial institutions with a platform for collaborative development while ensuring data security. The application of privacy computing will greatly improve the efficiency of banks in risk management and marketing promotion. Privacy computing should build an open financial ecosystem in financial scenario applications, attach importance to institutional innovation, provide a safe environment for data applications, and improve the quality and efficiency of data use. Balance data protection and data application to fully unleash the potential of data.
[0009] Biomedical privacy computing: The wave of medical informatization has further enriched the dimensions and scale of biomedical data. However, due to the compliance of data flow, privacy security and the protection of their own rights and interests, these data are often confined to a single institution. If data cannot be interconnected, the real value cannot be released. It is precisely driven by these factors that privacy-preserving computing, which is considered to be the optimal solution at the technical level, has quickly moved from behind the scenes to the front stage and entered a stage of rapid development. The application of privacy-preserving computing in medical scenarios is analyzed and studied from the perspectives of technology and practical application.
[0010] Privacy computing in health care: The difficulty in sharing, opening and utilizing medical and health data safely is one of the most important factors hindering the development of information technology in the current health care industry. The study explores the difficulties faced by the health care industry in actual application scenarios of data sharing and utilization, proposes application solutions for privacy computing in various application scenarios, and gives recommendations on the security of privacy computing models in corresponding scenarios based on actual needs.
[0011] Research on the Federated Distillation Data Sharing Model Based on Blockchain: Scattered and isolated massive data form "data islands" that make it impossible for data to interact and connect. How to safely and effectively share the knowledge information in the data while protecting the privacy of the original data has become a hot research issue. Based on the above, a blockchain-based federated distillation data sharing model (BFDS) is proposed. Different from the centralized architecture, blockchain is used to unite multiple participants to form a teacher network to achieve distributed collaborative work; by exchanging distillation outputs, the knowledge information in the data is transferred and the lightweight model is jointly trained; a multi-weight node trusted evaluation algorithm is proposed, which calls smart contracts to assign weights and generate traceable global soft labels to reduce the negative impact caused by quality differences among participants.
[0012] Medical data sharing model based on blockchain based on conditional proxy re-encryption: Medical data has high privacy and sensitivity. However, in the past, centralized storage was often adopted in the process of sharing medical data, which brought the risk of data privacy leakage and illegal tampering. In view of the above problems, this paper combines blockchain and proxy re-encryption technology to propose a blockchain medical data sharing model based on conditional proxy re-encryption. Hospitals at all levels form a consortium blockchain, and use the improved practical Byzantine consensus mechanism to select proxy nodes, which improves the efficiency of nodes reaching consensus. With the help of the massive storage capacity of cloud servers, the patient's medical data is stored in the cloud server, and the conditional summary ciphertext is saved in the consortium blockchain. The data requester performs conditional search on the consortium chain to obtain the patient's medical data. At the same time, the proxy re-encryption technology is used to achieve fine-grained sharing between specific data requesters and third-party users. Experimental results show that the blockchain medical data sharing model based on conditional proxy re-encryption has more advantages in terms of communication and computing overhead.
[0013] Research on the optimization of government data sharing mechanism under the background of digital government: In the context of digital government construction, the government urgently needs to promote government data sharing to optimize service levels in this way. In order to resolve the contradiction between data security and citizen privacy and avoid problems such as information islands and digital barriers, this article studies the construction of government data sharing mechanism under the background of digital government, and lists specific strategies, including achieving coordination between multiple subjects, coordination between governance objects and governance tools, coordination between technological innovation and citizen needs, coordination between information exchange and organizational institutions, and coordination between technical rationality and value rationality, in order to provide reference for relevant workers.
[0014] However, existing data center data sharing technologies face many challenges and problems, including data security and privacy protection, data quality and consistency, technical compatibility and integration, legal and ethical issues of data sharing, etc. These problems limit the development and application of detection technology and need to be solved through innovative methods. The specific problems are as follows:
[0015] Data security and privacy protection issues: Existing data sharing technologies face severe challenges in data security and privacy protection. With the frequent occurrence of data breaches, enterprises and individuals are paying more and more attention to the security and privacy of data. Traditional data sharing models often rely on centralized storage and transmission mechanisms, which makes data vulnerable to hacker attacks and unauthorized access during storage and transmission. In addition, centralized storage of data also increases the risk of data abuse or leakage. Although some technologies such as encrypted transmission and database encryption are used to improve data security, these measures cannot completely eliminate the risk of data leakage. At the same time, with the implementation of data privacy protection regulations, enterprises also need to ensure compliance with the requirements of these regulations when sharing data, which further increases the complexity and difficulty of data sharing.
[0016] Data quality and consistency issues: Another issue with data sharing is data quality and consistency. In the process of data sharing, data from different sources have problems such as inconsistent formats, inaccurate or outdated data, which will affect the availability and reliability of data. For example, different departments or organizations use different data standards and definitions, which leads to confusion or errors when integrating data. In addition, data inconsistency also leads to deviations in analysis results and affects the accuracy of decision-making. In order to solve these problems, it is necessary to establish unified data standards and specifications, as well as an effective data quality management mechanism. However, this requires a lot of coordination and cooperation, as well as the transformation and upgrading of existing systems, which will encounter many difficulties and challenges in actual operations.
[0017] Technical compatibility and integration issues: Existing data sharing technologies also have problems with technical compatibility and integration. With the rapid development of technology, new data formats, protocols, and interfaces continue to emerge, which brings compatibility issues to data sharing. Different systems and platforms are difficult to achieve seamless integration due to inconsistent technical standards. In addition, some old systems lack the ability to be compatible with new technologies, which limits the scope and efficiency of data sharing. In order to solve these problems, it is necessary to develop and promote unified technical standards and interfaces, as well as provide flexible integration solutions. However, this requires overcoming the differences between different technologies and coordinating the needs and expectations of different stakeholders, which will encounter many technical and management challenges in actual operations.
[0018] Legal and ethical issues of data sharing: Data sharing also involves legal and ethical issues. In the process of data sharing, issues such as data ownership, usage rights, and benefit distribution need to be considered. For example, the original provider of data has specific requirements and expectations for how the data is used and shared, and the user of the data needs to use the data reasonably under the premise of complying with laws and regulations. In addition, data sharing also involves sensitive issues such as personal privacy and intellectual property rights, and it is necessary to find a balance between protecting the rights and interests of data subjects and promoting data sharing. In order to solve these problems, it is necessary to establish clear data sharing policies and agreements, as well as effective supervision and dispute resolution mechanisms. However, this requires overcoming differences in legal systems and coordinating contradictions and conflicts between different stakeholders, which will encounter many legal and ethical challenges in actual operations.
[0019] Therefore, it is of great significance to develop a multi-data center surveillance image data sharing method based on federated learning and privacy computing that can both protect data privacy and promote collaboration among data centers. Summary of the invention
[0020] Purpose of the invention: The purpose of the present invention is to provide a multi-data center monitoring image data sharing method, system and device that can enhance data confidentiality and improve data analysis and processing capabilities.
[0021] Technical solution: A multi-data center monitoring image data sharing method according to the present invention comprises:
[0022] Step 1: Preprocess and protect the privacy of the data to be shared in the monitoring images of each data center;
[0023] Step 2: Encrypt the processed data and align the models of each data center;
[0024] Step 3: Each data center will aggregate and update the encrypted and aligned models through mutual communication on the central server, update the model parameters, build a joint model, and use the joint model to complete data sharing.
[0025] Furthermore, the step 1 specifically includes:
[0026] Step 1.1: Preprocess the data to be shared through cleaning, standardization and feature engineering operations;
[0027] Step 1.2: The pre-processed data is screened, de-identified, and desensitized to protect privacy and obtain the processed data.
[0028] Furthermore, the cleaning operation described in step 1.1 specifically includes identifying and processing missing values, outliers and duplicate records of the data;
[0029] The standardization operation is specifically to convert the data into a unified preset scale to eliminate the dimensional influence between different features;
[0030] The feature engineering operation specifically involves extracting features from raw data and combining or converting them to generate new features for model training.
[0031] Furthermore, the identification and processing of missing values adopts interpolation, deletion or filling methods to ensure the integrity and consistency of the data;
[0032] The identification and processing of outliers, identifying outliers through statistical analysis or machine learning algorithms, and correcting or deleting them;
[0033] The identification and processing of duplicate records, by comparing data items, delete similar data to avoid redundancy in the data set.
[0034] Furthermore, the screening described in step 1.2 specifically involves identifying sensitive fields in the data set, and through screening, determining the data that needs to be protected or removed according to the current data sharing needs;
[0035] De-identification refers to removing or replacing information that can uniquely identify an individual from the data, replacing the information with an anonymous identifier, or obfuscating the date and time stamp information to reduce the traceability of the data;
[0036] The data desensitization is specifically a method of modifying the data content to reduce the sensitivity of the data, converting specific values into broader categories, reducing the accuracy of the data, and generalizing specific information into broader regional information.
[0037] Furthermore, the step 2 specifically includes:
[0038] Step 2.1: Convert the processed data into an unreadable form and use a randomization mechanism to add random noise to the data. The mathematical model is represented by Share i (θ) = θ + r i , where Share i (θ) is the secret shared model parameters of the i-th data center, r i is random noise, θ is the data to be added with noise;
[0039] Step 2.2: Complete the model alignment of each data center by unifying the feature space, achieving consistent data distribution, achieving compatibility of model parameters, and achieving interface standardization.
[0040] Furthermore, the step 2.2 comprises:
[0041] Through the feature mapping function The original feature space Mapping to a new consistent feature space The data from different data centers are trained under a unified model architecture. The aggregation algorithm of federated learning is used to encrypt and align the data models of each data center, and then the model updates of each data center are aggregated to optimize the global model. The mathematical expression is: where θ global is the global model parameter, θ k is the model parameter of the kth data center, and K is the total number of data centers.
[0042] Furthermore, the step 3 is specifically as follows:
[0043] Each data center processes the data locally and generates model updates, which include weight adjustments and gradient information or optimizer status updates. After the model update is generated, it is transmitted based on encryption. Multiple rounds of iterative updates and aggregation are performed through an aggregation algorithm. After each round of iteration, the aggregated model parameters or updates are sent back to each data center to guide the next step of model training. After verification and testing, the construction of the joint model is completed, and data sharing is completed using the joint model.
[0044] Based on the same inventive concept, the present invention also provides a multi-data center monitoring image data sharing system, comprising:
[0045] A preprocessing module is used to preprocess and protect the privacy of the data to be shared in the monitoring images of each data center;
[0046] The encryption alignment module is used to encrypt the processed data and align the models of each data center;
[0047] The model building and sharing module is used by each data center to aggregate and update the encrypted and aligned models on the central server through mutual communication, update model parameters, build a joint model, and use the joint model to complete data sharing.
[0048] Based on the same inventive concept, the present invention also provides a computing device, comprising: one or more processors, one or more memories and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the multi-data center monitoring image data sharing method according to any one of the above items are implemented.
[0049] Beneficial effects: Compared with the existing technology, the present invention enhances data privacy and security through the level of data sharing technology; improves resource utilization efficiency and reduces costs through the enterprise integration management level of multiple data centers; ensures data security and enhances data confidentiality through federated learning and privacy computing technology; reduces direct data interaction between data centers and reduces data transmission costs through technical architecture design; and improves the analysis and processing capabilities of monitoring image data through the use of joint models. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0051] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0052] In the technical solution of this embodiment, there are mainly three parts: data preprocessing and privacy protection, data encryption and model alignment, and federated learning aggregation into a joint model. First, for data preprocessing and privacy protection, the main operations are to clean, standardize and perform basic operations of feature engineering on the data to provide a basis for model training, and use a variety of technical means such as data screening, de-identification, and data desensitization to achieve data privacy protection; secondly, data encryption and model alignment ensure the security of data during transmission, and realize the calculation of encrypted data through encryption technology, and at the same time align the model to ensure the collaborative work of models in different data centers; finally, federated learning is aggregated into a joint model, so that each data center can aggregate and update the encrypted and aligned models through mutual communication on the central server, update the relevant model parameters, and build a joint model that can reflect the data characteristics of all data centers.
[0053] As attached Figure 1 As shown, the multi-data center monitoring image data sharing method of this embodiment includes:
[0054] Step 1: Preprocess and protect the privacy of the data to be shared in the monitoring images of each data center;
[0055] Step 2: Encrypt the processed data and align the models of each data center;
[0056] Step 3: Each data center will aggregate and update the encrypted and aligned models through mutual communication on the central server, update the model parameters, build a joint model, and use the joint model to complete data sharing.
[0057] Specifically, step 1 includes:
[0058] Step 1.1: Data preprocessing
[0059] Data preprocessing is a key step in the federated learning process of the data center joint model. It involves multiple links such as data cleaning, data standardization, and feature engineering. Data cleaning is the first step in preprocessing. The data center needs to identify and process missing values, outliers, and duplicate records. Missing value processing usually uses methods such as interpolation, deletion, or filling to ensure the integrity and consistency of the data. Outlier detection uses statistical analysis or machine learning algorithms to identify data points that are significantly different from most data, and then decides whether to correct or delete these outliers. The identification and deletion of duplicate records is achieved by comparing the similarities between data items to avoid redundancy in the data set. Data standardization is another important step in preprocessing. It eliminates the dimensional effects between different features by converting data to a unified scale. Feature engineering is another key step in the data preprocessing process. It involves extracting features from raw data that are helpful for model training. Feature engineering is to select the most useful features for model training from a large number of candidate features, generate new features by combining or converting the original features, and convert the original features into a form that is easier for the model to process.
[0060] Step 1.2: Privacy Protection
[0061] Privacy protection is a key step in the federated learning process of the data center joint model. It involves a variety of technical means such as data screening, de-identification, and data desensitization. Data screening is the first step to ensure data privacy. The data center needs to thoroughly check the original data, identify and process information that leaks privacy. This includes identifying sensitive fields in the data set. Through screening, the data center can determine which data needs to be specially protected or removed from the data set. De-identification is one of the privacy protection technologies. It involves removing or replacing information that can uniquely identify individuals from the data. Replace information with anonymous identifiers, or blur information such as dates and timestamps to reduce the traceability of data. De-identification not only helps reduce the risk of data leakage, but also reduces the probability of data being re-identified to a certain extent. Data desensitization is another privacy protection technology that reduces the sensitivity of data by modifying the content of the data. Convert specific values into broader categories, thereby reducing the precision of the data. The method of generalizing specific information into broader regional information can reduce the privacy risk of data without significantly affecting the practicality of the data.
[0062] Step 2 includes:
[0063] Step 2.1: Data encryption
[0064] The data encryption step is a key link to ensure the security of data during storage, transmission and processing. Data encryption technology prevents unauthorized access by converting raw data into an unreadable form, while the method of adding noise is a strategy to further enhance privacy protection based on data encryption. This method is often used in conjunction with the privacy noise addition technique to protect the privacy of individual data by adding carefully calculated random noise to the data while allowing collective statistical analysis of the data. Noise is added before the data enters the model training step, thereby protecting privacy at the source of the data to prevent unauthorized access to long-term stored data and protecting the data in use from being traced to specific data individuals. In order to achieve effective noise addition, data centers can adopt a variety of randomization mechanisms to achieve privacy protection by adding random noise to the data, further enhancing the privacy protection of the data during the joint model training process.
[0065] In terms of data encryption, privacy noise is added to the data without leaking the respective input data. Its mathematical model can be expressed as Share i (θ) = θ + r i , where Share i (θ) is the secret shared model parameters of the i-th data center, r i is random noise. In this way, even if data leakage occurs during the model update process, the original model parameters cannot be restored. The integration of these technologies not only improves the accuracy and generalization ability of the model, but also optimizes the utilization of resources and the security of data, providing strong support for subsequent model training.
[0066] Step 2.2: Model alignment
[0067] The model alignment step plays a crucial role in federated learning, ensuring that data from different data centers can be effectively integrated under a unified model architecture. This step involves multiple technical details, including the unification of feature space, consistency of data distribution, compatibility of model parameters, and standardization of interfaces. The unification of feature space is the basis of model alignment. Data from different data centers differ in feature selection, feature type, or feature scale. In order to achieve model alignment, these features need to be processed uniformly to ensure that data from different data centers have the same feature dimension and scale when input into the model. The consistency of data distribution is also important for model alignment. Data from different data centers come from different populations or environments, resulting in differences in data distribution. This distribution difference will affect the generalization ability of the model, so measures need to be taken to reduce the distribution difference, help the model better adapt to the data distribution of different data centers, and improve the adaptability of the model on data from all data centers. The compatibility of model parameters is another key aspect of model alignment. In federated learning, models from different data centers differ in parameter initialization, learning rate, or regularization strategy. In order to achieve model alignment, a unified model training protocol is formulated to ensure that models in different data centers can follow the same rules during the training process, thereby improving the consistency and synergy of the models.
[0068] In terms of model alignment, this patent proposes an alignment strategy based on feature space mapping to ensure that data from different data centers are consistent in feature representation. The original feature space Mapping to a new consistent feature space This enables data from different data centers to be trained under a unified model architecture. When using the aggregation algorithm in federated learning, the data model is obtained by encrypting and aligning each data center, and then the model updates of each data center are aggregated to optimize the global model. Its mathematical expression can be expressed as where θ global is the global model parameter, θ k is the model parameter of the kth data center, and K is the total number of data centers. This approach not only improves the generalization ability of the model, but also reduces the dependence on the central server and reduces the communication overhead.
[0069] Step 3 includes: The process of federated learning aggregation into a joint model is a complex and sophisticated technical link, which involves collecting model updates from each participating data center and integrating these updates through a secure aggregation algorithm to form a globally optimized joint model. This process first requires each data center to process the data locally and generate model updates, which include weight adjustments, gradient information, or the state of the optimizer. After the model updates are generated, how to securely transmit these updates from one data center to the aggregation server or another data center is a key issue in the aggregation process. In order to protect data privacy and security, encryption technology is used to ensure the security of updates during transmission. The aggregation algorithm is used to effectively integrate updates from different data centers, balancing the contributions from different data sources while taking into account the diversity of data and the computing power of data centers. In order to ensure that the aggregated model can accurately reflect the data characteristics of all data centers, multiple rounds of iterative updates and aggregation are required. After each round of iteration, the aggregated model parameters or updates are sent back to each data center to guide the next step of model training. Finally, the process of federated learning aggregating into a joint model requires using part of the data to train the model, comparing the results and predicted values, and verifying and testing based on the accuracy to ensure that the aggregated model not only performs well on the data of each data center, but also maintains a high generalization ability on unseen test data. Through the comprehensive application of these technologies, federated learning can achieve cross-data center model training and optimization while protecting data privacy, and build a joint model with superior performance.
[0070] Based on the same inventive concept, this embodiment also provides a multi-data center monitoring image data sharing system, including:
[0071] A preprocessing module is used to preprocess and protect the privacy of the data to be shared in the monitoring images of each data center;
[0072] The encryption alignment module is used to encrypt the processed data and align the models of each data center;
[0073] The model building and sharing module is used by each data center to aggregate and update the encrypted and aligned models on the central server through mutual communication, update model parameters, build a joint model, and use the joint model to complete data sharing.
[0074] Based on the same inventive concept, this embodiment also provides a computing device, including: one or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the multi-data center monitoring image data sharing method according to any one of the above items are implemented.
Claims
1. A method for sharing monitoring image data among multiple data centers, characterized in that: include: Step 1: Preprocess and protect the privacy of the data to be shared in the monitoring images of each data center; Step 2: Encrypt the processed data and align the models of each data center; Step 3: Each data center will aggregate and update the encrypted and aligned models through mutual communication on the central server, update the model parameters, build a joint model, and use the joint model to complete data sharing.
2. The multi-data center monitoring image data sharing method according to claim 1, characterized in that: The step 1 specifically includes: Step 1.1: Preprocess the data to be shared through cleaning, standardization and feature engineering operations; Step 1.2: The pre-processed data is screened, de-identified, and desensitized to protect privacy and obtain the processed data.
3. The multi-data center monitoring image data sharing method according to claim 2, characterized in that: The cleaning operation described in step 1.1 is specifically to identify and process missing values, outliers and duplicate records in the data; The standardization operation is specifically to convert the data into a unified preset scale to eliminate the dimensional influence between different features; The feature engineering operation specifically involves extracting features from raw data and combining or converting them to generate new features for model training.
4. The multi-data center monitoring image data sharing method according to claim 3 is characterized in that: The identification and processing of missing values adopts interpolation, deletion or filling methods to ensure the integrity and consistency of data; The identification and processing of outliers, identifying outliers through statistical analysis or machine learning algorithms, and correcting or deleting them; The identification and processing of duplicate records, by comparing data items, delete similar data to avoid redundancy in the data set.
5. The multi-data center monitoring image data sharing method according to claim 2, characterized in that: The screening described in step 1.2 specifically involves identifying sensitive fields in the data set and, through screening, determining the data that needs to be protected or removed based on the current data sharing needs; De-identification refers to removing or replacing information that can uniquely identify an individual from the data, replacing the information with an anonymous identifier, or obfuscating the date and time stamp information to reduce the traceability of the data; The data desensitization is specifically a method of modifying the data content to reduce the sensitivity of the data, converting specific values into broader categories, reducing the accuracy of the data, and generalizing specific information into broader regional information.
6. The multi-data center monitoring image data sharing method according to claim 1, characterized in that: The step 2 specifically includes: Step 2.1: Convert the processed data into an unreadable form and use a randomization mechanism to add random noise to the data. The mathematical model is represented by Share i (γ) = γ + r i , where Share i (θ) is the secret shared model parameters of the i-th data center, r i is random noise, θ is the data to be added with noise; Step 2.2: Complete the model alignment of each data center by unifying the feature space, achieving consistent data distribution, achieving compatibility of model parameters, and achieving interface standardization.
7. The multi-data center monitoring image data sharing method according to claim 6, characterized in that: The step 2.2 comprises: Through the feature mapping function f: The original feature space Mapping to a new consistent feature space The data from different data centers are trained under a unified model architecture. The aggregation algorithm of federated learning is used to encrypt and align the data models of each data center, and then the model updates of each data center are aggregated to optimize the global model. The mathematical expression is: where θ global is the global model parameter, θ k is the model parameter of the kth data center, and K is the total number of data centers.
8. The multi-data center monitoring image data sharing method according to claim 1, characterized in that: The step 3 is specifically as follows: Each data center processes the data locally and generates model updates, which include weight adjustments and gradient information or optimizer status updates. After the model update is generated, it is transmitted based on encryption. Multiple rounds of iterative updates and aggregation are performed through aggregation algorithms. After each round of iteration, the aggregated model parameters or updates are sent back to each data center to guide the next step of model training. After verification and testing, the construction of the joint model is completed, and data sharing is completed using the joint model.
9. A multi-data center monitoring image data sharing system, characterized in that: include: A preprocessing module is used to preprocess and protect the privacy of the data to be shared in the monitoring images of each data center; The encryption alignment module is used to encrypt the processed data and align the models of each data center; The model building and sharing module is used by each data center to aggregate and update the encrypted and aligned models on the central server through mutual communication, update model parameters, build a joint model, and use the joint model to complete data sharing.
10. A computing device, characterized in that include: One or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the multi-data center monitoring image data sharing method according to any one of claims 1 to 8 are implemented.
