Private cloud co-processing platform for internal data and internet connection data

Abstract

The invention discloses a private cloud co-processing platform for internal data and internet connection data, and relates to the technical field of cloud computing, the platform comprises a data integration module, a data cleaning and preprocessing module, a data exchange area and an analysis engine; the data integration module performs integration processing on the acquired data to obtain integrated data; the data cleaning and preprocessing module is connected with the data integration module and is used for cleaning and preprocessing integrated data; the data exchange area is connected with the data cleaning and preprocessing module and provides a physical isolation mechanism; and the analysis engine is connected with the data exchange area, provides a cooperative working environment, and performs data mining and mode recognition on the preprocessed data by adopting a data analysis algorithm under a physical isolation mechanism. According to the method and the device, effective integration of internal and external data can be realized, complete physical isolation of the internal data and the Internet is ensured, the data security is improved, the requirement of real-time analysis is met, and efficient processing of the data is realized.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a private cloud collaborative processing platform for internal data and Internet-connected data. Background Art

[0002] With the rapid development of information technology, enterprises need to process a large amount of data from different sources, including data generated by internal business systems and data from the external Internet. Traditional data processing methods often cannot meet the needs of real-time analysis, and there are also problems with data security. Therefore, it is imperative to develop a private cloud collaborative processing platform that can effectively integrate internal and external data, ensure data security, provide advanced analytical capabilities, and be physically isolated from the Internet. Summary of the invention

[0003] The above-mentioned problems existing in the prior art have not been solved. The present application provides a private cloud collaborative processing platform for internal data and Internet-connected data.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a private cloud collaborative processing platform for internal data and Internet-connected data, including:

[0006] A data integration module is used to acquire data and perform integrated processing on the acquired data to obtain integrated data;

[0007] A data cleaning and preprocessing module, connected to the data integration module, for cleaning and preprocessing the integrated data to obtain preprocessed data;

[0008] A data exchange area, connected to the data cleaning and preprocessing module, for providing a physical isolation mechanism;

[0009] The analysis engine is connected to the data exchange area and is used to provide a collaborative work environment and to use a data analysis algorithm to perform data mining and pattern recognition on the pre-processed data under the physical isolation mechanism to obtain data processing results; the collaborative work environment is used to support multi-user collaboration.

[0010] Optionally, the private cloud collaborative processing platform for internal data and Internet-connected data further includes:

[0011] A visualization tool is connected to the analysis engine and is used to perform visualization processing on the data processing result to obtain a visualization processing result.

[0012] Optionally, a secure transmission protocol is used for data transmission between components in the private cloud collaborative processing platform for internal data and Internet-connected data.

[0013] Optionally, the data integration module includes:

[0014] Internal data source access unit, used to obtain data;

[0015] A data synchronization unit, connected to the internal data source access unit, and used to implement data synchronization by adopting a data synchronization strategy;

[0016] The data mapping and conversion unit is connected to the data synchronization unit and the data cleaning and preprocessing module, and is used to map and convert the synchronized data to obtain the integrated data.

[0017] Optionally, the data synchronization strategy includes: a scheduled synchronization mode and a real-time synchronization mode.

[0018] Optionally, the secure transmission protocol includes: data encryption processing, identity authentication, data integrity verification and data access control.

[0019] Optionally, the data cleaning and preprocessing module includes:

[0020] A data quality inspection unit connected to the data integration module, used to automatically detect the integrated data and perform data cleaning on the detected integrated data to obtain cleaned data;

[0021] A data conversion unit, connected to the data quality inspection unit, is used to perform standardization processing on the cleaned data to obtain standardized processed data;

[0022] A data fusion unit, connected to the data conversion unit and the data exchange area respectively, for associating and integrating the standardized processed data to obtain associated integrated data;

[0023] The data desensitization unit is connected to the data fusion unit and is used to perform a desensitization operation on the associated integrated data to obtain pre-processed data.

[0024] Optionally, the data analysis algorithm adopts a machine learning algorithm; the input of the machine learning algorithm is the preprocessed data, and the output is the data processing result.

[0025] Optionally, the analysis engine processes the preprocessed data using a real-time stream data processing method.

[0026] Optionally, a database is provided in the analysis engine for storing preprocessed data and data processing results.

[0027] According to the specific embodiments provided in this application, this application has the following technical effects:

[0028] The present application provides a private cloud collaborative processing platform for internal data and Internet-connected data. By setting up a data integration module to perform integrated processing on the acquired data, effective integration of internal and external data can be achieved. By setting up a data exchange area to provide a physical isolation mechanism, it can ensure that internal data and the Internet are completely physically isolated, thereby improving data security. By setting up a data integration module, a data cleaning and preprocessing module, and an analysis engine, and providing a collaborative working environment in the analysis engine, the needs of real-time analysis can be met and efficient data processing can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 An architectural diagram of a private cloud collaborative processing platform for internal data and Internet-connected data provided in one embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] In an exemplary embodiment, Figure 1 As shown, a private cloud collaborative processing platform for internal data and Internet-connected data is provided, including: a data integration module, a data cleaning and preprocessing module, a data exchange area and an analysis engine.

[0034] The data integration module is used to acquire data and to perform integrated processing on the acquired data to obtain integrated data.

[0035] The data cleaning and preprocessing module is connected to the data integration module. The data cleaning and preprocessing module is used to clean and preprocess the integrated data to obtain preprocessed data.

[0036] The data exchange area is connected to the data cleaning and preprocessing module. The data exchange area is used to provide a physical isolation mechanism.

[0037] The analysis engine is connected to the data exchange area. The analysis engine is used to provide a collaborative working environment and to use data analysis algorithms to perform data mining and pattern recognition on pre-processed data under a physical isolation mechanism to obtain data processing results. The collaborative working environment is used to support multi-user collaboration.

[0038] Based on the above description, the private cloud collaborative processing platform for internal data and Internet-connected data provided by this application can achieve efficient and secure data processing and collaborative work, while ensuring that internal data is completely physically isolated from the Internet.

[0039] In another exemplary embodiment of the present application, in order to provide intuitive data display and help users understand the analysis results, in this embodiment, a visualization tool can also be set in the private cloud collaborative processing platform for the internal data and the Internet-connected data. The visualization tool is connected to the analysis engine to visualize the data processing results and obtain a visualization processing result. The visualization processing result may include:

[0040] (1) Interactive reports: Provide flexible reporting and dashboard functions to facilitate users to view analysis results.

[0041] (2) Chart display: supports multiple chart types, such as line charts, bar charts, pie charts, heat maps, etc.

[0042] (3) User interface view: Design a friendly user interface so that non-technical personnel can easily operate it.

[0043] (4) Customized views: Allow users to customize their own data views and dashboards according to their needs. The function of users customizing their own data views and dashboards according to their needs usually requires a combination of front-end and back-end technologies to provide a flexible and easy-to-use interface.

[0044] A. The front-end implementation is to use React.js and D3.js to build a user-friendly interface that allows users to select and configure data views and dashboards.

[0045] B. The backend implementation method is: Use Node.js and Express to build a simple backend service to provide a data interface.

[0046] Further, in this embodiment, a database may be used to store and manage data. For example, PostgreSQL may be used to store data.

[0047] Furthermore, in this embodiment, a database or a file may be used to store the user's configuration information, such as the selected chart type, chart parameters, etc.

[0048] By combining front-end and back-end technologies, users can customize their own data views and dashboards according to their needs. Front-end technology is used to build the user interface, back-end technology is used to process data requests and data storage, and data visualization tools provide rich charts and graphics support.

[0049] In another exemplary embodiment of the present application, in order to ensure the security and integrity of data in the internal network, a secure transmission protocol is used between the components in the private cloud collaborative processing platform for internal data and Internet-connected data provided by the present application for data transmission. The secure transmission protocol can achieve the following functions:

[0050] (1) Encryption technology: Use Transport Layer Security (TLS) or other similar encryption technologies to protect the security of data in the internal network. Among them, encryption algorithms such as AES are used to encrypt data for easy storage and transmission.

[0051] (2) Authentication: Implement a two-factor authentication mechanism to ensure that only authorized users can access data.

[0052] (3) Data integrity verification: Digital signature technology is used to ensure that data has not been tampered with during transmission.

[0053] (4) Access control: A role-based access control mechanism that assigns different access levels according to the permissions of different users.

[0054] In another exemplary embodiment of the present application, in order to effectively integrate internal and external data to meet the needs of real-time analysis, in this embodiment, the data integration module provided above may include: an internal data source access unit, a data synchronization unit, and a data mapping and conversion unit.

[0055] The internal data source access unit is used to obtain data. For example, data can be automatically obtained from the internal system through various methods such as API interface and database connection.

[0056] The data synchronization unit is connected to the internal data source access unit. The data synchronization unit is used to implement data synchronization using a data synchronization strategy. The data synchronization strategy can support two modes: scheduled synchronization and real-time synchronization. The appropriate synchronization strategy is selected according to the importance and update frequency of the data.

[0057] The data mapping and conversion unit is connected with the data synchronization unit and the data cleaning and preprocessing module. The data mapping and conversion unit can support the configuration of mapping relationships between data fields and the conversion of data types to obtain integrated data and ensure the consistency and processability of data.

[0058] Furthermore, the data integration module can also support multiple data formats, such as structured data (SQL database), semi-structured data (XML, JSON) and unstructured data (text files, images, videos, etc.).

[0059] In another exemplary embodiment of the present application, in order to improve data quality and prepare for subsequent analysis, in this embodiment, the data cleaning and preprocessing module provided above may include: a data quality checking unit, a data conversion unit, a data fusion unit and a data desensitizing unit.

[0060] The data quality inspection unit is connected to the data integration module to automatically detect the integrated data and perform data cleaning on the detected integrated data to obtain cleaned data. For example, incomplete data such as outliers and missing values ​​can be automatically detected and removed.

[0061] The data conversion unit is connected to the data quality inspection unit and is used to perform standardization processing on the cleaned data to obtain standardized processed data and ensure the consistency of the data.

[0062] The data fusion unit is connected to the data conversion unit and the data exchange area respectively, and is used to associate and integrate the standardized processed data to obtain associated integrated data, wherein the data from different sources are mainly associated and integrated to form a unified data view.

[0063] The data desensitization unit is connected to the data fusion unit to perform desensitization operations on the associated integrated data to obtain pre-processed data to protect personal privacy and commercial secrets. Among them, data desensitization mainly desensitizes sensitive information, such as replacing it with asterisks or random characters.

[0064] In another exemplary embodiment of the present application, the process of associating and integrating data from different sources includes:

[0065] 1) Data matching:

[0066] Use unique identifiers (such as customer ID) or natural keys (such as name, address) to match the same entity in different data sources.

[0067] This process also requires dealing with data conflicts, for example, if the same customer has different phone numbers in different systems, it is necessary to determine which one is the most accurate.

[0068] 2) Data integration:

[0069] Merge the matched data into a comprehensive data set. For example, suppose a company's internal data includes a customer relationship management system (CRM) and a financial system, and also needs to obtain market trend data from the external Internet. Merge the basic customer information in the CRM with the transaction records in the financial system.

[0070] This step can solve data inconsistencies, such as inconsistent units (such as currency units, date formats, etc.).

[0071] 3) Data association:

[0072] Establish relationships between different data sets. For example, associate a customer's purchase history with marketing campaign data to analyze the impact of marketing campaigns on sales.

[0073] In another exemplary embodiment of the present application, in order to utilize advanced data analysis algorithms for data mining and pattern recognition, in this embodiment, the above-mentioned data analysis algorithm may adopt a machine learning algorithm (such as cluster analysis, regression analysis, deep learning, etc.). The input of the machine learning algorithm is preprocessed data, and the output is the data processing result.

[0074] Data analysis algorithms are usually used to perform in-depth analysis on cleaned and preprocessed data to extract valuable information. The input and output of these machine learning algorithms are as follows:

[0075] (1) Input data is usually collected from internal systems and the external Internet and cleaned and pre-processed. Such data may include but is not limited to the following types:

[0076] a. Structured data, including: numerical data and categorical data. Among them, sales, customer age, product price, etc. are numerical data. Customer gender, product category, region, etc. are categorical data.

[0077] b. Semi-structured data, including: JSON / XML data, such as log files, data returned by API, etc.

[0078] c. Unstructured data, including text data, image data, and video data. For example, customer reviews and social media posts are text data. Product pictures and images taken by surveillance cameras are image data. Surveillance videos and product demonstration videos are video data.

[0079] (2) The output data depends on the specific analysis objectives and the machine learning algorithm used. Common output types include:

[0080] a. Prediction of values: The main operations include regression analysis and classification analysis. Regression analysis is to predict continuous values, such as predicting future sales, house prices, etc. Classification analysis is to predict category labels, such as predicting whether a customer will buy a product, whether an email is spam, etc.

[0081] b. Clustering results, which include: Cluster analysis, which is to divide the data into different groups. For example, customers can be divided into different market segments, products can be divided into different categories, etc.

[0082] c. Anomaly detection results, including: Anomaly detection. Anomaly detection is mainly used to identify abnormal points in data, such as detecting credit card fraud transactions, equipment failures, etc.

[0083] d. Feature importance, which includes: Feature selection. Feature selection is mainly to identify the features that have the greatest impact on model predictions, for example, which factors have the greatest impact on sales.

[0084] e. Recommendation system results, including: Recommendation system. The recommendation system mainly recommends personalized products or services to users, such as movies, books, commodities, etc.

[0085] Furthermore, in order to achieve rapid response, the analysis engine provided in the present application can also process the pre-processed data using a real-time streaming data processing method.

[0086] Among them, real-time stream processing can be implemented using Apache Kafka (a high-throughput distributed message queue system, often used to build real-time data pipelines) and Apache Flink (a high-performance stream processing framework that supports batch processing and stream processing, with the characteristics of low latency and high throughput).

[0087] Furthermore, the implementation architecture of the real-time stream processing given above mainly includes:

[0088] (1) Data producer: sends real-time data to Kafka topics.

[0089] (2) Apache Kafka: As a message queue, it stores and delivers real-time data.

[0090] (3) Apache Flink: Reads data from Apache Kafka for real-time processing and analysis.

[0091] (4) Data consumer: Receives the results processed by Apache Flink for further application or storage.

[0092] (5) Data warehouse: Build a high-performance data warehouse to store cleaned and processed data for subsequent query and analysis.

[0093] (6) Distributed computing framework: Use a distributed computing framework (such as Apache Hadoop or Apache Spark) to process large-scale data sets.

[0094] In another exemplary embodiment of the present application, in order to support multi-user collaboration, project management, authority management, message notification and other functions, the collaborative work environment provided by the present application above mainly includes the following functions:

[0095] (1) Multi-user collaboration: Supports multiple users to log in at the same time and perform data processing and analysis. The multi-user collaboration function allows multiple users to log in at the same time and perform data processing and analysis. For example, the multi-user collaboration function is achieved through the following technologies:

[0096] (1.1) User authentication and session management: Ensure the identity authentication and session management of each user.

[0097] (1.2) Real-time communication: Supports real-time communication and collaboration between users.

[0098] Based on the above-mentioned technology, the implementation means include:

[0099] 1-1) OAuth 2.0: used for third-party authentication.

[0100] 1-2) JWT (JSON Web Tokens): used to generate and verify user tokens.

[0101] 1-3) WebSocket: used to achieve real-time communication and supports bidirectional data flow.

[0102] 1-4)Socket.io: A WebSocket-based library that supports cross-browser real-time communication.

[0103] (2) Project Management: Provides project management and version control functions to facilitate team members to track project progress. Project management functions allow users to create, manage and track project progress. For example, project management functions are implemented through the following technologies:

[0104] (2.1) Database: stores project information and tasks.

[0105] (2.2) RESTful API: provides an interface for project management.

[0106] (2.3) Front-end framework: to build the user interface, such as React.js or Vue.js.

[0107] Based on the above-mentioned technology, the implementation means include:

[0108] 2-1)MySQL or PostgreSQL: relational database used to store project and task data.

[0109] 2-2) MongoDB: A NoSQL database used to store flexible project data.

[0110] 2-3) Express.js: used to build RESTful API.

[0111] 2-4) React.js or Vue.js: used to build the front-end user interface.

[0112] (3) Permission management: Implement role-based permission management to ensure data security and compliance. The permission management function ensures that each user can only access and operate the data for which they have permission. For example, the permission management function is implemented through the following technologies:

[0113] (3.1) RBAC (Role-Based Access Control): role-based access control.

[0114] (3.2)ABAC (Attribute-Based Access Control): attribute-based access control.

[0115] Based on the above-mentioned technology, the implementation means include:

[0116] 3-1) Passport.js: used to implement various authentication strategies.

[0117] 3-2) JSON Web Tokens (JWT): used to generate and verify user tokens.

[0118] 3-3) Database: stores user, role and permission information.

[0119] (4) Message notification: Notify users of project status and important events via email or platform messages.

[0120] In another exemplary embodiment of the present application, in order to ensure that external data can be imported normally while not allowing internal data to flow out directly, in this embodiment, the data exchange area set up above can be implemented through a variety of technical means, such as network isolation, firewall rules, data transmission protocols, data access control and audit logs. To ensure physical isolation, a secure data exchange area and a secure gateway can be set up to ensure that there is no direct connection between internal data and the Internet. Based on this, the physical isolation mechanism is implemented through the architectural design of the data exchange area in this application, specifically:

[0121] (1) Data exchange area architecture design. Design a reasonable data exchange area architecture and deploy the data exchange area in an independent network segment, separate from the private cloud environment and the Internet, to ensure data security and availability. For example, import external data into the data exchange area through a secure interface (such as API, SFTP). The functions of the designed data exchange area architecture may include:

[0122] (1.1) Data import: Import external data into the data exchange area through a secure interface (such as API, SFTP).

[0123] (1.2) Data processing: Clean, convert and verify the imported data in the data exchange area.

[0124] (1.3) Data export: Export the processed data to the internal network through a secure interface (such as API, database query).

[0125] Based on the above, the implementation of the data exchange area architecture function can be described as:

[0126] 1. Import external data.

[0127] API interface: Provides a RESTful API interface, allowing external systems to upload data via HTTPS.

[0128] SFTP Server: Provides an SFTP server to allow external systems to upload files via SSH.

[0129] 2. Data processing.

[0130] Data cleaning: Use ETL tools (such as Apache NiFi, Talend) to clean and transform the imported data.

[0131] Data Validation: Use data validation tools (such as Apache Spark, Pandas) to validate and quality check the data.

[0132] 3. Data export.

[0133] API interface: Provides a RESTful API interface, allowing internal systems to query processed data through HTTPS.

[0134] Database query: Allow internal systems to access processed data through the database query interface.

[0135] In another exemplary embodiment of the present application, a security gateway and no Internet connection may be set to implement a physical isolation mechanism.

[0136] Security Gateway: Use a security gateway to strictly filter and audit all incoming and outgoing data.

[0137] No Internet connection: Ensure that there is no direct network connection between the private cloud environment and the Internet. All external data introduced must undergo strict security checks and isolation measures.

[0138] Based on the above, it can be ensured that Internet data undergoes strict security checks before entering the private cloud environment, and there is no direct network connection between the private cloud environment and the Internet.

[0139] In another exemplary embodiment of the present application, in order to achieve physical isolation, a dedicated network and an air gap may also be set up. The dedicated network deploys the private cloud environment in an independent physical network and is not directly connected to the Internet.

[0140] Air gapping completely disconnects networks at the physical level, ensuring there is no physical path to directly access the internet.

[0141] Furthermore, in addition to physical isolation, that is, deploying the private cloud environment in an independent physical network and physically isolating it from the Internet, the following isolation operations can also be performed:

[0142] 1) Logical isolation: Use firewalls and VLANs to separate the data exchange area from the private cloud environment and the Internet. Logical isolation mainly includes:

[0143] (1) Virtual Local Area Network (VLAN): Divide the private cloud environment and data exchange area into different VLANs.

[0144] Network Address Translation (NAT): Use NAT to translate the internal IP addresses of the private cloud environment into external IP addresses, increasing the complexity and security of the network.

[0145] 2) Firewalls and intrusion detection systems, including:

[0146] Firewall rules: Configure strict firewall rules to allow only specific IP addresses and ports to access the data exchange area.

[0147] Intrusion Detection System (IDS): monitors network traffic, detects and alerts on potential attacks.

[0148] 3) Data verification and cleaning: Strictly verify and cleanse external data.

[0149] Data validation: Use data validation tools (such as Apache Spark, Pandas) to validate the data to ensure that the data format is correct and there are no outliers.

[0150] Data cleaning: Use ETL tools (such as Apache NiFi, Talend) to clean data and remove invalid or malicious data.

[0151] 4) Logging and auditing: Record access and operation logs of all data exchange areas for auditing and tracking.

[0152] Logging: Record the time, user, operation, and result of each data access.

[0153] Log analysis: Regularly analyze logs to detect abnormal behavior and potential security threats.

[0154] 5) Two-factor authentication (2FA).

[0155] Two-factor authentication: Use hardware tokens, SMS verification codes, biometrics, etc. for secondary verification.

[0156] Authentication and Authorization: Use protocols such as OAuth and SAML for authentication and authorization.

[0157] Through the above description of this embodiment, a secure data exchange area can be implemented to ensure that there is no direct network connection between the private cloud environment and the Internet, and that all external data introduced must undergo strict security checks and isolation measures. Specifically, through physical isolation, logical isolation, firewalls and intrusion detection systems, data exchange areas, data verification and cleaning, logging and auditing, two-factor authentication, data desensitization and encryption and other technical means, the security of the private cloud environment can be effectively protected.

[0158] In addition, during the internal communication process of the entire platform, data communication is only allowed within the internal network, and any form of external data outflow is prohibited.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0160] Those skilled in the art can understand that all or part of the processes in the above embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0161] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0162] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A private cloud collaborative processing platform for internal data and Internet-connected data, characterized in that: The private cloud collaborative processing platform for internal data and Internet-connected data includes: A data integration module is used to acquire data and perform integrated processing on the acquired data to obtain integrated data; A data cleaning and preprocessing module, connected to the data integration module, for cleaning and preprocessing the integrated data to obtain preprocessed data; A data exchange area, connected to the data cleaning and preprocessing module, for providing a physical isolation mechanism; The analysis engine is connected to the data exchange area and is used to provide a collaborative work environment and to use a data analysis algorithm to perform data mining and pattern recognition on the pre-processed data under the physical isolation mechanism to obtain data processing results; the collaborative work environment is used to support multi-user collaboration.

2. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 1 is characterized in that: The private cloud collaborative processing platform for internal data and Internet-connected data also includes: A visualization tool is connected to the analysis engine and is used to perform visualization processing on the data processing result to obtain a visualization processing result.

3. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 1 is characterized in that: A secure transmission protocol is used for data transmission between components in the private cloud collaborative processing platform for internal data and Internet-connected data.

4. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 1 is characterized in that: The data integration module includes: Internal data source access unit, used to obtain data; A data synchronization unit, connected to the internal data source access unit, and used to implement data synchronization by adopting a data synchronization strategy; The data mapping and conversion unit is connected to the data synchronization unit and the data cleaning and preprocessing module, and is used to map and convert the synchronized data to obtain the integrated data.

5. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 4 is characterized in that: The data synchronization strategy includes: a timing synchronization mode and a real-time synchronization mode.

6. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 3 is characterized in that: The secure transmission protocol includes: data encryption processing, identity authentication, data integrity verification and data access control.

7. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 1 is characterized in that: The data cleaning and preprocessing module includes: A data quality inspection unit connected to the data integration module, used to automatically detect the integrated data and perform data cleaning on the detected integrated data to obtain cleaned data; A data conversion unit, connected to the data quality inspection unit, is used to perform standardization processing on the cleaned data to obtain standardized processed data; A data fusion unit, connected to the data conversion unit and the data exchange area respectively, for associating and integrating the standardized processed data to obtain associated integrated data; The data desensitization unit is connected to the data fusion unit and is used to perform a desensitization operation on the associated integrated data to obtain pre-processed data.

8. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 1 is characterized in that: The data analysis algorithm adopts a machine learning algorithm; the input of the machine learning algorithm is the preprocessed data, and the output is the data processing result.

9. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 1, characterized in that: The analysis engine processes the pre-processed data in a real-time stream data processing manner.

10. The private cloud collaborative processing platform for internal data and Internet-connected data according to claim 9, characterized in that: The analysis engine is provided with a database for storing preprocessed data and data processing results.

Images