P2P cloud management service system and implementation method

Abstract

The invention relates to a P2P cloud management service system and an implementation method, the P2P cloud management service system comprises a rule service module, a multi-weight acquisition service module and a local service rapid deployment module, the rule service module, the multi-weight acquisition service module and the local service rapid deployment module are mutually connected, the rule service module is used for storing services, data and scheduling algorithms, and the multi-weight acquisition service module is connected with the local service rapid deployment module. And the multi-weight acquisition service module is used for matching learning service and business results and updating information in the rule service module, and the local service rapid deployment module is used for service iteration updating and rapid deployment. Through combination of the rule service set, the multi-weight acquisition service and the local service rapid deployment, a comprehensive solution is provided, and the single node fault problem in the cloud service can be effectively solved. The system constructed based on the P2P service has the attribute characteristics of high resource utilization rate, high reliability, decentralization, no single fault node and high expansibility.

Description

Technical Field

[0001] The present invention belongs to the technical field of information system management, and in particular to a P2P cloud management service system and an implementation method thereof. Background Art

[0002] There are many technical solutions for service resource monitoring on the market, which mainly focus on monitoring service resources and service status, and basically only collect capabilities within a certain range. In actual sites, due to the complexity of business and technology, it is difficult to solve the problem based on only one solution and one technical positioning.

[0003] The defects and shortcomings of the existing technologies on the market are:

[0004] 1. Cloud services generally face high QPS concurrent access. Existing monitoring solutions are basically for a certain function, a single point of resource monitoring and analysis. When anomalies occur, they only reflect the anomalies of local services. Problems can only be solved locally and layer by layer. Moreover, comprehensive anomalies often require timely human intervention and point-by-point investigation. The entire problem investigation work has never had a global perspective and is passive.

[0005] 2. Traditional services ultimately rely on on-site status data, problem reproduction, and real-time comprehensive analysis to solve problems. This requires not only human initiative and ability, but also sufficient design in the early stages.

[0006] 3. Traditional information collection is essentially single-point, and collection needs to be designed in advance; the summary and analysis of the collected data often lags behind the accident. When a new problem occurs, there is often no abnormal data collection at the time of the problem, and the old design often cannot solve the new problem;

[0007] 4. Release and verification are often independent. Verification is required before release. It is not possible to verify at the same time as release. Summary of the invention

[0008] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a P2P cloud management service system and implementation method, which provides a comprehensive solution through the combination of rule service set, multi-weighted collection service and local service rapid deployment, and can effectively solve the single node failure problem in cloud services.

[0009] The present invention solves the technical problem by adopting the following technical solutions:

[0010] A P2P cloud management service system includes a rule service module, a multi-weighted collection service module and a local service rapid deployment module, wherein the rule service module, the multi-weighted collection service module and the local service rapid deployment module are interconnected, the rule service module is used to store services, data and scheduling algorithms, and serves as the underlying foundation of the system, the multi-weighted collection service module is used to match the results of learning services and businesses, and update the information in the rule service module, and the local service rapid deployment module is used for iterative updates and rapid deployment of the service system.

[0011] A method for implementing a P2P cloud management service system comprises the following steps:

[0012] Step 1: Build a rule service set through the rule service module;

[0013] Step 2: Based on the rule service set constructed in step 1, and on the basis of data collection and analysis services and complex environment scheduling algorithms, the multi-weighted collection service module constructs a multi-weighted collection service for the redundant mesh service + dynamic routing service system applicable to the Internet;

[0014] Step 3: Based on the rule service set and multi-weighted collection services, the local service rapid deployment module builds a local service rapid deployment mechanism;

[0015] Step 4: Build a system through steps 1 to 3, and provide management services through the system.

[0016] Furthermore, the step 1 comprises the following steps:

[0017] Step 1.1: Build a unified standard hardware and software environment, taking the server as the unit, and unify the hardware used and the software deployed on the hardware;

[0018] Step 1.2: Based on specific business logic, service performance requirements and data interaction conditions, a basic scheduling algorithm is constructed; as the business complexity and scenario complexity increase, the basic scheduling algorithm is combined with the weight strategy to form a weighted scheduling algorithm;

[0019] Step 1.3: Based on the unified standard hardware and software environment in step 1.1 and the weight scheduling algorithm in step 1.2, extract service features, including hardware features, software service features, and data features to form a rule service set.

[0020] Furthermore, the step 2 comprises the following steps:

[0021] Step 2.1: Based on the data source of the rule service set constructed in the rule service module, service, operation data and basic software and hardware environment information are collected in real time. The collected service data includes performance data, log information and routing path.

[0022] Step 2.2: At the beginning of the service, a basic rule service set is preset based on experience; based on this rule service set, iterative learning is performed based on the data collected in step 2.2, new rule service set data is output, and the original service set is solidified and enriched to complete self-learning;

[0023] Step 2.3: Construct a complex environment scheduling algorithm based on the data set in step 2.2. The complex environment scheduling algorithm outputs scheduling rules that match the business based on service deployment and service logic. According to the complexity of the environment, the weights are dynamically adjusted to achieve the target business interaction logic and form a scheduling algorithm.

[0024] Furthermore, the step 3 comprises the following steps:

[0025] Step 3.1: Integrate the rule service set and multi-weighted collection service into the Docker image to ensure the consistency and repeatability of the service;

[0026] Step 3.2: Relying on the multi-weighted collection service, dynamically update the Docker image and deploy it based on the learning feedback of the collected data on the deployed service to achieve release as a service;

[0027] Step 3.3: Based on Docker technology, quickly package and deploy services to ensure the consistency of service deployment and achieve rapid deployment of local services.

[0028] The advantages and positive effects of the present invention are:

[0029] 1. The present invention includes a rule service module, a multi-weighted acquisition service module and a local service rapid deployment module, wherein the rule service module, the multi-weighted acquisition service module and the local service rapid deployment module are interconnected, the rule service module is used to store services, data and scheduling algorithms, and serves as the underlying foundation of the system, the multi-weighted acquisition service module is used to match the results of learning services and businesses, and update the information in the rule service module, and the local service rapid deployment module is used for service iteration update and rapid deployment. The present invention provides a comprehensive solution through the combination of rule service set, multi-weighted acquisition service and local service rapid deployment, which can effectively solve the single node failure problem in cloud services. At the same time, the system constructed based on the P2P service of the present invention itself has the attributes of high resource utilization, high reliability, decentralization, no single failure node, and high scalability.

[0030] 2. The multi-weighted acquisition service used in the present invention is itself based on a learning service model and a training model for a complex business environment. Its own feedback mechanism enables the service to dynamically adjust weights in a complex environment, optimize service quality, and improve system adaptability.

[0031] 3. The local service rapid deployment mechanism constructed by the present invention is an integrated release mechanism based on the Docker technology, which realizes the rapid deployment and consistency verification of services and simplifies the system maintenance process.

[0032] 4. The present invention optimizes the overall service quality according to the actual service status by dynamically adjusting the service weight, thereby improving the flexibility and maintainability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 An overall schematic diagram of the rule service set of the present invention;

[0034] Figure 2 This is the overall flow chart of the multi-weighted acquisition service of the present invention;

[0035] Figure 3 The overall flow chart of the rapid deployment of local services of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below with reference to the accompanying drawings.

[0037] Peer-to-peer (P2P) service is a distributed computing technology in which each node (peer) is both a client and a server, and can communicate directly with other nodes without the need for data transmission through a central server. P2P services are widely used in file sharing, streaming media, VoIP (Internet phone), blockchain and other fields.

[0038] The characteristics of P2P services are: distributed architecture, decentralization, support for resource sharing, support for dynamic topology, and reliable security.

[0039] Key technologies include: network topology, routing algorithms, and data distribution; common P2P network topologies include pure P2P networks (such as Gnutella), structured P2P networks (such as Chord and Kademlia), and hybrid P2P networks. How nodes find other nodes and how to pass messages and distribute data are one of the core issues of P2P systems. Typical routing algorithms include Chord and Kademlia; data distribution and replication algorithms in P2P systems, such as the BitTorrent protocol, ensure efficient data distribution and reliability. In terms of security, it mainly protects the communication security and data privacy between nodes, and prevents attacks from malicious nodes and data tampering.

[0040] A P2P cloud management service system includes a rule service module, a multi-weighted collection service module and a local service rapid deployment module, wherein the rule service module, the multi-weighted collection service module and the local service rapid deployment module are interconnected, the rule service module is used to store services, data and scheduling algorithms, and serves as the underlying foundation of the system, the multi-weighted collection service module is used to match the results of learning services and businesses, and update the information in the rule service module, and the local service rapid deployment module is used for iterative updates and rapid deployment of the service system.

[0041] The rule service set is a collection of services, data and scheduling algorithms, and is the foundation of services; the multi-weighted collection service mechanism is the foundation for the operation of the entire service system and the result of learning about service and business matching. At the same time, this learning enriches the rule service set and the multi-weighted collection service mechanism; the local service rapid deployment mechanism provides solutions for service iteration updates and rapid deployment.

[0042] The rule service set is an abstraction of services in business diversity, a collection of services that constitute many software functions of business objectives, and the output product of the combination of services and weight scheduling algorithms. As the complexity of business scenarios is learned, new differences become new samples of the rule service set, constantly supplementing and improving the capabilities of the service set.

[0043] Multi-weighted collection service is a rule-based service set. It is based on data collection and analysis services and complex environment scheduling algorithms. It is suitable for the Internet's redundant mesh service + dynamic routing service system. Through multi-weighted collection service processing, the service can find a new path based on the rule service set to complete the final data interaction when an exception occurs.

[0044] The local service rapid deployment mechanism can effectively reduce release risks.

[0045] A method for implementing a P2P cloud management service system comprises the following steps:

[0046] Step 1: Build a rule service set through the rule service module;

[0047] Step 1.1: Build a unified standard hardware and software environment, taking the server as the unit, and unify the hardware used and the software deployed on the hardware;

[0048] Step 1.2: Based on specific business logic, service performance requirements and data interaction conditions, a basic scheduling algorithm is constructed; as the business complexity and scenario complexity increase, the basic scheduling algorithm is combined with the weight strategy to form a weighted scheduling algorithm;

[0049] Step 1.3: Based on the unified standard hardware and software environment in step 1.1 and the weight scheduling algorithm in step 1.2, extract service features, including hardware features, software service features, and data features to form a rule service set.

[0050] Step 2: Based on the rule service set constructed in step 1, and on the basis of data collection and analysis services and complex environment scheduling algorithms, the multi-weighted collection service module constructs a multi-weighted collection service for the redundant mesh service + dynamic routing service system applicable to the Internet;

[0051] Step 2.1: Based on the data source of the rule service set constructed in the rule service module, service, operation data and basic software and hardware environment information are collected in real time. The collected service data includes performance data, log information and routing path.

[0052] Step 2.2: At the beginning of the service, a basic rule service set is preset based on experience; based on this rule service set, iterative learning is performed based on the data collected in step 2.2, new rule service set data is output, and the original service set is solidified and enriched to complete self-learning;

[0053] Step 2.3: Construct a complex environment scheduling algorithm based on the data set in step 2.2. The complex environment scheduling algorithm outputs scheduling rules that match the business based on service deployment and service logic. According to the complexity of the environment, the weights are dynamically adjusted to achieve the target business interaction logic and form a scheduling algorithm.

[0054] Step 3: Based on the rule service set and multi-weighted collection services, the local service rapid deployment module builds a local service rapid deployment mechanism;

[0055] Step 3.1: Integrate the rule service set and multi-weighted collection service into the Docker image to ensure the consistency and repeatability of the service;

[0056] Step 3.2: Relying on the multi-weighted collection service, dynamically update the Docker image and deploy it based on the learning feedback of the collected data on the deployed service to achieve release as a service;

[0057] Step 3.3: Based on Docker technology, quickly package and deploy services to ensure the consistency of service deployment and achieve rapid deployment of local services.

[0058] Step 4: Build a system through steps 1 to 3, and provide management services through the system.

[0059] According to the above-mentioned P2P cloud management service system and implementation method, the effect of the present invention is verified by building a corresponding system and performing tests.

[0060] like Figure 1As shown, the specific implementation method of forming a rule service set is:

[0061] After building a unified standard software and hardware environment, feature collection is performed. Feature collection mainly relies on feature extraction algorithms and tools. Feature data is predefined in combination with software and hardware resources, log collection, and service capabilities. After being processed by feature algorithms, feature data is stored in the database to form a rule service set. The rule service set is a collection of data, paths, and rules.

[0062] Instructions for building a unified standard software and hardware environment:

[0063] Hardware selection: Choose a standardized hardware platform, such as an x86 architecture server, to ensure hardware compatibility and scalability.

[0064] Operating system: Choose a popular and stable server operating system, such as Linux distributions CentOS or Ubuntu, to ensure system stability and security.

[0065] Software environment: The software environment is uniformly configured, including P2P technology, Prometheus, Random Forest and Gradient Boosting Tree algorithms, using log systems (such as ELK Stack), data collection engines (Telegraf and Logstash), Grafana+Loki+Prometheus to build visual data maps, docker mechanism and related technologies (Docker Compose or Kubernetes), Java technology, Python technology, MySQL technology, multiple message queues (RabbitMQ, Kafka and mqtt), weighted least response time scheduling (Weighted Least Response Time, WLRT), weighted least connection scheduling (Weighted Least Connections, WLC) and self-developed streaming media services, including self-implemented software such as streaming media gateway, P2P basic services, and a software environment based on P2P services.

[0066] Weighted scheduling algorithm generation.

[0067] Data collection tool: Deploy Prometheus as a data collection tool to monitor the performance indicators of service nodes in real time, such as CPU usage, memory utilization, network bandwidth, MQ messages, P2P interaction messages, media interaction messages, etc.

[0068] Feature extraction algorithm: Use machine learning algorithms, improved on the basis of random forest and gradient boosting tree algorithms, to collect services that have successfully completed data interaction flows and extract key service features.

[0069] Scheduling algorithm: Weighted Least Response Time (WLRT) and Weighted Least Connections (WLC) are two commonly used scheduling algorithms in the system;

[0070] The basic scheduling algorithm is based on specific business logic, service performance requirements, and data interaction conditions. As the complexity of business and scenarios increases, the basic scheduling algorithm needs to be combined with the weight strategy to form a weighted scheduling algorithm;

[0071] Generate rule service set description:

[0072] Based on the unified standard hardware and software environment and weighted scheduling algorithm, service features are extracted, including hardware features, software service features, and data features, to form a rule service set. The rule service set is an abstraction of services in business diversity, a collection of services that constitute many software functions of business goals, and the output product of the combination of services and weighted scheduling algorithms. As the complexity of business scenarios is learned, new differences become new samples of the rule service set, which continuously supplements and improves the capabilities of the service set.

[0073] Rule service set verification: Verify the generated rule service set to ensure its accuracy and reliability.

[0074] like Figure 2 As shown, the specific implementation method of the multi-weighted collection service is:

[0075] It is a redundant mesh service + dynamic routing service system suitable for the Internet based on a rule-based service set, based on data collection and analysis services and complex environment scheduling algorithms. Through multi-weighted collection service processing, the service can find a new path based on the rule-based service set to complete the final data interaction when an abnormality occurs.

[0076] Data collection and analysis services:

[0077] Data source: Based on the data source, services, operation data and basic software and hardware environment information are collected in real time, including performance data, log information, routing paths, etc.

[0078] Data collection tools: Deploy data collection tools such as Telegraf and Logstash to monitor and collect the operation data of service nodes in real time.

[0079] Rule Service Set

[0080] At the beginning of deploying services, it is necessary to preset a basic rule service set based on experience. Based on this rule service set, iterative learning is carried out according to the collected data, and new rule service set data is output and the original service set is solidified and enriched to complete self-learning.

[0081] Complex environment scheduling algorithm

[0082] Based on the service deployment and service logic, the scheduling rules that match the business are output. According to the complexity of the environment, the weights are dynamically adjusted to achieve the target business interaction logic and form a scheduling algorithm.

[0083] Environmental monitoring: Real-time monitoring of the complexity of the service environment, such as network latency, load balancing, etc.

[0084] Dynamic weight adjustment: According to the environmental monitoring results, the weight of the service node is dynamically adjusted to adapt to changes in complex environments.

[0085] Service quality optimization: By dynamically adjusting weights, the overall service quality is optimized and the system's adaptability and stability are improved.

[0086] Grafana+Loki+Prometheus generates a visual data map

[0087] Grafana:

[0088] Introduction: Grafana is an open source data visualization tool that supports multiple data sources, including logs, indicators, etc.

[0089] Function: Grafana can generate real-time charts and monitoring panels to show the path through which data flows.

[0090] Loki:

[0091] Introduction: Grafana Loki is a log aggregation system designed for storing and querying log data.

[0092] Functionality: Loki is able to capture log streams and store them in a queryable database.

[0093] Prometheus:

[0094] Introduction: Prometheus is an open source monitoring and alerting tool that focuses on time series data.

[0095] Function: Prometheus can collect system monitoring data and generate charts and alarms.

[0096] When used together for message processing, Loki captures log streams, Prometheus collects system metrics, and Grafana generates graphs and monitoring panels to show the path of data flow.

[0097] The specific implementation method of the local service rapid deployment mechanism is:

[0098] The deployment mechanism is based on the docker mechanism, integrating its own collection, analysis and processing services, adjusting the basic data itself in real time, and effectively solving the problem that release and testing cannot be carried out at the same time in conventional software releases - full self-testing is required before release, and the business cannot bear the risk of service abnormalities. With this mechanism, service deployment performs self-path optimization selection through the rule service set. In the worst case, the previous version of normal service can be quickly deployed through docker to achieve normal service, effectively improving the processing speed of service abnormalities and effectively improving the fault tolerance against service abnormalities.

[0099] Integrate and solidify existing capabilities:

[0100] Algorithm code: Write the service algorithm code as an executable program to ensure the repeatability and consistency of the algorithm.

[0101] Docker image building: Use Dockerfile to define the Docker image building process and integrate the service algorithm into the Docker image.

[0102] Image verification: Verify the built Docker image to ensure that the service algorithm rules it contains are correct.

[0103] Service self-iteration update:

[0104] Service modularization: The cloud management service system is divided into multiple service modules, and each module is independently packaged as a Docker image.

[0105] Module dependency management: Use Docker Compose or Kubernetes to manage the dependencies of service modules to ensure that modules work together.

[0106] Image repository: Upload the packaged Docker image to an image repository (such as Docker Hub, Harbor) to facilitate subsequent service deployment.

[0107] Local service rapid deployment mechanism:

[0108] Environment preparation: Prepare the Docker operating environment in the target environment, including Docker Engine and Kubernetes cluster.

[0109] Image deployment: Deploy Docker images to the target environment through Kubernetes or Docker Swarm to ensure rapid deployment and consistency verification of services.

[0110] Service monitoring: Deploy monitoring tools such as Prometheus and Grafana to monitor the running status of services in real time and ensure high availability and stability of services.

[0111] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. A P2P cloud management service system, characterized in that: It includes a rule service module, a multi-weighted acquisition service module and a local service rapid deployment module, wherein the rule service module, the multi-weighted acquisition service module and the local service rapid deployment module are interconnected. The rule service module is used to store services, data and scheduling algorithms, and serves as the underlying foundation of the system. The multi-weighted acquisition service module is used to match the results of learning services and businesses, and update the information in the rule service module. The local service rapid deployment module is used for iterative updates and rapid deployment of the service system.

2. A system for implementing the P2P cloud management service system according to claim 1, characterized in that: The following steps are involved: Step 1: Build a rule service set through the rule service module; Step 2: Based on the rule service set constructed in step 1, and on the basis of data collection and analysis services and complex environment scheduling algorithms, the multi-weighted collection service module constructs a multi-weighted collection service for the redundant mesh service + dynamic routing service system applicable to the Internet; Step 3: Based on the rule service set and multi-weighted collection services, the local service rapid deployment module builds a local service rapid deployment mechanism; Step 4: Build a system through steps 1 to 3, and provide management services through the system.

3. The method for implementing a P2P cloud management service system according to claim 2, characterized in that: The step 1 comprises the following steps: Step 1.1: Build a unified standard hardware and software environment, taking the server as the unit, and unify the hardware used and the software deployed on the hardware; Step 1.2: Based on specific business logic, service performance requirements and data interaction conditions, a basic scheduling algorithm is constructed; as the business complexity and scenario complexity increase, the basic scheduling algorithm is combined with the weight strategy to form a weighted scheduling algorithm; Step 1.3: Based on the unified standard hardware and software environment in step 1.1 and the weight scheduling algorithm in step 1.2, extract service features, including hardware features, software service features, and data features to form a rule service set.

4. The method for implementing a P2P cloud management service system according to claim 2, characterized in that: The step 2 comprises the following steps: Step 2.1: Based on the data source of the rule service set constructed in the rule service module, service, operation data and basic software and hardware environment information are collected in real time. The collected service data includes performance data, log information and routing path. Step 2.2: At the beginning of the service, a basic rule service set is preset based on experience; based on this rule service set, iterative learning is performed based on the data collected in step 2.2, new rule service set data is output, and the original service set is solidified and enriched to complete self-learning; Step 2.3: Construct a complex environment scheduling algorithm based on the data set in step 2.

2. The complex environment scheduling algorithm outputs scheduling rules that match the business based on service deployment and service logic. According to the complexity of the environment, the weights are dynamically adjusted to achieve the target business interaction logic and form a scheduling algorithm.

5. The method for implementing a P2P cloud management service system according to claim 2, characterized in that: The step 3 comprises the following steps: Step 3.1: Integrate the rule service set and multi-weighted collection service into the Docker image to ensure the consistency and repeatability of the service; Step 3.2: Relying on the multi-weighted collection service, dynamically update the Docker image and deploy it based on the learning feedback of the collected data on the deployed service to achieve release as a service; Step 3.3: Based on Docker technology, quickly package and deploy services to ensure the consistency of service deployment and achieve rapid deployment of local services.

Images