Full-link monitoring and operation and maintenance management method for point mall supply chain platform under multi-cloud architecture

Abstract

The invention discloses a full-link monitoring and operation and maintenance management method for a point mall supply chain platform under a multi-cloud architecture, which can monitor full-link nodes of the supply chain platform in real time and provide a basis for optimizing resource configuration and improving system performance. The monitoring data is analyzed and processed through the star large model, and when the monitoring data of the system is found to be abnormal, an alarm notice can be sent out in time to remind operation and maintenance personnel to carry out fault processing, so that problems during use of a user are avoided. When it is monitored that some resources are low in long-term utilization rate and zombie applications may occur, an alarm notice is sent to remind operation and maintenance personnel to optimize resource allocation, and cost is saved. Stable operation of the point shopping mall supply chain platform is guaranteed, and user experience is improved. The stable platform operation and the timely fault processing can improve the system performance, optimize the operation cost, enhance the credibility and loyalty of the user to the platform, and promote the improvement of the activeness and retention rate of the user. According to the full-link monitoring and fault processing method under the multi-cloud architecture, unified management and monitoring of a plurality of cloud platforms can be realized, so that the management process is simplified, the management cost is reduced, and the flexibility and expandability of the whole system are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the fields of multi-cloud architecture, supply chain platform operation and maintenance, security management, data analysis and artificial intelligence, in particular to a method for full-link monitoring and operation and maintenance management of a supply chain platform of a points mall under a multi-cloud architecture. BACKGROUND

[0002] The exchange demand of the points mall continuously drives the commodity procurement and inventory management of the supply chain platform, and the user behavior data of the points mall provides important market feedback for the supply chain platform, which continuously optimizes the commodity structure and inventory management strategy, thereby continuously improving the user experience of the points mall and promoting business growth. At present, cloud computing technology has the advantages of flexible expansion, high availability and on-demand payment, which provides enterprises with more flexible, efficient and economical choices in IT infrastructure. With the popularization of cloud computing technology, the supply chain platform actively embraces cloud native technology. In order to better avoid risks, realize optimal allocation of resources, and ensure platform stability, the supply chain platform adopts multi-cloud architecture to disperse risks. By fully utilizing its micro-service architecture, containerized deployment, DevOps and other characteristics, the urgent needs of the supply chain platform for independent development, rapid deployment and continuous delivery are met, but at the same time, some new problems and challenges are also accompanied.

[0003] 1. Increased platform complexity: With the adoption of multi-cloud architecture, especially the promotion of micro-service architecture, the original single system is split into multiple independent, loosely coupled independent business units. However, this makes it more complex to sort out the dependencies between various services and cloud environments, and security protection, resource allocation and other issues become particularly important. Therefore, during the daily maintenance of the platform, a large amount of time and effort of operation and maintenance engineers is required.

[0004] 2. Difficulty in locating platform faults and security issues Under the multi-cloud architecture, micro-services have brought about a sharp increase in the number of application services and middleware, exacerbating the complexity of communication and governance between cloud environments, middleware and services. This complexity significantly increases the dimensions of security risks. Once a fault occurs in a certain link of the platform, it will cause a chain reaction and trigger a series of related derivative problems, and it is difficult for operation and maintenance engineers to analyze the root cause of the problem and timely and accurately find the fault point.

[0005] 3. Low resource utilization In a multi-cloud architecture, low resource utilization, zombie hosts, and uneven resource allocation are common problems. Some resources on the cloud platform are not fully utilized, resulting in resource waste. Services that have stopped running or are no longer needed still occupy cloud resources but do not generate any business value, consuming resources while increasing the enterprise's cloud consumption costs. Resources purchased in the cloud environment are not reasonably allocated, resulting in some services or applications being limited in performance due to insufficient resources, while other services or applications have too many resources and are not fully utilized, which can lead to a decline in overall system performance and increase the complexity of operation and maintenance.

[0006] 4. Missing and disordered monitoring data After the implementation of a multi-cloud architecture, operation and maintenance engineers can monitor the response logs of each service and some open-source monitoring software for some indicators of the platform. However, monitoring cannot fully meet the monitoring needs of the supply chain platform, covering all process components of the system, especially the interaction points across multiple services or platforms. These monitoring blind spots can be the source of faults and security issues. When the platform encounters an exception, these monitors can issue alerts, but the resulting alert information is chaotic, reducing the availability of alert information.

[0007] 5. Unable to provide early warnings and accurate solutions In a multi-cloud architecture, existing monitoring alerts are triggered after a fault occurs, and operation and maintenance engineers cannot provide early warnings of the fault. Even if the fault occurs and the problem is successfully located, an accurate solution cannot be formed. Incorrect solutions not only fail to solve the problem but also trigger secondary platform failures, further exacerbating the difficulty of operation and maintenance.

[0008] The above problems, although link tracking technology can master the calling relationship between microservices and automatic fault location tools can be used for platform monitoring, but when the platform fails, due to monitoring blind spots, insufficient data mining capabilities, and missing solutions, alert bombardment and alert loss may still occur, making it difficult for the points mall supply chain platform to accurately locate and handle faults. SUMMARY

[0009] Module 1: Full-link monitoring (attached Figure 1 : S01)

[0010] The module focuses on the implementation of a comprehensive monitoring solution for the integral mall supply chain platform in a multi-cloud architecture, ensuring that every link from the front-end page display, core application, internal and external associated interfaces, middleware services, network communication, to the underlying cloud environment basic resources is included in a seamless monitoring system. Through full-link and dead-angle-free coverage strategies, every link of the platform can be effectively monitored, providing timely and accurate positioning of platform security and fault problems, thereby ensuring platform use experience. System maintenance, security protection, and utilization can be monitored in three dimensions.

[0011] The system maintenance monitoring is as follows: First, starting from the front-end page, simulate the actual business scenario, and establish page monitoring for the entire business process. The page is monitored and recorded regularly, including response time, response status (HTTP status code such as 200, 302), and whether the HTML elements have changed. Second, the services relied on by the front-end application are comprehensively monitored, covering interface response status, interface response time, and interface exception log quantity, ensuring that the running status of each service can be monitored in real time. Subsequently, to avoid the impact of external capability exceptions on the supply chain platform, the external interfaces relied on by the services are also monitored, including response status, interface response time, and interface exception logs. Internal services also rely on some middleware to achieve functionality and optimize platform performance, such as Redis, MongoDB, Elasticsearch, RabbitMQ, etc. The high availability of these middleware is crucial to ensuring the stability of the supply chain platform. We need to monitor related middleware, including state, query speed, cache hit rate, memory occupancy, connection number, slow query number, and other key resource indicators to ensure the high performance and stability of the cloud environment. At the same time, monitoring the cloud environment is also indispensable, and the CPU usage, memory usage, disk usage, and ingress / egress traffic of the cloud server should also be monitored. Finally, network is the basis and core of communication between cloud environments, and network device monitoring is crucial for the stability of the multi-cloud architecture platform. From the communication level, we should monitor network availability, latency, packet loss rate, network bandwidth, and throughput. For network servers, we need to monitor CPU usage, disk space, memory usage, connection number, etc.

[0012] The specific aspects of security protection monitoring are as follows: first, for the Ingress traffic of each cloud environment, continuously monitor abnormal response status codes and IP access frequency, and organize them according to time series to track and analyze security risks. In particular, the frequent occurrence of abnormal HTTP status codes and the high access frequency of a certain IP address indicate that the server has abnormal behavior or security problems. At the same time, monitoring resources in the CDN (Content Delivery Network) server is also crucial, monitoring the integrity of static resources in the server, including but not limited to HTML, CSS, JS, etc., to ensure that relevant files have not been tampered with by third parties. This aspect largely monitors the security of the platform and content. Finally, the user behavior habits collected in multiple dimensions on the front-end page are integrated to form a user portrait, and then the user behavior is analyzed through a large model to identify the general rules and trends of user behavior. Based on the above trends, define related baselines to quickly perceive suspicious or illegal behavior.

[0013] The specific aspects of utilization rate monitoring are as follows: when the platform utilization rate is too low, dynamically release or allocate resources to avoid resource waste. When the platform utilization rate is too high, dynamically expand to avoid resource shortages causing platform failures.

[0014] Module 2: Monitoring data collection (Appendix Figure 1 : S02)

[0015] The main function of this module is to standardize and collect data formed by platform monitoring. First, build an Elasticsearch cluster isolated from the supply chain platform to ensure the independence of monitoring data. Through isolation, avoid potential interference of monitoring data on the operation of the supply chain platform. Through scripting, collect real-time performance data from applications, middleware, and cloud services through HTTP request API interfaces, GraphQL, or other protocols; collect platform log information through log management tools. Subsequently, this module will govern the original data collected by monitoring. This step designs data standardization to process different format data from different links into a unified data format, and number the monitoring components on each cloud environment and link. For example: A cloud environment number 1000, A cloud environment Redis number 1002.

[0016] The final processed data forms a standard log log format, including link number, link name, previous link number, response time, exception, related terms, response code, status code, timeout, etc. Specific examples are as follows: A cloud environment Redis, number 1002, previous link number: 1000, response time **s, exception reason: too many slow queries **, current instance indicators: memory ***, cache hit rate ****.

[0017] The standardized data facilitates subsequent data analysis and utilization. Finally, the completed data is stored in the built Elasticsearch cluster.

[0018] Module 3: Monitoring data cleaning, analysis and mining (attached Figure 1 : S03) The main function of this module is to analyze and process the collected monitoring data through the formation of a large model to locate problems in the platform. First, a set of abnormal monitoring index models is constructed for each monitoring point of the supply chain platform. The abnormal index model is as follows:

[0019] In combination with the above model, the monitoring data in the Elasticsearch cluster and the threshold are checked at regular intervals to obtain the current monitoring index value, which is then compared with the preset threshold. At the same time, it needs to be determined whether the value exceeding the abnormality is due to the jitter generated within a certain interval, causing the abnormality. By using statistical analysis methods such as calculating the moving average and standard deviation, the stability and volatility of the data are evaluated. If the current value deviates significantly from the recent average level and this deviation is not caused by a sustained trend, we can consider this abnormality to be accidental and ignore this abnormal information. The reference code is as follows:

[0020] This module uses a sliding window to divide the data block for the processed data stream of monitoring information, and divides each monitoring data window. Once it is found that the monitoring data deviates from the preset index range, it can quickly capture these abnormal data and store them in the platform's abnormal information database.

[0021] Secondly, the star model is used to analyze the business logic of the platform in depth, and a comprehensive dependency relationship network is constructed to ensure the integrity and clarity of the link association relationship.

[0022] When the platform encounters a fault, in order to ensure the accuracy of the alarm information, especially in the case of multiple fault problems faced by the platform, which may cause cross-positioning difficulties, an alarm model is formed to standardize and require alarm data.

[0023] With the previously constructed dependency relationship network, the star model is continued to rely on to deeply comb and analyze the emerging abnormal data, quickly and accurately lock the fault node, effectively avoiding the problem of fault positioning delay caused by the wrong peak of multiple node alarms.

[0024] At the same time, through the star model combined with the actual situation, the link is continuously optimized and analyzed to optimize the existing problems.

[0025] For example: if the database storing order information encounters I / O latency, triggering a chain reaction that causes the supply chain management system to respond slowly when querying the order list, this chain reaction may cause the front-end page to respond timeout, the downstream services associated with the order also face the problem of response timeout, abnormal log surge, and database I / O latency extension, etc. At this time, the star model quickly intervenes, accurately identifies the root cause of the problem through comprehensive analysis and comparison of these abnormal data, and provides a clear direction and basis for subsequent fault resolution.

[0026] Module 4: Fault warning and governance approach (attached Figure 1 : S04) Figure 1 is the full-link monitoring and operation and maintenance management process diagram of the integral mall supply chain platform under the cloud architecture

[0027] The main function of this module is to use the star model to deeply iterate and analyze the alarm information in the abnormal information database, significantly improving the system's early warning capability. Through this optimization process, when the platform has a problem, it can quickly detect and accurately locate potential problems. Further, combined with the pre-set fault handling strategy in the monitoring model, the star model can continuously iterate and optimize and upgrade these handling schemes, making them increasingly perfect, forming a precise and efficient solution system for operation and maintenance engineers. Similarly, when some indicators of the platform only show slight fluctuations and have not yet constituted a significant impact, but the core monitoring indicators are about to fail, the star model can implement dynamic monitoring and acute perception, so that it can form a platform fault pre-processing scheme in advance before the fault actually occurs, effectively avoiding the possibility of failure, and ensuring the stable operation of the system.

[0028] By connecting various operation and maintenance capabilities, based on the star model, the operation and maintenance engineers only need to efficiently review the fault solution generated by the star model. Once the review is passed, the scheme is automatically executed to achieve immediate and automated solution to platform failure.

[0029] Module 5: Monitoring fault alarm (attached Figure 1 : S05) The main function of this module is to complete the alarm data cleaning, mining and analysis, and then alarm through email, SMS, phone and other forms after locating the platform fault information. Based on the alarm model, the alarm information is formed. The higher the fault level in the alarm information, the more timely the operation and maintenance personnel are informed by phone, such as network, "create order success log" number, etc. The fault level in the alarm information is low, and the operation and maintenance personnel are informed by SMS, email, etc. such as response timeout of services that have no impact on platform stability and user use due to abnormal fluctuations, etc.

Claims

1. A method for end-to-end monitoring and operation and maintenance management of an points mall supply chain platform under a multi-cloud architecture, characterized in that: A seamless monitoring system is implemented, encompassing every aspect of the platform, from front-end page display and core applications to internal and external interfaces, middleware services, network communication, and even the underlying cloud infrastructure. Through a comprehensive, end-to-end coverage strategy, every stage of the platform is effectively monitored. Monitoring data is processed into a unified format and stored in an Elasticsearch cluster isolated from the supply chain platform, ensuring data independence and preventing potential interference with the platform's operation. Anomaly monitoring indicator models and alarm models are built for each monitoring point on the supply chain platform. A large-scale model is used to analyze the platform's business logic and construct a dependency network. Anomaly data is analyzed using this large-scale model to quickly and accurately pinpoint faulty nodes, effectively mitigating delays in fault location caused by staggered alarm times from multiple nodes. The system also dynamically senses relevant platform indicators, proactively predicting potential risks. Finally, fault information is alerted via email, SMS, and telephone, providing solutions or one-click solutions to operations engineers. This invention includes a full-link monitoring module, a monitoring data collection module, a monitoring data cleaning, analysis and mining module, a fault early warning and management module, and a monitoring fault alarm module.

2. A method for end-to-end monitoring and operation and maintenance management of a points mall supply chain platform under a multi-cloud architecture as described in claim 1, characterized in that: The supply chain platform end-to-end monitoring module integrates every link, from front-end page display, core applications, internal and external interfaces, middleware services, network communication to the underlying cloud environment infrastructure, into a seamless monitoring system. This allows for effective monitoring of every aspect of the platform, providing a foundation for identifying platform performance and security risks.

3. A method for end-to-end monitoring and operation and maintenance management of a points mall supply chain platform under a multi-cloud architecture as described in claim 1, characterized in that: Monitoring data collection module: Processes monitoring data into a unified data format and stores it in an Elasticsearch cluster isolated from the supply chain platform. This ensures the independence of the monitoring data and avoids potential interference with the operation of the supply chain platform.

4. A method for end-to-end monitoring and operation and maintenance management of a points mall supply chain platform under a multi-cloud architecture as described in claim 1, characterized in that: The monitoring data cleaning, analysis, and mining module constructs anomaly monitoring indicator models and monitoring alarm models for each monitoring point on the supply chain platform. It utilizes the Xingchen Big Data Model to perform business logic analysis on the platform and build a dependency network. Anomaly data is processed and analyzed through the Big Data Model to quickly and accurately pinpoint faulty nodes, effectively avoiding delays in fault location caused by staggered alarm times from multiple nodes. Furthermore, it dynamically senses relevant platform indicators, proactively predicting potential risks to the platform.

5. A method for end-to-end monitoring and operation and maintenance management of a points mall supply chain platform under a multi-cloud architecture as described in claim 1, characterized in that: The monitoring fault alarm module and fault early warning and management module utilize the Xingchen Big Data Model to perform in-depth iterative learning and analysis of alarm information in the anomaly information database, improving the system's early warning capabilities. Combined with pre-defined fault handling strategies within the monitoring model, the Xingchen Big Data Model can continuously iterate, optimize, and upgrade these solutions, providing operations engineers with a precise and efficient solution system. When certain platform metrics show only minor fluctuations without significant impact, but core monitoring metrics show signs of failure, the Xingchen Big Data Model can implement dynamic monitoring and keen perception, thereby generating platform fault pre-processing plans before the actual fault occurs.

6. A method for end-to-end monitoring and operation and maintenance management of a points mall supply chain platform under a multi-cloud architecture as described in claim 1, characterized in that: Monitoring fault alarm module: It can send alarms via email, SMS, telephone and other means to locate fault information of the platform.

Images