Object storage monitoring method

Abstract

The invention provides an object storage monitoring method, and belongs to the technical field of monitoring storage, and the method comprises the steps: 1, obtaining a monitoring demand of a storage system, and obtaining a corresponding distributed monitoring tool; 2, configuring a proper monitoring rule based on an actual storage service scene according to a distributed monitoring tool, and detecting a plurality of statistical functions of the file system in real time; 3, acquiring real-time performance data of the storage system according to the plurality of statistical functions, and monitoring the working state of the storage system in real time; 4, determining the health condition of object storage according to the working state, and obtaining the abnormity and potential reasons of the storage system; and step 5, carrying out object storage early warning notification according to the abnormity and the potential reasons, and realizing object storage monitoring. The problems that real-time data of a storage system cannot be obtained, the reliability of object storage is reduced, meanwhile, the health condition of object storage cannot be accurately checked, and the real-time monitoring and abnormal condition coping capacity of maintenance personnel on the stored data is reduced are solved.

Description

Technical Field

[0001] The present invention relates to the field of monitoring storage technology, and in particular to an object storage monitoring method. Background Art

[0002] With the popularization and development of the Internet, object storage has become the preferred way for many enterprises and individuals to store data.

[0003] However, traditional data storage methods cannot detect in real time statistical functions such as new additions, deletions, calls, file size changes, file quantity growth, and file status of the file system, resulting in the inability to obtain real-time data from the storage system, reducing the data security and reliability of object storage. At the same time, traditional storage methods cannot accurately check the health of object storage and cannot issue early warning information in a timely manner, reducing the maintenance personnel's ability to monitor the stored data in real time and respond to abnormal situations.

[0004] Therefore, the present invention proposes an object storage monitoring method. Summary of the invention

[0005] The present invention provides an object storage monitoring method to solve the problem that the real-time data of the storage system cannot be obtained in the prior art, which reduces the data security and reliability of the object storage. At the same time, the traditional storage method cannot accurately check the health of the object storage and cannot issue early warning information in time, which reduces the maintenance personnel's real-time monitoring of the storage data and the ability to respond to abnormal situations.

[0006] In one aspect, the present invention provides an object storage monitoring method, comprising: Step 1: Obtain monitoring requirements of the storage system, and obtain corresponding distributed monitoring tools according to the monitoring requirements; Step 2: configuring appropriate monitoring rules according to the distributed monitoring tool based on the actual storage business scenario, and detecting multiple statistical functions of the file system in real time according to the monitoring rules; Step 3: acquiring real-time performance data of the storage system according to multiple statistical functions, and monitoring the working status of the storage system in real time according to the real-time performance data; Step 4: Determine the health status of the object storage according to the working status, and obtain the abnormality and potential causes of the storage system according to the health status; Step 5: Perform object storage warning notifications based on monitoring and alarm thresholds according to anomalies and potential causes to implement object storage monitoring.

[0007] According to an object storage monitoring method provided by the present invention, before obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tool according to the monitoring requirements, the method further includes: Obtain historical monitoring events of the storage system, determine the monitoring object based on the historical monitoring events, and obtain data attributes of the monitoring object; Determine the storage space of the monitored object according to the data attributes, and determine the dormant storage space and active storage space in the storage system according to the distribution of the storage space; Determine the data flow parameters corresponding to the dormant storage space and the active storage space respectively, and determine the data storage selection gain factors of the dormant storage space and the active storage space respectively according to the data flow parameters; Respectively obtain virtual space configuration parameters of the dormant storage space and the active storage space, and determine monitoring factors for the dormant storage space and the active storage space according to the virtual space configuration parameters; Constructing monitoring rules for the dormant storage space and the active storage space respectively based on the monitoring factor and the data storage selection gain factors of the dormant storage space and the active storage space respectively; Configure monitoring policies for dormant storage space and active storage space respectively according to monitoring rules; Determine the respective weighted monitoring indicators of the dormant storage space and the active storage space according to the monitoring strategy, and obtain the mapped virtual resources of the weighted monitoring indicators; According to the mapped virtual resources, the abnormal attributes of the monitoring states of the dormant storage space and the active storage space are evaluated through a preset virtual call function; According to the abnormal attributes of the monitoring status, the storage resources of the dormant storage space and the active storage space are optimized to ensure that the monitoring requirements are met.

[0008] According to an object storage monitoring method provided by the present invention, a monitoring requirement of a storage system is obtained, and a corresponding distributed monitoring tool is obtained according to the monitoring requirement, including: Get the basic settings and configuration of the storage system, and based on the basic settings and configuration, get the dependencies of applications and services; determining the interaction of the storage system with computing and network resources based on the dependencies of the applications and services; Obtain important events of the storage system based on the usage and performance requirements of the storage system; Determine the key areas that need to be monitored based on the important events mentioned; Determining monitoring requirements for the storage system based on the storage system's interactions with computing and network resources and key areas; Obtain corresponding distributed monitoring tools based on industry standards according to the monitoring requirements.

[0009] According to an object storage monitoring method provided by the present invention, after obtaining the dependency relationship between the application program and the service according to the basic settings and configurations, the method further includes: Split the application into multiple independent microservices according to the microservice architecture, and obtain the business logic and data model of each microservice; Obtain appropriate storage systems and computing resources based on the business logic and data model of each microservice; Establishing a lightweight communication protocol between microservices according to the storage system and computing resources, and determining communication between multiple microservices according to the communication protocol; Use monitoring tools to track the performance of each microservice and the communication between multiple microservices; Obtain the performance status of each microservice and the communication anomalies between multiple microservices, quickly isolate the abnormal services based on the service grid, and switch the abnormal services to backup services.

[0010] According to an object storage monitoring method provided by the present invention, according to the distributed monitoring tool, appropriate monitoring rules are configured based on actual storage business scenarios, and multiple statistical functions of the file system are detected in real time according to the monitoring rules, including: Obtain multiple relevant information of actual storage business scenarios; Acquire storage device type information and other relevant parameters in the system environment according to the multiple relevant information; Configure appropriate monitoring rules based on the distributed monitoring tool according to the storage device type information and other relevant parameters; Obtaining basic information of the file system, and determining attributes of the file system according to the basic information; Based on the file system attributes and storage business requirements, multiple statistical functions of the file system are detected in real time based on monitoring rules.

[0011] According to an object storage monitoring method provided by the present invention, real-time performance data of a storage system is obtained according to multiple statistical functions, and the working status of the storage system is monitored in real time according to the real-time performance data, including: Deploy a statistical agent in the storage system, and track the activity and performance of each statistical function in real time according to the statistical agent; Get real-time performance data of the storage system based on the storage environment according to the activity and performance of each statistical function; Acquire output indicators and log information of the system according to the real-time performance data; The output indicators and log information are analyzed, and the working status of the storage system is monitored in real time based on the preset working indicator parameters according to the analysis results.

[0012] According to an object storage monitoring method provided by the present invention, the health status of the object storage is determined according to the working status, and the abnormality and potential causes of the storage system are obtained according to the health status, including: Obtain metadata information of the object storage, and construct an index table of the object storage according to the metadata information; Determine the most recent access time and the most recent modification time of each object according to the index table of the object storage; Determine the health of the object storage based on the working state of the storage system according to the most recent access time and the most recent modification time of each object; Obtain abnormal conditions in the storage system based on the health of the object storage and all relevant log information in the storage system; According to the abnormal situation, potential causes of the abnormality are obtained based on the code and configuration of the storage system and the application.

[0013] According to an object storage monitoring method provided by the present invention, object storage early warning notification is performed based on monitoring and alarm thresholds according to anomalies and potential causes to implement object storage monitoring, including: Set normal ranges and thresholds for object storage system operation status and code anomalies based on historical log information and storage system load. Obtain real-time operation data of the object storage system and application program based on anomalies and potential causes; Determine the real-time operation status and real-time code information of the object storage system based on the operation data; According to the real-time operation status and real-time code information, object storage early warning notification is performed based on the normal range and threshold of the object storage system operation status and code anomalies to realize object storage monitoring.

[0014] Compared with the prior art, the present invention has the following beneficial effects: By obtaining the corresponding distributed monitoring tools based on monitoring needs, multiple statistical functions and working status of the file system can be detected in real time, so as to evaluate the health of object storage and the cause of abnormalities, and make early warning notifications for object storage. It can obtain real-time data of the storage system, improve the data security and reliability of object storage, and at the same time, accurately check the health of object storage, issue early warning information in time, and improve the maintenance personnel's real-time monitoring of storage data and ability to respond to abnormal situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1It is a flowchart of an object storage monitoring method provided by an embodiment of the present invention; Figure 2 It is a flowchart of obtaining a distributed monitoring tool according to the monitoring requirements of a storage system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Embodiment 1: An object storage monitoring method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the method mainly includes the following steps: Step 1: Obtain monitoring requirements of the storage system, and obtain corresponding distributed monitoring tools according to the monitoring requirements; Step 2: configuring appropriate monitoring rules according to the distributed monitoring tool based on the actual storage business scenario, and detecting multiple statistical functions of the file system in real time according to the monitoring rules; Step 3: acquiring real-time performance data of the storage system according to multiple statistical functions, and monitoring the working status of the storage system in real time according to the real-time performance data; Step 4: Determine the health status of the object storage according to the working status, and obtain the abnormality and potential causes of the storage system according to the health status; Step 5: Perform object storage warning notifications based on monitoring and alarm thresholds according to anomalies and potential causes to implement object storage monitoring.

[0019] In this embodiment, the monitoring requirements of the storage system mainly refer to the monitoring and management of its performance, availability, reliability and other aspects.

[0020] In this embodiment, the distributed monitoring tool refers to a tool that can implement real-time monitoring and alarming of system resources, application programs, network traffic, etc. in a distributed system, such as Zabbix and Nagios.

[0021] In this embodiment, the monitoring rules may be: access frequency, upload / download rate, and file size change of the monitoring object.

[0022] In this embodiment, the statistical functions include: the amount of new additions, deletions, calls, growth in the number of files, and file status of the file system.

[0023] In this embodiment, the real-time performance data includes: the number of input / output operations per second and the throughput.

[0024] In this embodiment, the health status of the object storage refers to the comprehensive performance of the storage system in terms of performance, reliability, and security during normal use.

[0025] The beneficial effects of the above technical solution are: by obtaining the corresponding distributed monitoring tools according to monitoring needs, multiple statistical functions and working status of the file system are detected in real time, so as to evaluate the health of object storage and the cause of abnormalities, and make object storage early warning notifications, it is possible to obtain real-time data of the storage system, improve the data security and reliability of object storage, and at the same time, accurately check the health of object storage, issue early warning information in time, and improve the maintenance personnel's real-time monitoring of storage data and the ability to respond to abnormal situations.

[0026] Embodiment 2: Based on Example 1, before obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tool according to the monitoring requirements, the embodiment of the present invention further includes: Obtain historical monitoring events of the storage system, determine the monitoring object based on the historical monitoring events, and obtain data attributes of the monitoring object; Determine the storage space of the monitored object according to the data attributes, and determine the dormant storage space and active storage space in the storage system according to the distribution of the storage space; Determine the data flow parameters corresponding to the dormant storage space and the active storage space respectively, and determine the data storage selection gain factors of the dormant storage space and the active storage space respectively according to the data flow parameters; Respectively obtain virtual space configuration parameters of the dormant storage space and the active storage space, and determine monitoring factors for the dormant storage space and the active storage space according to the virtual space configuration parameters; Constructing monitoring rules for the dormant storage space and the active storage space respectively based on the monitoring factor and the data storage selection gain factors of the dormant storage space and the active storage space respectively; Configure monitoring policies for dormant storage space and active storage space respectively according to monitoring rules; Determine the respective weighted monitoring indicators of the dormant storage space and the active storage space according to the monitoring strategy, and obtain the mapped virtual resources of the weighted monitoring indicators; According to the mapped virtual resources, the abnormal attributes of the monitoring states of the dormant storage space and the active storage space are evaluated through a preset virtual call function; According to the abnormal attributes of the monitoring status, the storage resources of the dormant storage space and the active storage space are optimized to ensure that the monitoring requirements are met.

[0027] In this embodiment, the historical monitoring events of the storage system refer to all monitoring-related event records since the storage system was enabled, including device startup, configuration modification, performance monitoring, and troubleshooting.

[0028] In this embodiment, the monitoring object generally refers to a hardware or software system that needs to be monitored and managed, such as a server, a network device, a database, an application program, or a storage system.

[0029] In this embodiment, the data attribute of the monitored object refers to the information describing the properties, status, and behavior of the monitored object, such as time series data.

[0030] In this embodiment, the storage space of the monitored object generally refers to a disk or other storage medium, such as a solid state drive, used to store data and files in a computer system.

[0031] In this embodiment, the dormant storage space in the storage system generally refers to files or directories that have not been accessed or written for a long time and are in a dormant state because they have not been accessed for a long time.

[0032] In this embodiment, the active storage space in the storage system refers to files or directories that have been recently accessed or written, which are active and may be modified or deleted at any time.

[0033] In this embodiment, the data flow parameters of the dormant storage space include: least recently used algorithm, average waiting time, and cache size.

[0034] In this embodiment, the data flow parameters of the active storage space include: least recently used algorithm, average waiting time, cache size, and most recently used time.

[0035] In this embodiment, the data storage selection gain factor of the dormant storage space refers to the access preference of the storage system to the data block in the dormant state relative to the active state, and describes the strength of the relationship between the probability of the data block being accessed and the time point of its most recent access. When a data block has not been accessed for a long time, the data storage selection gain factor of the dormant storage space will increase, indicating that the system is more inclined to continue using the data block in future accesses.

[0036] In this embodiment, the data storage selection gain factor of the active storage space refers to the access preference of the storage system to the data block in the active state relative to the dormant state, and describes the relationship between the access frequency and time of the data block by the system within a given time period. When a data block is frequently accessed, the data storage selection gain factor of the active storage space will be larger, and when a data block has not been accessed for a long time, the data storage selection gain factor of the active storage space will be smaller.

[0037] In this embodiment, the virtual space configuration parameters of the dormant storage space include: a dormant time threshold, an activation condition, and a maximum dormant time.

[0038] In this embodiment, the virtual space configuration parameters of the active storage space include: an active time threshold, an activation condition, and a minimum usage time.

[0039] In this embodiment, the monitoring factor is a method used to evaluate or quantify a specific factor.

[0040] In this embodiment, the monitoring rule for the dormant storage space is, for example: when at least one file in the storage space is in a dormant state for more than a preset time, it is considered that the storage space is no longer in use.

[0041] In this embodiment, the monitoring rule for the active storage space is, for example: when the number of files in the storage space is lower than a preset threshold, the storage space is considered to be overcrowded.

[0042] In this embodiment, mapping virtual resources with an emphasis on monitoring indicators refers to a technology and method for dynamically managing and allocating virtual resources according to the requirements of the applications for different monitoring indicators when allocating physical resources to different applications or services.

[0043] In this embodiment, the abnormal attributes of the monitoring status refer to some abnormal or inconsistent phenomena or indicators that appear when monitoring the storage system status.

[0044] The beneficial effects of the above technical solution are: optimizing the storage resources of the dormant storage space and the active storage space according to the abnormal attributes of the monitoring status to determine whether they meet the monitoring requirements, which can more reasonably utilize storage resources and avoid idling or waste, thereby improving the storage efficiency of the entire storage system. At the same time, by analyzing the monitoring status of the storage system, potential problems and faults can be discovered and resolved in a timely manner, thereby ensuring the stability and reliability of the storage system.

[0045] Embodiment 3: Based on Example 2, the embodiment of the present invention obtains the monitoring requirements of the storage system, and obtains the corresponding distributed monitoring tools according to the monitoring requirements, such as Figure 2 As shown, including: S01: Obtain the basic settings and configuration of the storage system, and obtain the dependencies of applications and services based on the basic settings and configuration; S02: Determine the interaction between the storage system and computing and network resources according to the dependency relationship between the application and the service; S03: Obtain important events of the storage system according to the usage and performance requirements of the storage system; S04: Determine the key areas that need to be monitored based on the important events; S05: Determine monitoring requirements for the storage system according to the interaction between the storage system and computing and network resources and key areas; S06: Obtain corresponding distributed monitoring tools based on industry standards according to the monitoring requirements.

[0046] In this embodiment, the basic settings of the storage system include: architecture design, data model, backup and recovery, monitoring and alarm.

[0047] In this embodiment, the configuration of the storage system refers to a series of parameter settings and adjustments performed on the storage system, such as: LUN mapping: Map one or more partitions on a physical storage device to a logical volume to facilitate management and access to data.

[0048] Cache strategy: Set cache type, size, update frequency and other parameters based on application needs and data access patterns to improve performance and reduce latency.

[0049] Access control: Set up user and permission management mechanisms based on business needs and security requirements, such as assigning administrators and limiting access rights for specific users.

[0050] In this embodiment, the dependency relationship between the application program and the service refers to the interdependence and interaction relationship between the application programs or services, such as: required dependency relationship, optional dependency relationship, and accidental dependency relationship.

[0051] In this embodiment, the interaction between the storage system and the computing and network resources refers to the data exchange and communication between them, so that different applications and services can store and access data in different locations.

[0052] In this embodiment, the monitoring requirements of the storage system mainly refer to the monitoring and management of its performance, availability, reliability and other aspects.

[0053] The beneficial effects of the above technical solution are: obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tools based on the monitoring requirements, which can improve the monitoring capabilities of the object storage system, ensure the compatibility and adaptability of the monitoring tools and the system, and enhance the reliability and credibility of the monitoring data.

[0054] Embodiment 4: Based on Embodiment 3, after obtaining the dependency relationship between the application program and the service according to the basic settings and configurations, the embodiment of the present invention further includes: Split the application into multiple independent microservices according to the microservice architecture, and obtain the business logic and data model of each microservice; Obtain appropriate storage systems and computing resources based on the business logic and data model of each microservice; Establishing a lightweight communication protocol between microservices according to the storage system and computing resources, and determining communication between multiple microservices according to the communication protocol; Use monitoring tools to track the performance of each microservice and the communication between multiple microservices; Obtain the performance status of each microservice and the communication anomalies between multiple microservices, quickly isolate the abnormal services based on the service grid, and switch the abnormal services to backup services.

[0055] In this embodiment, the microservice architecture splits the application into multiple small services, each of which has its own independent responsibilities and boundaries, and these services can communicate through APIs or other protocols.

[0056] In this embodiment, microservices are one or more services with self-coordination capabilities. Each microservice is independently responsible for one or more business functions and communicates with other microservices through lightweight interfaces.

[0057] In this embodiment, the business logic of the microservice refers to the specific business functions and processing flows that the microservice is responsible for.

[0058] In this embodiment, the data model of a microservice refers to a methodology for describing the internal data structure and organization of a microservice, including regulations and constraints on data representation, relationships, and access.

[0059] In this embodiment, computing resources refer to hardware and software resources that can be used to perform computing tasks.

[0060] In this embodiment, the lightweight communication protocol is a protocol used to implement communication between devices in a distributed system.

[0061] In this embodiment, the performance of the microservice generally refers to the performance of the microservice during deployment and use, including system throughput, latency, and resource utilization.

[0062] In this embodiment, the communication between microservices refers to the message transmission and interaction between microservices.

[0063] In this embodiment, the service grid is a technical framework for solving communication problems between microservices.

[0064] The beneficial effects of the above technical solution are: splitting the application into multiple independent microservices and obtaining the business logic and data model of each microservice, obtaining suitable storage systems and computing resources, establishing a lightweight communication protocol, determining communication anomalies between multiple microservices, and switching abnormal services to backup services. This can maintain the availability and stability of the system when failures or performance problems occur, and at the same time, can reduce downtime and impact scope, help ensure business continuity and minimize data loss and other adverse consequences.

[0065] Embodiment 5: Based on Example 4, the embodiment of the present invention configures appropriate monitoring rules based on the distributed monitoring tool based on the actual storage business scenario, and detects multiple statistical functions of the file system in real time according to the monitoring rules, including: Obtain multiple relevant information of actual storage business scenarios; Acquire storage device type information and other relevant parameters in the system environment according to the multiple relevant information; Configure appropriate monitoring rules based on the distributed monitoring tool according to the storage device type information and other relevant parameters; Obtaining basic information of the file system, and determining attributes of the file system according to the basic information; Based on the file system attributes and storage business requirements, multiple statistical functions of the file system are detected in real time based on monitoring rules.

[0066] In this embodiment, multiple pieces of relevant information of the actual storage business scenario include: storage process, data characteristics, and performance requirements.

[0067] In this embodiment, the storage device type information includes: hard disk, solid state drive, capacity, and cache mechanism.

[0068] In this embodiment, other relevant parameters in the system environment include: operating system version, application version, and hardware configuration.

[0069] In this embodiment, the distributed monitoring tool refers to a tool that can implement real-time monitoring and alarming of system resources, application programs, network traffic, etc. in a distributed system, such as Zabbix and Nagios.

[0070] In this embodiment, the monitoring rules are sets of instructions for defining and configuring how to identify, trigger, and process monitoring events in the monitoring tool.

[0071] In this embodiment, the basic information of the file system includes: allocation unit, the number of disk blocks occupied by each file in the file system, the number of sectors contained in each allocation unit in the file system, and the version number of the file system.

[0072] In this embodiment, the attributes of the file system refer to a set of characteristics or parameters inherent in the file system that describe the behavior of the file system, such as: the type of the file system, the data structure of the file system, the usage, location and size of each cluster in the file system.

[0073] In this embodiment, the storage service demand refers to the corresponding functions and services provided by the storage system to meet the requirements of different business applications on storage capacity and performance, such as storage capacity demand.

[0074] The beneficial effects of the above technical solution are: by configuring appropriate monitoring rules based on the distributed monitoring tool based on the actual storage business scenario, the pertinence and adaptability of the monitoring rules can be ensured, thereby obtaining more accurate and reliable monitoring data. At the same time, according to the monitoring rules, multiple statistical functions of the file system are detected in real time, and the functional status of the system can be obtained quickly and accurately, thereby ensuring the frequency and coverage of monitoring and improving monitoring efficiency.

[0075] Embodiment 6: Based on Example 5, the embodiment of the present invention obtains real-time performance data of the storage system according to multiple statistical functions, and monitors the working status of the storage system in real time according to the real-time performance data, including: Deploy a statistical agent in the storage system, and track the activity and performance of each statistical function in real time according to the statistical agent; Get real-time performance data of the storage system based on the storage environment according to the activity and performance of each statistical function; Acquire output indicators and log information of the system according to the real-time performance data; The output indicators and log information are analyzed, and the working status of the storage system is monitored in real time based on the preset working indicator parameters according to the analysis results.

[0076] In this embodiment, the statistical agent is used to monitor the running status of the application or server and collect various indicator data, such as the creation, reading, and modification of each record.

[0077] In this embodiment, the statistical functions include: the file system's new increments, deletions, calls, file size changes, file quantity growth, and file status.

[0078] In this embodiment, the storage environment refers to a combination of hardware devices and software systems that provide data storage functions for computer systems and applications, including physical devices, virtualization technology, storage management software and other components.

[0079] In this embodiment, the real-time performance data includes: the number of input / output operations per second and the throughput.

[0080] In this embodiment, the output indicators of the system may be: response time, error rate, and delay time.

[0081] In this embodiment, the preset working index parameter refers to an index parameter set in advance, such as a response time of 0.1 second to 0.2 second.

[0082] The beneficial effects of the above technical solution are: obtaining real-time performance data of the storage system according to the statistical function, monitoring the working status of the storage system in real time according to the real-time performance data, being able to optimize the performance of the storage system in real time, and improving the reliability of the object storage system.

[0083] Embodiment 7: Based on Example 6, the embodiment of the present invention determines the health status of the object storage according to the working status, and obtains the abnormality and potential causes of the storage system according to the health status, including: Obtain metadata information of the object storage, and construct an index table of the object storage according to the metadata information; Determine the most recent access time and the most recent modification time of each object according to the index table of the object storage; Determine the health of the object storage based on the working state of the storage system according to the most recent access time and the most recent modification time of each object; Obtain abnormal conditions in the storage system based on the health of the object storage and all relevant log information in the storage system; According to the abnormal situation, potential causes of the abnormality are obtained based on the code and configuration of the storage system and the application.

[0084] In this embodiment, the metadata information includes: the name, type, creation time, and modification time of the object.

[0085] In this embodiment, the index table of object storage refers to a special type of data structure created in the object storage system to facilitate query and access to objects. It organizes the metadata information of the objects according to certain rules and structures, and stores it in the index table in the form of key-value pairs.

[0086] In this embodiment, the working status of the storage system includes: Read status: refers to the efficiency and speed of the storage system in reading data from the disk or other storage media, including indicators such as average seek time, average return time, and average latency.

[0087] Write status: refers to the efficiency and speed of the storage system writing data to the disk or other storage media, including indicators such as average seek time, average write time and average latency.

[0088] Performance status: refers to the performance of the storage system under different loads, including indicators such as the maximum number of concurrent accesses and throughput.

[0089] Capacity status: refers to the storage space usage and remaining space available in the storage system, including total capacity, used capacity, free capacity, and utilization rate.

[0090] In this embodiment, the health status of the object storage refers to the comprehensive performance of the storage system in terms of performance, reliability, and security during normal use.

[0091] In this embodiment, abnormal situations that may occur in the storage system include: storage pool unavailability, data corruption, and virus infection.

[0092] The beneficial effects of the above technical solution are: by building an index table through metadata information, the speed of finding objects in the object storage system can be accelerated, the query efficiency can be improved, and abnormal conditions and potential causes can be obtained according to the health of the object storage, so that problems can be discovered and repaired in time, avoiding data loss and performance degradation problems from causing greater impact on the system. At the same time, by quickly identifying and resolving abnormal conditions, fault recovery time and downtime can be shortened, and storage efficiency and monitoring effects can be improved.

[0093] Embodiment 8: Based on Example 7, the embodiment of the present invention performs object storage warning notification based on monitoring and alarm thresholds according to the anomaly and potential causes to implement object storage monitoring, including: Set normal ranges and thresholds for object storage system operation status and code anomalies based on historical log information and storage system load. Obtain real-time operation data of the object storage system and application program based on anomalies and potential causes; Determine the real-time operation status and real-time code information of the object storage system based on the operation data; According to the real-time operation status and real-time code information, object storage early warning notification is performed based on the normal range and threshold of the object storage system operation status and code anomalies to realize object storage monitoring.

[0094] In this embodiment, the load condition of the storage system refers to indicators of the amount of data and the number of requests processed by the storage system within a certain period of time, and reflects the current operating status and performance level of the storage system.

[0095] In this embodiment, the object storage system operation status refers to checking, monitoring and analyzing a specific object storage system to determine whether it is in normal working condition and can provide stable and reliable services to users, including: normal operation, abnormal operation, and suspended operation.

[0096] In this embodiment, code anomalies refer to errors or abnormal behaviors in the program. In the object storage system, code anomalies include: file read and write errors, permission access failure, file size limit exceeded, file does not exist, network connection problems, system downtime, etc.

[0097] In this embodiment, potential causes of object storage system abnormalities include: programming errors, CPU overheating, memory leaks, software vulnerabilities, network failures, and database failures.

[0098] In this embodiment, the real-time operation data of the object storage system refers to various indicators and log information generated by the storage system within a specific time period, such as: read and write operations, IOPS, throughput, latency, capacity utilization, and access mode.

[0099] In this embodiment, the running data of the application refers to various indicators and log information generated during the running of the application, such as: CPU usage, memory usage, and network traffic.

[0100] In this embodiment, the real-time code information of the object storage system generally refers to the internal implementation code or middleware code of the system, which includes the core functions and logical implementation of the object storage system, such as algorithms and implementations of file system management, permission control, caching mechanism, asynchronous operation, etc.

[0101] The beneficial effects of the above technical solution are: object storage early warning notifications are performed based on monitoring and alarm thresholds according to anomalies and potential causes, object storage monitoring is realized, anomalies in the storage system can be discovered in time, the problem can be avoided from expanding and the scope of impact can be avoided, and efficient monitoring and management of the object storage system can be realized.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An object storage monitoring method, characterized in that: include: Step 1: Obtain monitoring requirements of the storage system, and obtain corresponding distributed monitoring tools according to the monitoring requirements; Step 2: configuring appropriate monitoring rules according to the distributed monitoring tool based on the actual storage business scenario, and detecting multiple statistical functions of the file system in real time according to the monitoring rules; Step 3: acquiring real-time performance data of the storage system according to multiple statistical functions, and monitoring the working status of the storage system in real time according to the real-time performance data; Step 4: Determine the health status of the object storage according to the working status, and obtain the abnormality and potential causes of the storage system according to the health status; Step 5: Perform object storage warning notifications based on monitoring and alarm thresholds according to anomalies and potential causes to implement object storage monitoring.

2. The object storage monitoring method according to claim 1, characterized in that: Before obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tool according to the monitoring requirements, the method further includes: Obtain historical monitoring events of the storage system, determine the monitoring object based on the historical monitoring events, and obtain data attributes of the monitoring object; Determine the storage space of the monitored object according to the data attributes, and determine the dormant storage space and active storage space in the storage system according to the distribution of the storage space; Determine the data flow parameters corresponding to the dormant storage space and the active storage space respectively, and determine the data storage selection gain factors of the dormant storage space and the active storage space respectively according to the data flow parameters; Respectively obtain virtual space configuration parameters of the dormant storage space and the active storage space, and determine monitoring factors for the dormant storage space and the active storage space according to the virtual space configuration parameters; Constructing monitoring rules for the dormant storage space and the active storage space respectively based on the monitoring factor and the data storage selection gain factors of the dormant storage space and the active storage space respectively; Configure monitoring policies for dormant storage space and active storage space respectively according to monitoring rules; Determine the respective weighted monitoring indicators of the dormant storage space and the active storage space according to the monitoring strategy, and obtain the mapped virtual resources of the weighted monitoring indicators; According to the mapped virtual resources, the abnormal attributes of the monitoring states of the dormant storage space and the active storage space are evaluated through a preset virtual call function; According to the abnormal attributes of the monitoring status, the storage resources of the dormant storage space and the active storage space are optimized to ensure that the monitoring requirements are met.

3. The object storage monitoring method according to claim 1, characterized in that: Obtaining the monitoring requirements of the storage system, and obtaining the corresponding distributed monitoring tools according to the monitoring requirements, including: Get the basic settings and configuration of the storage system, and based on the basic settings and configuration, get the dependencies of applications and services; determining the interaction of the storage system with computing and network resources based on the dependencies of the applications and services; Obtain important events of the storage system based on the usage and performance requirements of the storage system; Determine the key areas that need to be monitored based on the important events mentioned; Determining monitoring requirements for the storage system based on the storage system's interactions with computing and network resources and key areas; Obtain corresponding distributed monitoring tools based on industry standards according to the monitoring requirements.

4. The object storage monitoring method according to claim 2, characterized in that: After obtaining the dependencies of the application and services according to the basic setup and configuration, it also includes: Split the application into multiple independent microservices according to the microservice architecture, and obtain the business logic and data model of each microservice; Obtain appropriate storage systems and computing resources based on the business logic and data model of each microservice; Establishing a lightweight communication protocol between microservices according to the storage system and computing resources, and determining communication between multiple microservices according to the communication protocol; Use monitoring tools to track the performance of each microservice and the communication between multiple microservices; Obtain the performance status of each microservice and the communication anomalies between multiple microservices, quickly isolate the abnormal services based on the service grid, and switch the abnormal services to backup services.

5. The object storage monitoring method according to claim 1, characterized in that: According to the distributed monitoring tool, appropriate monitoring rules are configured based on the actual storage business scenario, and multiple statistical functions of the file system are detected in real time according to the monitoring rules, including: Obtain multiple relevant information of actual storage business scenarios; Acquire storage device type information and other relevant parameters in the system environment according to the multiple relevant information; Configure appropriate monitoring rules based on the distributed monitoring tool according to the storage device type information and other relevant parameters; Obtaining basic information of the file system, and determining attributes of the file system according to the basic information; Based on the file system attributes and storage business requirements, multiple statistical functions of the file system are detected in real time based on monitoring rules.

6. The object storage monitoring method according to claim 1, characterized in that: Acquiring real-time performance data of the storage system according to multiple statistical functions, and monitoring the working status of the storage system in real time according to the real-time performance data, including: Deploy a statistical agent in the storage system, and track the activity and performance of each statistical function in real time according to the statistical agent; Get real-time performance data of the storage system based on the storage environment according to the activity and performance of each statistical function; Acquire output indicators and log information of the system according to the real-time performance data; The output indicators and log information are analyzed, and the working status of the storage system is monitored in real time based on the preset working indicator parameters according to the analysis results.

7. The object storage monitoring method according to claim 1, characterized in that: Determine the health status of the object storage according to the working status, and obtain the abnormality and potential causes of the storage system according to the health status, including: Obtain metadata information of the object storage, and construct an index table of the object storage according to the metadata information; Determine the most recent access time and the most recent modification time of each object according to the index table of the object storage; Determine the health of the object storage based on the working state of the storage system according to the most recent access time and the most recent modification time of each object; Obtain abnormal conditions in the storage system based on the health of the object storage and all relevant log information in the storage system; According to the abnormal situation, potential causes of the abnormality are obtained based on the code and configuration of the storage system and the application.

8. The object storage monitoring method according to claim 1, characterized in that: Based on the monitoring and alarm thresholds, object storage warning notifications are provided according to the anomalies and potential causes to achieve object storage monitoring, including: Set normal ranges and thresholds for object storage system operation status and code anomalies based on historical log information and storage system load. Obtain real-time operation data of the object storage system and application program based on anomalies and potential causes; Determine the real-time operation status and real-time code information of the object storage system based on the operation data; According to the real-time operation status and real-time code information, object storage early warning notification is performed based on the normal range and threshold of the object storage system operation status and code anomalies to realize object storage monitoring.

Images