Object storage monitoring method

Abstract

The present invention provides an object storage monitoring method, belonging to the field of storage monitoring technology, comprising: step 1: obtaining monitoring requirements of a storage system and obtaining a corresponding distributed monitoring tool; step 2: configuring appropriate monitoring rules based on the distributed monitoring tool and actual storage business scenarios, and detecting multiple statistical functions of the file system in real time; step 3: obtaining real-time performance data of the storage system based on the multiple statistical functions, and monitoring the working status of the storage system in real time; step 4: determining the health of the object storage based on the working status, obtaining abnormalities and potential causes of the storage system; and step 5: issuing object storage early warning notifications based on the abnormalities and potential causes, thereby realizing object storage monitoring. This method solves the problem of being unable to obtain real-time data from the storage system, which reduces the reliability of the object storage, and simultaneously, being unable to accurately check the health of the object storage, which reduces the maintenance personnel's ability to monitor the storage data in real time and respond to abnormal situations.

Description

Technical Field

[0001] The present invention relates to the field of monitoring storage technology, and in particular to an object storage monitoring method. Background Art

[0002] With the popularization and development of the Internet, object storage has become the preferred way for many enterprises and individuals to store data.

[0003] However, traditional data storage methods are unable to detect statistical functions such as new additions, deletions, calls, file size changes, file quantity growth, and file status in the file system in real time, resulting in the inability to obtain real-time data from the storage system, reducing the data security and reliability of object storage. At the same time, traditional storage methods cannot accurately check the health of object storage and cannot issue early warning information in a timely manner, which reduces the maintenance personnel's ability to monitor the stored data in real time and respond to abnormal situations.

[0004] Therefore, the present invention proposes an object storage monitoring method. Summary of the Invention

[0005] The present invention provides an object storage monitoring method to solve the problem in the prior art that real-time data of the storage system cannot be obtained, which reduces the data security and reliability of the object storage. At the same time, traditional storage methods cannot accurately check the health of the object storage and cannot issue early warning information in a timely manner, which reduces the maintenance personnel's ability to monitor the stored data in real time and respond to abnormal situations.

[0006] In one aspect, the present invention provides an object storage monitoring method, comprising:

[0007] Step 1: Obtain the monitoring requirements of the storage system and obtain the corresponding distributed monitoring tools based on the monitoring requirements;

[0008] Step 2: Configure appropriate monitoring rules based on the distributed monitoring tool and the actual storage business scenario, and detect multiple statistical functions of the file system in real time according to the monitoring rules;

[0009] Step 3: Acquire real-time performance data of the storage system according to multiple statistical functions, and monitor the working status of the storage system in real time according to the real-time performance data;

[0010] Step 4: Determine the health of the object storage based on the working status, and obtain storage system abnormalities and potential causes based on the health status;

[0011] Step 5: Provide object storage warning notifications based on monitoring and alarm thresholds according to anomalies and potential causes to implement object storage monitoring.

[0012] According to an object storage monitoring method provided by the present invention, before obtaining monitoring requirements of a storage system and obtaining a corresponding distributed monitoring tool according to the monitoring requirements, the method further includes:

[0013] Obtain historical monitoring events of the storage system, determine the monitoring object based on the historical monitoring events, and obtain the data attributes of the monitoring object;

[0014] Determine the storage space of the monitored object based on data attributes, and determine the dormant storage space and active storage space in the storage system based on the distribution of the storage space;

[0015] Determine data flow parameters corresponding to the dormant storage space and the active storage space respectively, and determine data storage selection gain factors corresponding to the dormant storage space and the active storage space respectively according to the data flow parameters;

[0016] respectively obtaining virtual space configuration parameters of the dormant storage space and the active storage space, and determining monitoring factors for the dormant storage space and the active storage space according to the virtual space configuration parameters;

[0017] Building monitoring rules for the dormant storage space and the active storage space based on the monitoring factor and the data storage selection gain factors of the dormant storage space and the active storage space;

[0018] Configure monitoring policies for dormant storage spaces and active storage spaces based on monitoring rules.

[0019] Determine the respective priority monitoring indicators of the dormant storage space and the active storage space according to the monitoring policy, and obtain the mapped virtual resources of the priority monitoring indicators;

[0020] Evaluate the abnormal attributes of the monitoring status of the dormant storage space and the active storage space respectively through a preset virtual call function according to the mapped virtual resources;

[0021] The storage resources of the dormant storage space and the active storage space are optimized according to the abnormal attributes of the monitoring status to ensure that they meet the monitoring requirements.

[0022] According to an object storage monitoring method provided by the present invention, monitoring requirements of a storage system are obtained, and corresponding distributed monitoring tools are obtained according to the monitoring requirements, including:

[0023] Obtain the basic settings and configuration of the storage system, and based on the basic settings and configuration, obtain the dependencies of applications and services;

[0024] determining the interaction of storage systems with computing and network resources based on dependencies between said applications and services;

[0025] Obtain important events of the storage system based on the usage and performance requirements of the storage system;

[0026] Determine the key areas that need to be monitored based on the important events;

[0027] Determining monitoring requirements for the storage system based on its interaction with computing and network resources and key areas;

[0028] Obtain corresponding distributed monitoring tools based on industry standards according to the monitoring requirements.

[0029] According to an object storage monitoring method provided by the present invention, after obtaining the dependency relationship between applications and services based on basic settings and configurations, the method further includes:

[0030] Split the application into multiple independent microservices based on the microservice architecture and obtain the business logic and data model of each microservice;

[0031] Obtain appropriate storage systems and computing resources based on the business logic and data model of each microservice;

[0032] Establishing a lightweight communication protocol between microservices based on the storage system and computing resources, and determining communication between multiple microservices based on the communication protocol;

[0033] Use monitoring tools to track the performance of each microservice and the communication between multiple microservices;

[0034] Obtain the performance status of each microservice and the communication anomalies between multiple microservices, quickly isolate the abnormal services based on the service grid, and switch the abnormal services to backup services.

[0035] According to an object storage monitoring method provided by the present invention, appropriate monitoring rules are configured based on the distributed monitoring tool and actual storage business scenarios, and multiple statistical functions of the file system are detected in real time according to the monitoring rules, including:

[0036] Obtain multiple relevant information about actual storage business scenarios;

[0037] Acquire storage device type information and other relevant parameters in the system environment according to the plurality of relevant information;

[0038] Configure appropriate monitoring rules based on the distributed monitoring tool according to the storage device type information and other relevant parameters;

[0039] Obtaining basic information of the file system, and determining attributes of the file system based on the basic information;

[0040] Based on the file system attributes and storage business requirements, multiple statistical functions of the file system are detected in real time based on monitoring rules.

[0041] According to the present invention, an object storage monitoring method is provided, which obtains real-time performance data of a storage system based on multiple statistical functions, and monitors the working status of the storage system in real time based on the real-time performance data, including:

[0042] Deploy a statistics agent in the storage system and track the activity and performance of each statistical function in real time according to the statistics agent;

[0043] Obtain real-time performance data of the storage system based on the storage environment based on the activity and performance of each statistical function;

[0044] Obtaining system output indicators and log information based on the real-time performance data;

[0045] The output indicators and log information are analyzed, and according to the analysis results, the working status of the storage system is monitored in real time based on preset working indicator parameters.

[0046] According to an object storage monitoring method provided by the present invention, determining the health status of the object storage according to the working status, and obtaining storage system abnormalities and potential causes according to the health status, including:

[0047] Obtain metadata information of the object storage, and construct an index table of the object storage based on the metadata information;

[0048] Determine the most recent access time and the most recent modification time of each object according to the index table of the object storage;

[0049] Determine the health of the object storage based on the working status of the storage system according to the most recent access time and the most recent modification time of each object;

[0050] Obtain storage system anomalies based on the health of the object storage and all relevant log information in the storage system;

[0051] According to the abnormal situation, potential causes of the abnormality are obtained based on the code and configuration of the storage system and the application.

[0052] According to the present invention, an object storage monitoring method is provided, which performs object storage early warning notification based on monitoring and alarm thresholds according to anomalies and potential causes, thereby implementing object storage monitoring, including:

[0053] Set normal ranges and thresholds for object storage system operating status and code anomalies based on historical log information and storage system load.

[0054] Obtain real-time operation data of the object storage system and application operation data based on anomalies and potential causes;

[0055] Determine the real-time operating status and real-time code information of the object storage system based on the operating data;

[0056] According to the real-time operation status and real-time code information, object storage early warning notification is performed based on the normal range and threshold of the object storage system operation status and code anomalies to realize object storage monitoring.

[0057] Compared with the prior art, the present invention has the following advantages:

[0058] By obtaining the corresponding distributed monitoring tools based on monitoring needs, multiple statistical functions and working status of the file system can be detected in real time, thereby evaluating the health of object storage and the cause of abnormalities, and issuing object storage early warning notifications. It can obtain real-time data of the storage system, improve the data security and reliability of object storage, and at the same time, accurately check the health of object storage, issue early warning information in a timely manner, and improve the maintenance personnel's real-time monitoring of storage data and the ability to respond to abnormal situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 This is a flow chart of an object storage monitoring method provided by an embodiment of the present invention;

[0061] Figure 2 This is a flow chart of obtaining a distributed monitoring tool based on the monitoring requirements of a storage system, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0063] Example 1:

[0064] An embodiment of the present invention provides an object storage monitoring method, such as Figure 1 As shown, the method mainly includes the following steps:

[0065] Step 1: Obtain the monitoring requirements of the storage system and obtain the corresponding distributed monitoring tools based on the monitoring requirements;

[0066] Step 2: Configure appropriate monitoring rules based on the distributed monitoring tool and the actual storage business scenario, and detect multiple statistical functions of the file system in real time according to the monitoring rules;

[0067] Step 3: Acquire real-time performance data of the storage system according to multiple statistical functions, and monitor the working status of the storage system in real time according to the real-time performance data;

[0068] Step 4: Determine the health of the object storage based on the working status, and obtain storage system abnormalities and potential causes based on the health status;

[0069] Step 5: Provide object storage warning notifications based on monitoring and alarm thresholds according to anomalies and potential causes to implement object storage monitoring.

[0070] In this embodiment, the monitoring requirements of the storage system mainly refer to the monitoring and management of its performance, availability, reliability and other aspects.

[0071] In this embodiment, the distributed monitoring tool refers to a tool that can implement real-time monitoring and alarming of system resources, applications, network traffic, etc. in a distributed system, such as Zabbix and Nagios.

[0072] In this embodiment, the monitoring rules may include: access frequency, upload / download rate, and file size change of the monitored object.

[0073] In this embodiment, the statistical functions include: the file system's new increments, deletions, calls, file quantity growth, and file status.

[0074] In this embodiment, the real-time performance data includes: number of input / output operations per second and throughput.

[0075] In this embodiment, the health status of the object storage refers to the comprehensive performance of the storage system in terms of performance, reliability, and security during normal use.

[0076] The beneficial effects of the above technical solution are: obtaining the corresponding distributed monitoring tools through monitoring needs, detecting multiple statistical functions and working status of the file system in real time, thereby evaluating the health status of object storage and the cause of abnormalities, and performing object storage early warning notifications, being able to obtain real-time data of the storage system, improving the data security and reliability of object storage, and at the same time, being able to accurately check the health status of object storage, and issue early warning information in a timely manner, thereby improving the maintenance personnel's real-time monitoring of storage data and the ability to respond to abnormal situations.

[0077] Example 2:

[0078] Based on Example 1, before obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tool according to the monitoring requirements, the embodiment of the present invention further includes:

[0079] Obtain historical monitoring events of the storage system, determine the monitoring object based on the historical monitoring events, and obtain the data attributes of the monitoring object;

[0080] Determine the storage space of the monitored object based on data attributes, and determine the dormant storage space and active storage space in the storage system based on the distribution of the storage space;

[0081] Determine data flow parameters corresponding to the dormant storage space and the active storage space respectively, and determine data storage selection gain factors corresponding to the dormant storage space and the active storage space respectively according to the data flow parameters;

[0082] respectively obtaining virtual space configuration parameters of the dormant storage space and the active storage space, and determining monitoring factors for the dormant storage space and the active storage space according to the virtual space configuration parameters;

[0083] Building monitoring rules for the dormant storage space and the active storage space based on the monitoring factor and the data storage selection gain factors of the dormant storage space and the active storage space;

[0084] Configure monitoring policies for dormant storage spaces and active storage spaces based on monitoring rules.

[0085] Determine the respective priority monitoring indicators of the dormant storage space and the active storage space according to the monitoring policy, and obtain the mapped virtual resources of the priority monitoring indicators;

[0086] Evaluate the abnormal attributes of the monitoring status of the dormant storage space and the active storage space respectively through a preset virtual call function according to the mapped virtual resources;

[0087] The storage resources of the dormant storage space and the active storage space are optimized according to the abnormal attributes of the monitoring status to ensure that they meet the monitoring requirements.

[0088] In this embodiment, the historical monitoring events of the storage system refer to all monitoring-related event records since the storage system was enabled, including device startup, configuration modification, performance monitoring, and troubleshooting.

[0089] In this embodiment, the monitoring object generally refers to a hardware or software system that needs to be monitored and managed, such as a server, a network device, a database, an application, or a storage system.

[0090] In this embodiment, the data attributes of the monitored object refer to information describing the properties, status, and behavior of the monitored object, such as time series data.

[0091] In this embodiment, the storage space of the monitored object generally refers to a disk or other storage medium, such as a solid-state hard disk, used to store data and files in a computer system.

[0092] In this embodiment, the dormant storage space in the storage system generally refers to files or directories that have not been accessed or written to for a long time and are in a dormant state because they have not been accessed for a long time.

[0093] In this embodiment, the active storage space in the storage system refers to files or directories that have been recently accessed or written, which are active and may be modified or deleted at any time.

[0094] In this embodiment, the data flow parameters of the dormant storage space include: least recently used algorithm, average waiting time, and cache size.

[0095] In this embodiment, the data flow parameters of the active storage space include: least recently used algorithm, average waiting time, cache size, and most recently used time.

[0096] In this embodiment, the data storage selection gain factor of the dormant storage space refers to the access preference of the storage system to the data block in the dormant state relative to the active state, and describes the strength of the relationship between the probability of the data block being accessed and the time point of its most recent access. When a data block has not been accessed for a long time, the data storage selection gain factor of the dormant storage space will increase, indicating that the system is more inclined to continue using the data block in future accesses.

[0097] In this embodiment, the data storage selection gain factor of the active storage space refers to the access preference of the storage system to the data block in the active state relative to the dormant state, and describes the relationship between the frequency and time of the system's access to the data block within a given time period. When a data block is frequently accessed, the data storage selection gain factor of the active storage space will be larger, and when a data block has not been accessed for a long time, the data storage selection gain factor of the active storage space will be smaller.

[0098] In this embodiment, the virtual space configuration parameters of the dormant storage space include: a dormant time threshold, an activation condition, and a maximum dormant time.

[0099] In this embodiment, the virtual space configuration parameters of the active storage space include: an active time threshold, an activation condition, and a minimum usage time.

[0100] In this embodiment, the monitoring factor is a method used to evaluate or quantify a specific factor.

[0101] In this embodiment, the monitoring rule for the dormant storage space is, for example: when at least one file in the storage space is in a dormant state for more than a preset time, the storage space is considered to be no longer in use.

[0102] In this embodiment, the monitoring rule for the active storage space is, for example: when the number of files in the storage space is lower than a preset threshold, the storage space is considered to be overcrowded.

[0103] In this embodiment, mapping virtual resources with an emphasis on monitoring indicators refers to a technology and method for dynamically managing and allocating virtual resources according to the requirements of the applications for different monitoring indicators when allocating physical resources to different applications or services.

[0104] In this embodiment, the abnormal attributes of the monitoring status refer to some abnormal or inconsistent phenomena or indicators that appear when monitoring the storage system status.

[0105] The beneficial effects of the above technical solution are: optimizing the storage resources of the dormant storage space and the active storage space according to the abnormal attributes of the monitoring status to determine whether they meet the monitoring requirements, making more reasonable use of storage resources and avoiding idling or waste, thereby improving the storage efficiency of the entire storage system. At the same time, by analyzing the monitoring status of the storage system, potential problems and faults can be discovered and resolved in a timely manner, thereby ensuring the stability and reliability of the storage system.

[0106] Example 3:

[0107] Based on Example 2, the embodiment of the present invention obtains the monitoring requirements of the storage system and obtains the corresponding distributed monitoring tools according to the monitoring requirements, such as Figure 2 Shown, including:

[0108] S01: Obtain the basic settings and configuration of the storage system, and obtain the dependencies of applications and services based on the basic settings and configuration;

[0109] S02: Determine the interaction between the storage system and computing and network resources based on the dependency relationship between the application and the service;

[0110] S03: Obtain important events of the storage system based on the usage and performance requirements of the storage system;

[0111] S04: Determine the key areas that need to be monitored based on the important events;

[0112] S05: Determine monitoring requirements for the storage system based on interactions between the storage system and computing and network resources and key areas;

[0113] S06: Obtain corresponding distributed monitoring tools based on industry standards according to the monitoring requirements.

[0114] In this embodiment, the basic settings of the storage system include: architecture design, data model, backup and recovery, monitoring and alarm.

[0115] In this embodiment, the configuration of the storage system refers to a series of parameter settings and adjustments performed on the storage system, such as:

[0116] LUN mapping: Map one or more partitions on a physical storage device to a logical volume to facilitate management and access to data.

[0117] Caching strategy: Based on the application's needs and data access patterns, set cache parameters such as type, size, and update frequency to improve performance and reduce latency.

[0118] Access control: Set up user and permission management mechanisms based on business needs and security requirements, such as assigning administrators and limiting access rights for specific users.

[0119] In this embodiment, the dependency relationship between applications and services refers to the interdependence and interaction relationship between applications or services, such as required dependency, optional dependency, and accidental dependency.

[0120] In this embodiment, the interaction between the storage system and the computing and network resources refers to the data exchange and communication between them, so that different applications and services can store and access data in different locations.

[0121] In this embodiment, the monitoring requirements of the storage system mainly refer to the monitoring and management of its performance, availability, reliability and other aspects.

[0122] The beneficial effects of the above technical solution are: obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tools based on the monitoring requirements can improve the monitoring capabilities of the object storage system, ensure the compatibility and adaptability of the monitoring tools and the system, and enhance the reliability and credibility of the monitoring data.

[0123] Example 4:

[0124] Based on Example 3, after obtaining the dependency relationship between the application and the service according to the basic settings and configurations, this embodiment of the present invention further includes:

[0125] Split the application into multiple independent microservices based on the microservice architecture and obtain the business logic and data model of each microservice;

[0126] Obtain appropriate storage systems and computing resources based on the business logic and data model of each microservice;

[0127] Establishing a lightweight communication protocol between microservices based on the storage system and computing resources, and determining communication between multiple microservices based on the communication protocol;

[0128] Use monitoring tools to track the performance of each microservice and the communication between multiple microservices;

[0129] Obtain the performance status of each microservice and the communication anomalies between multiple microservices, quickly isolate the abnormal services based on the service grid, and switch the abnormal services to backup services.

[0130] In this embodiment, the microservice architecture splits the application into multiple small services, each of which has its own independent responsibilities and boundaries, and these services can communicate through APIs or other protocols.

[0131] In this embodiment, microservices are one or more services with self-coordination capabilities. Each microservice is independently responsible for one or more business functions and communicates with other microservices through lightweight interfaces.

[0132] In this embodiment, the business logic of a microservice refers to the specific business functions and processing flows that the microservice is responsible for.

[0133] In this embodiment, the data model of a microservice refers to a methodology for describing the internal data structure and organization of a microservice, including regulations and constraints on data representation, relationships, and access.

[0134] In this embodiment, computing resources refer to hardware and software resources that can be used to perform computing tasks.

[0135] In this embodiment, the lightweight communication protocol is a protocol used to implement communication between devices in a distributed system.

[0136] In this embodiment, the performance of a microservice generally refers to the performance of the microservice during deployment and use, including system throughput, latency, and resource utilization.

[0137] In this embodiment, the communication between microservices refers to the message transmission and interaction between microservices.

[0138] In this embodiment, the service grid is a technical framework for solving communication problems between microservices.

[0139] The beneficial effects of the above technical solution are: splitting the application into multiple independent microservices and obtaining the business logic and data model of each microservice, obtaining suitable storage systems and computing resources, establishing a lightweight communication protocol, determining communication anomalies between multiple microservices, and switching abnormal services to backup services. This can maintain the availability and stability of the system when failures or performance problems occur. At the same time, it can reduce downtime and the scope of impact, help ensure business continuity and minimize data loss and other adverse consequences.

[0140] Example 5:

[0141] Based on Example 4, this embodiment of the present invention configures appropriate monitoring rules based on the distributed monitoring tool and actual storage business scenarios, and detects multiple statistical functions of the file system in real time according to the monitoring rules, including:

[0142] Obtain multiple relevant information about actual storage business scenarios;

[0143] Acquire storage device type information and other relevant parameters in the system environment according to the plurality of relevant information;

[0144] Configure appropriate monitoring rules based on the distributed monitoring tool according to the storage device type information and other relevant parameters;

[0145] Obtaining basic information of the file system, and determining attributes of the file system based on the basic information;

[0146] Based on the file system attributes and storage business requirements, multiple statistical functions of the file system are detected in real time based on monitoring rules.

[0147] In this embodiment, multiple pieces of relevant information of the actual storage business scenario include: storage process, data characteristics, and performance requirements.

[0148] In this embodiment, the storage device type information includes: hard disk, solid state drive, capacity, and cache mechanism.

[0149] In this embodiment, other relevant parameters in the system environment include: operating system version, application version, and hardware configuration.

[0150] In this embodiment, the distributed monitoring tool refers to a tool that can implement real-time monitoring and alarming of system resources, applications, network traffic, etc. in a distributed system, such as Zabbix and Nagios.

[0151] In this embodiment, the monitoring rule is an instruction set used to define and configure how to identify, trigger, and process monitoring events in the monitoring tool.

[0152] In this embodiment, the basic information of the file system includes: allocation units, the number of disk blocks occupied by each file in the file system, the number of sectors contained in each allocation unit in the file system, and the version number of the file system.

[0153] In this embodiment, the attributes of the file system refer to a set of characteristics or parameters inherent in the file system that describe the behavior of the file system, such as: the type of the file system, the data structure of the file system, the usage, location and size of each cluster in the file system.

[0154] In this embodiment, the storage service demand refers to the corresponding functions and services provided by the storage system to meet the storage capacity and performance requirements of different business applications, such as storage capacity demand.

[0155] The beneficial effects of the above technical solution are: by configuring appropriate monitoring rules based on the actual storage business scenarios according to the distributed monitoring tool, the pertinence and adaptability of the monitoring rules can be guaranteed, thereby obtaining more accurate and reliable monitoring data. At the same time, according to the monitoring rules, multiple statistical functions of the file system are detected in real time, which can quickly and accurately obtain the functional status of the system, ensure the frequency and coverage of monitoring, and improve monitoring efficiency.

[0156] Example 6:

[0157] Based on Example 5, this embodiment of the present invention obtains real-time performance data of a storage system based on multiple statistical functions, and monitors the working status of the storage system in real time based on the real-time performance data, including:

[0158] Deploy a statistics agent in the storage system and track the activity and performance of each statistical function in real time according to the statistics agent;

[0159] Obtain real-time performance data of the storage system based on the storage environment based on the activity and performance of each statistical function;

[0160] Obtaining system output indicators and log information based on the real-time performance data;

[0161] The output indicators and log information are analyzed, and according to the analysis results, the working status of the storage system is monitored in real time based on preset working indicator parameters.

[0162] In this embodiment, the statistics agent is used to monitor the running status of the application or server and collect various indicator data, such as the creation, reading, and modification of each record.

[0163] In this embodiment, the statistical functions include: file system new increments, deletions, calls, file size changes, file quantity growth, and file status.

[0164] In this embodiment, the storage environment refers to a combination of hardware devices and software systems that provide data storage functions for computer systems and applications, including multiple components such as physical devices, virtualization technology, and storage management software.

[0165] In this embodiment, the real-time performance data includes: number of input / output operations per second and throughput.

[0166] In this embodiment, the output indicators of the system may be: response time, error rate, and delay time.

[0167] In this embodiment, the preset working index parameter refers to an index parameter set in advance, such as a response time of 0.1 to 0.2 seconds.

[0168] The beneficial effects of the above technical solution are: obtaining real-time performance data of the storage system based on the statistical function, monitoring the working status of the storage system in real time based on the real-time performance data, being able to optimize the performance of the storage system in real time, and improving the reliability of the object storage system.

[0169] Example 7:

[0170] Based on Example 6, this embodiment of the present invention determines the health of the object storage according to the working status, and obtains the abnormality and potential cause of the storage system according to the health status, including:

[0171] Obtain metadata information of the object storage, and construct an index table of the object storage based on the metadata information;

[0172] Determine the most recent access time and the most recent modification time of each object according to the index table of the object storage;

[0173] Determine the health of the object storage based on the working status of the storage system according to the most recent access time and the most recent modification time of each object;

[0174] Obtain storage system anomalies based on the health of the object storage and all relevant log information in the storage system;

[0175] According to the abnormal situation, potential causes of the abnormality are obtained based on the code and configuration of the storage system and the application.

[0176] In this embodiment, the metadata information includes: the name, type, creation time, and modification time of the object.

[0177] In this embodiment, the index table of object storage refers to a special type of data structure created in the object storage system to facilitate querying and accessing objects. It organizes the metadata information of the objects according to certain rules and structures, and stores it in the index table in the form of key-value pairs.

[0178] In this embodiment, the working status of the storage system includes:

[0179] Read status: refers to the efficiency and speed of the storage system in reading data from disks or other storage media, including indicators such as average seek time, average return time, and average latency.

[0180] Write status: refers to the efficiency and speed of the storage system writing data to the disk or other storage media, including indicators such as average seek time, average write time and average latency.

[0181] Performance status: refers to the performance of the storage system under different loads, including indicators such as the maximum number of concurrent accesses and throughput.

[0182] Capacity status: refers to the storage space usage and remaining space available in the storage system, including indicators such as total capacity, used capacity, free capacity, and utilization rate.

[0183] In this embodiment, the health status of the object storage refers to the comprehensive performance of the storage system in terms of performance, reliability, and security during normal use.

[0184] In this embodiment, abnormal situations that may occur in the storage system include: storage pool unavailability, data corruption, and virus infection.

[0185] The beneficial effects of the above technical solution are: by building an index table through metadata information, the speed of finding objects in the object storage system can be accelerated, the query efficiency can be improved, and abnormal conditions and potential causes can be obtained according to the health of the object storage, so that problems can be discovered and repaired in time, avoiding data loss and performance degradation problems from causing greater impact on the system. At the same time, by quickly identifying and resolving abnormal conditions, the fault recovery time and downtime can be shortened, and the storage efficiency and monitoring effect can be improved.

[0186] Example 8:

[0187] Based on Example 7, this embodiment of the present invention performs object storage warning notification based on anomalies and potential causes based on monitoring and alarm thresholds to implement object storage monitoring, including:

[0188] Set normal ranges and thresholds for object storage system operating status and code anomalies based on historical log information and storage system load.

[0189] Obtain real-time operation data of the object storage system and application operation data based on anomalies and potential causes;

[0190] Determine the real-time operating status and real-time code information of the object storage system based on the operating data;

[0191] According to the real-time operation status and real-time code information, object storage early warning notification is performed based on the normal range and threshold of the object storage system operation status and code anomalies to realize object storage monitoring.

[0192] In this embodiment, the load of the storage system refers to indicators of the amount of data and the number of requests processed by the storage system within a certain period of time, and reflects the current operating status and performance level of the storage system.

[0193] In this embodiment, the object storage system operating status refers to the inspection, monitoring and analysis of a specific object storage system to determine whether it is in normal working condition and can provide stable and reliable services to users, including normal operation, abnormal operation, and suspended operation.

[0194] In this embodiment, code exceptions refer to errors or abnormal behaviors in the program. In the object storage system, code exceptions include: file read and write errors, permission access failure, file size limit exceeded, file does not exist, network connection problems, system downtime, etc.

[0195] In this embodiment, potential causes of object storage system abnormalities include: programming errors, CPU overheating, memory leaks, software vulnerabilities, network failures, and database failures.

[0196] In this embodiment, the real-time operation data of the object storage system refers to various indicators and log information generated by the storage system within a specific time period, such as: read and write operations, IOPS, throughput, latency, capacity utilization, and access patterns.

[0197] In this embodiment, the running data of the application refers to various indicators and log information generated during the running of the application, such as CPU usage, memory usage, and network traffic.

[0198] In this embodiment, the real-time code information of the object storage system generally refers to the internal implementation code or middleware code of the system, which includes the core functions and logical implementation of the object storage system, such as: algorithms and implementations of file system management, permission control, caching mechanism, asynchronous operations, etc.

[0199] The beneficial effects of the above technical solution are: object storage early warning notifications are carried out based on monitoring and alarm thresholds according to anomalies and potential causes, object storage monitoring is realized, anomalies in the storage system can be discovered in time, the problem can be avoided from expanding and the scope of impact can be expanded, and efficient monitoring and management of the object storage system can be achieved.

[0200] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for monitoring object storage, characterized in that: include: Step 1: Obtain the monitoring requirements of the storage system and obtain the corresponding distributed monitoring tools based on the monitoring requirements; Step 2: Configure appropriate monitoring rules based on the distributed monitoring tool and the actual storage business scenario, and detect multiple statistical functions of the file system in real time according to the monitoring rules; Step 3: Acquire real-time performance data of the storage system according to multiple statistical functions, and monitor the working status of the storage system in real time according to the real-time performance data; Step 4: Determine the health of the object storage based on the working status, and obtain storage system abnormalities and potential causes based on the health status; Step 5: Provide object storage warning notifications based on monitoring and alarm thresholds based on anomalies and potential causes, thus enabling object storage monitoring. Obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tools based on the monitoring requirements includes: Obtain the basic settings and configuration of the storage system, and based on the basic settings and configuration, obtain the dependencies of applications and services; determining the interaction of storage systems with computing and network resources based on dependencies between said applications and services; Obtain important events of the storage system based on the usage and performance requirements of the storage system; Determine the key areas that need to be monitored based on the important events; Determining monitoring requirements for the storage system based on its interaction with computing and network resources and key areas; Obtain corresponding distributed monitoring tools based on industry standards according to the monitoring requirements; The multiple statistical functions include: file system new increments, deletions, calls, file size changes, file quantity growth, and file status.

2. The object storage monitoring method according to claim 1, characterized in that: Before obtaining the monitoring requirements of the storage system and obtaining the corresponding distributed monitoring tool according to the monitoring requirements, the following steps are also included: Obtain historical monitoring events of the storage system, determine the monitoring object based on the historical monitoring events, and obtain the data attributes of the monitoring object; Determine the storage space of the monitored object based on data attributes, and determine the dormant storage space and active storage space in the storage system based on the distribution of the storage space; Determine data flow parameters corresponding to the dormant storage space and the active storage space respectively, and determine data storage selection gain factors corresponding to the dormant storage space and the active storage space respectively according to the data flow parameters; respectively obtaining virtual space configuration parameters of the dormant storage space and the active storage space, and determining monitoring factors for the dormant storage space and the active storage space according to the virtual space configuration parameters; Building monitoring rules for the dormant storage space and the active storage space based on the monitoring factor and the data storage selection gain factors of the dormant storage space and the active storage space; Configure monitoring policies for dormant storage spaces and active storage spaces based on monitoring rules. Determine the respective priority monitoring indicators of the dormant storage space and the active storage space according to the monitoring policy, and obtain the mapped virtual resources of the priority monitoring indicators; Evaluate the abnormal attributes of the monitoring status of the dormant storage space and the active storage space respectively through a preset virtual call function according to the mapped virtual resources; Optimize the storage resources of the dormant storage space and the active storage space based on the abnormal attributes of the monitoring status to ensure that they meet the monitoring requirements; The data storage selection gain factor of the dormant storage space refers to the strength of the relationship between the probability of accessing a data block and the time point when the data block was last accessed when the storage system is in the dormant state. When a data block has not been accessed for a long time, the data storage selection gain factor of the dormant storage space will increase, indicating that the system is more likely to continue to use the data block in future accesses; The data storage selection gain factor of the active storage space refers to the relationship between the access frequency and time of the data block within a given time period when the storage system is in an active state. The more frequently a data block is accessed, the larger the data storage selection gain factor of the active storage space is, and the longer a data block is not accessed, the smaller the data storage selection gain factor of the active storage space is.

3. The object storage monitoring method according to claim 2, characterized in that: After obtaining the dependencies of the application and services according to the basic setup and configuration, it also includes: Split the application into multiple independent microservices based on the microservice architecture and obtain the business logic and data model of each microservice; Obtain appropriate storage systems and computing resources based on the business logic and data model of each microservice; Establishing a lightweight communication protocol between microservices based on the storage system and computing resources, and determining communication between multiple microservices based on the communication protocol; Use monitoring tools to track the performance of each microservice and the communication between multiple microservices; Obtain the performance status of each microservice and the communication anomalies between multiple microservices, quickly isolate the abnormal services based on the service grid, and switch the abnormal services to backup services.

4. The object storage monitoring method according to claim 1, wherein: Configure appropriate monitoring rules based on the distributed monitoring tool and the actual storage business scenario, and detect multiple statistical functions of the file system in real time according to the monitoring rules, including: Obtain multiple relevant information about actual storage business scenarios; Acquire storage device type information and other relevant parameters in the system environment according to the plurality of relevant information; Configure appropriate monitoring rules based on the distributed monitoring tool according to the storage device type information and other relevant parameters; Obtaining basic information of the file system, and determining attributes of the file system based on the basic information; Based on the file system attributes and storage business requirements, multiple statistical functions of the file system are detected in real time based on monitoring rules.

5. The object storage monitoring method according to claim 1, wherein: Acquiring real-time performance data of the storage system according to multiple statistical functions, and monitoring the working status of the storage system in real time according to the real-time performance data, including: Deploy a statistics agent in the storage system and track the activity and performance of each statistical function in real time according to the statistics agent; Obtain real-time performance data of the storage system based on the storage environment based on the activity and performance of each statistical function; Obtaining system output indicators and log information based on the real-time performance data; The output indicators and log information are analyzed, and according to the analysis results, the working status of the storage system is monitored in real time based on preset working indicator parameters.

6. The object storage monitoring method according to claim 1, wherein: Determining the health of the object storage based on the working status, and obtaining storage system abnormalities and potential causes based on the health status, including: Obtain metadata information of the object storage, and construct an index table of the object storage based on the metadata information; Determine the most recent access time and the most recent modification time of each object according to the index table of the object storage; Determine the health of the object storage based on the working status of the storage system according to the most recent access time and the most recent modification time of each object; Obtain storage system anomalies based on the health of the object storage and all relevant log information in the storage system; According to the abnormal situation, potential causes of the abnormality are obtained based on the code and configuration of the storage system and the application.

7. The object storage monitoring method according to claim 1, wherein: Provide object storage warning notifications based on monitoring and alarm thresholds based on anomalies and potential causes, enabling object storage monitoring, including: Set normal ranges and thresholds for object storage system operating status and code anomalies based on historical log information and storage system load. Obtain real-time operation data of the object storage system and application operation data based on anomalies and potential causes; Determine the real-time operating status and real-time code information of the object storage system based on the operating data; According to the real-time operation status and real-time code information, object storage early warning notification is performed based on the normal range and threshold of the object storage system operation status and code anomalies to realize object storage monitoring.

Images