A storage management method, apparatus, device, and machine-readable storage medium

Abstract

This disclosure provides a storage management method, apparatus, device, and machine-readable storage medium. The method includes: configuring storage node devices with high read priority; obtaining storage access services requiring low latency reads and attaching the storage access services to the high read priority storage node devices; and having the high read priority storage node devices return associated stored data based solely on the access requests of the storage access services requiring low latency reads. Through the technical solution of this disclosure, by configuring high read priority storage nodes according to the volume of read services, and having these high read priority storage nodes dedicated to handling corresponding read services, the problem of poor read performance caused by continuous large-scale data writes is improved with minimal impact on the overall data write performance of the storage cluster devices, and the data return latency of read services is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a storage management method, apparatus, device and machine-readable storage medium. Background Technology

[0002] With the development of technologies such as high-definition video, image processing, and video surveillance, user data capacity is increasing rapidly, and users' demands for read and write performance of stored data are also rising simultaneously. When purchasing storage products, users typically specify performance requirements to meet business needs. In many business scenarios, storage performance is not just a matter of speed; it can even affect the normal operation of the business itself. For example, in document digitization, archives and libraries digitize paper books and store them on storage servers. As more and more digitized books are stored, the capacity of the storage cluster will increase significantly. Simultaneously, a large number of users will access the system concurrently. If the read and write performance of the storage cluster is poor and the read / write latency is high, it will reduce user experience and hinder the progress of digitization. Therefore, improving the read and write performance of the storage cluster is crucial. In a video surveillance application scenario, when retrieving surveillance video, the large volume of video writes often affects the video surveillance reading speed, impacting the application. In this case, it is urgent to solve the read performance problem. Summary of the Invention

[0003] In view of this, the present disclosure provides a storage management method, apparatus, electronic device, and machine-readable storage medium to improve the aforementioned problem of poor read performance caused by continuous large-scale data writing.

[0004] The specific technical solution is as follows:

[0005] This disclosure provides a storage management method applied to a storage cluster device. The method includes: configuring a high read priority storage node device, wherein the read priority of the high read priority storage node device is higher than the write priority, and higher than the read and write priority of non-high read priority storage node devices; obtaining a storage access service requesting low latency read access, and attaching the storage access service to the high read priority storage node device; and having the high read priority storage node device return associated stored data based solely on the access request of the storage access service requesting low latency read access.

[0006] As a technical solution, configuring a storage node device with high read priority, wherein the high read priority storage node device is one whose read request priority is higher than the write request priority, and higher than the read request and write request priorities of storage node devices with non-high read priority, includes: reading a pre-saved configuration file, and configuring the storage node device associated with the read priority configuration information as a high read priority storage node device according to the read priority configuration information recorded in the configuration file.

[0007] As a technical solution, the step of acquiring storage access services that require low read latency and attaching such storage access services to storage node devices with high read priority includes: acquiring storage access services that require low read latency according to the service type, wherein the service type includes monitoring video playback and / or image processing.

[0008] As a technical solution, the step of having only high-priority storage node devices read low-latency storage access service access requests according to the requirements and returning associated stored data includes: having only high-priority storage node devices read low-latency storage access service access requests according to the requirements, generating high-priority data read requests, adding high-priority data read requests to the head of the request processing queue, and reading and returning associated stored data according to high-priority data read requests.

[0009] This disclosure also provides a storage management device applied to a storage cluster device. The device includes: a configuration module for configuring high read priority storage node devices, wherein the high read priority storage node device has a higher priority for read requests than for write requests, and higher than the priority of read requests and write requests from non-high read priority storage node devices; a mounting module for acquiring storage access services requiring low latency reads and mounting such storage access services to the high read priority storage node devices; and a data module for returning associated stored data by the high read priority storage node devices only, based on the access requests from the storage access services requiring low latency reads.

[0010] As a technical solution, configuring a storage node device with high read priority, wherein the high read priority storage node device is one whose read request priority is higher than the write request priority, and higher than the read request and write request priorities of storage node devices with non-high read priority, includes: reading a pre-saved configuration file, and configuring the storage node device associated with the read priority configuration information as a high read priority storage node device according to the read priority configuration information recorded in the configuration file.

[0011] As a technical solution, the step of acquiring storage access services that require low read latency and attaching such storage access services to storage node devices with high read priority includes: acquiring storage access services that require low read latency according to the service type, wherein the service type includes monitoring video playback and / or image processing.

[0012] As a technical solution, the step of having only high-priority storage node devices read low-latency storage access service access requests according to the requirements and returning associated stored data includes: having only high-priority storage node devices read low-latency storage access service access requests according to the requirements, generating high-priority data read requests, adding high-priority data read requests to the head of the request processing queue, and reading and returning associated stored data according to high-priority data read requests.

[0013] This disclosure also provides an electronic device, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the aforementioned storage management method.

[0014] This disclosure also provides a machine-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the aforementioned storage management method.

[0015] The technical solution provided in this disclosure brings at least the following beneficial effects:

[0016] Based on the volume of read operations, high-priority storage nodes are configured to handle the corresponding read operations. This improves the poor read performance caused by continuous large amounts of data writing with minimal impact on the overall data write performance of the storage cluster, and reduces the data return latency of read operations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments of this disclosure or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this disclosure.

[0018] Figure 1 This is a flowchart of a storage management method according to one embodiment of the present disclosure;

[0019] Figure 2This is a structural diagram of a storage management device according to one embodiment of the present disclosure;

[0020] Figure 3 This is a hardware structure diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0021] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0023] Distributed file system architectures suffer from read / write performance preemption issues, specifically how to guarantee read performance requirements under heavy write loads. In video surveillance scenarios, video is written continuously throughout the day, with even higher write volumes at busy intersections. In emergencies requiring video retrieval, the speed of retrieval cannot be guaranteed. High write volumes lead to insufficient read performance, resulting in slow video reading and long operation times.

[0024] One technical solution can enhance storage cluster capabilities, improve read and write performance, and meet the requirements of read services. The optimization plan includes: streamlining the storage cluster I / O process, i.e., the read and write service processing flow; adding time markers to critical I / O processes for latency statistics; analyzing the time consumption of read and write performance at each stage when service interruptions occur to identify bottlenecks; identifying time-consuming points in read services and then optimizing performance for different modules; and increasing cache capacity to improve cache hit rate.

[0025] However, increasing the cache capacity still presents problems in scenarios with a large number of write operations. A large number of write operations will generate many write requests, which will continuously update the contents of the cache, causing the read cache to become invalid. In addition, when there are a large number of write operations, the disk will also generate a high load due to the large number of writes, and the corresponding latency will also increase when responding to read requests.

[0026] In view of this, the present disclosure provides a storage management method, apparatus, electronic device, and machine-readable storage medium to improve the aforementioned problem of poor read performance caused by continuous large-scale data writing.

[0027] The specific technical solution is described below.

[0028] In one embodiment, this disclosure provides a storage management method applied to a storage cluster device. The method includes: configuring a high read priority storage node device, wherein the read priority of the high read priority storage node device is higher than the write priority, and higher than the read and write priorities of non-high read priority storage node devices; obtaining a storage access service requesting low latency read access, and attaching the storage access service to the high read priority storage node device; and having the high read priority storage node device return associated stored data based solely on the access request of the storage access service requesting low latency read access.

[0029] Specifically, such as Figure 1 This includes the following steps:

[0030] Step S11: Configure storage node devices with high read priority;

[0031] Step S12: Obtain the storage access service that requires low read latency and attach the storage access service to the storage node device with high read priority;

[0032] Step S13: Return the associated stored data.

[0033] Based on the volume of read traffic, storage nodes with high read priority are configured. These high-priority nodes are dedicated to handling corresponding read traffic, thus improving the poor read performance caused by continuous large-scale data writes and reducing data return latency for read traffic with minimal impact on the overall data write performance of the storage cluster. This implementation method meets the read latency requirements of upper-layer services without increasing costs and with minimal impact on write traffic, thus simultaneously satisfying the performance requirements of both read and write traffic.

[0034] In one implementation, configuring a storage node device with high read priority, wherein the high read priority storage node device is one whose read request priority is higher than the write request priority, and higher than the read request and write request priorities of storage node devices with non-high read priority, includes: reading a pre-saved configuration file, and configuring the storage node device associated with the read priority configuration information as a high read priority storage node device according to the read priority configuration information recorded in the configuration file.

[0035] In one implementation, obtaining a storage access service requiring low read latency and attaching the storage access service to a storage node device with high read priority includes: obtaining a storage access service requiring low read latency based on the service type, wherein the service type includes surveillance video playback and / or image processing.

[0036] In one implementation, the step of having only high-read-priority storage node devices read access requests for low-latency storage access services according to the requirements and returning associated stored data includes: having only high-read-priority storage node devices read access requests for low-latency storage access services according to the requirements, generating high-priority data read requests, adding high-priority data read requests to the head of request processing queues, and reading and returning associated stored data according to high-priority data read requests.

[0037] In one implementation, specific storage nodes are selected as high-read-priority storage nodes based on the specifications of the storage cluster. These high-read-priority storage nodes can be configured via a configuration file. When a storage node starts up, it checks the configuration file; if a high-read-priority flag is found, the priority of read requests from this node is increased. Storage nodes in the cluster without a high-read-priority flag maintain a consistent read / write priority, i.e., normal priority. Low-latency services, such as video retrieval services, are mounted to high-read-priority storage nodes. Read requests from these high-read-priority nodes have the highest priority in the storage cluster's I / O request processing queue, resulting in the fastest response time and ensuring low latency for related services.

[0038] In one embodiment, this disclosure also provides a storage management device, such as... Figure 2 This device, applied to a storage cluster, includes: a configuration module 21 for configuring high-read-priority storage node devices, wherein the read request priority of the high-read-priority storage node device is higher than the write request priority, and higher than the read request and write request priorities of non-high-read-priority storage node devices; a mounting module 22 for acquiring storage access services requiring low-latency reads and mounting such storage access services to the high-read-priority storage node devices; and a data module 23 for returning associated stored data by the high-read-priority storage node devices based solely on the access requests of the low-latency read storage access services.

[0039] In one implementation, configuring a storage node device with high read priority, wherein the high read priority storage node device is one whose read request priority is higher than the write request priority, and higher than the read request and write request priorities of storage node devices with non-high read priority, includes: reading a pre-saved configuration file, and configuring the storage node device associated with the read priority configuration information as a high read priority storage node device according to the read priority configuration information recorded in the configuration file.

[0040] In one implementation, obtaining a storage access service requiring low read latency and attaching the storage access service to a storage node device with high read priority includes: obtaining a storage access service requiring low read latency based on the service type, wherein the service type includes surveillance video playback and / or image processing.

[0041] In one implementation, the step of having only high-read-priority storage node devices read access requests for low-latency storage access services according to the requirements and returning associated stored data includes: having only high-read-priority storage node devices read access requests for low-latency storage access services according to the requirements, generating high-priority data read requests, adding high-priority data read requests to the head of request processing queues, and reading and returning associated stored data according to high-priority data read requests.

[0042] The implementation methods of the apparatus are the same as or similar to the corresponding implementation methods, and will not be described again here.

[0043] In one embodiment, this disclosure provides an electronic device including a processor and a machine-readable storage medium. The machine-readable storage medium stores machine-executable instructions that can be executed by the processor. The processor executes the machine-executable instructions to implement the aforementioned storage management method. From a hardware perspective, a hardware architecture diagram can be found... Figure 3 As shown.

[0044] In one embodiment, this disclosure provides a machine-readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the aforementioned storage management method.

[0045] Here, a machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0046] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0047] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this disclosure, the functions of each unit can be implemented in one or more software and / or hardware.

[0048] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, embodiments of this disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0049] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0050] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0051] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0052] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (which may include, but are not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] The above description is merely an embodiment of this disclosure and is not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

1. A storage management method, characterized in that, Applied to storage cluster devices, the method includes: Configure storage node devices with high read priority, wherein the read priority of the storage node device is higher than the write priority, and higher than the read and write priority of storage node devices with non-high read priority; Obtain storage access requests that require low latency reads and attach these requests to storage node devices with high read priority. The storage node device with high read priority reads the access request of the low-latency storage access service according to the requirements and returns the associated stored data. The configuration of a high-read-priority storage node device, wherein the high-read-priority storage node device is one in which the read request priority is higher than the write request priority, and higher than the read request and write request priorities of non-high-read-priority storage node devices, includes: Read the pre-saved configuration file and, based on the read priority configuration information recorded in the configuration file, configure the storage node device associated with that read priority configuration information as a high read priority storage node device.

2. The method according to claim 1, characterized in that, The process of acquiring a storage access service that requires low latency reads and attaching that storage access service to a storage node device with high read priority includes: Based on the service type, obtain storage access services that require low latency, such as surveillance video playback and / or image processing.

3. The method according to claim 1, characterized in that, The access request for low-latency storage access services, which is read by only the storage node device with high read priority according to the requirements, and returns the associated stored data, includes: The storage node device with high read priority reads the access request of the low-latency storage access service according to the requirements, generates a high-priority data read request, adds the high-priority data read request to the head of the request processing queue, and reads and returns the associated stored data according to the high-priority data read request.

4. A storage management device, characterized in that, Applied to storage cluster devices, the apparatus includes: A configuration module is used to configure storage node devices with high read priority. The storage node device with high read priority is one whose read request priority is higher than the write request priority, and also higher than the read request and write request priorities of storage node devices with non-high read priority. The mounting module is used to acquire storage access services that require low read latency and mount the storage access service to a storage node device with high read priority. The data module is used to read access requests for low-latency storage access services by storage node devices with high read priority according to the requirements, and return the associated stored data. The configuration of a high-read-priority storage node device, wherein the high-read-priority storage node device is one in which the read request priority is higher than the write request priority, and higher than the read request and write request priorities of non-high-read-priority storage node devices, includes: Read the pre-saved configuration file and, based on the read priority configuration information recorded in the configuration file, configure the storage node device associated with that read priority configuration information as a high read priority storage node device.

5. The apparatus according to claim 4, characterized in that, The process of acquiring a storage access service that requires low latency reads and attaching that storage access service to a storage node device with high read priority includes: Based on the service type, obtain storage access services that require low latency, such as surveillance video playback and / or image processing.

6. The apparatus according to claim 4, characterized in that, The access request for low-latency storage access services, which is read by only the storage node device with high read priority according to the requirements, and returns the associated stored data, includes: The storage node device with high read priority reads the access request of the low-latency storage access service according to the requirements, generates a high-priority data read request, adds the high-priority data read request to the head of the request processing queue, and reads and returns the associated stored data according to the high-priority data read request.

7. An electronic device, characterized in that, include: A processor and a machine-readable storage medium storing machine-executable instructions that can be executed by the processor to implement the method of any one of claims 1-3.

8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method described in any one of claims 1-3.

Images