Distributed storage service request processing method, device and distributed storage system

Abstract

The present invention relates to the field of computer technology, and discloses a distributed storage service request processing method, device and distributed storage system, the method comprising: obtaining the current total service delay and current service throughput of the distributed storage system; determining the current allowable value of the service request throttle according to the current total service delay and current service throughput of the distributed storage system; when receiving multiple service requests sent by a user end, transmitting the corresponding number of service requests to the target task queue of the target storage node according to the current allowable value of the service request throttle; and storing the service data in the corresponding target storage device in response to the service requests in the target task queue. By limiting the number of service requests sent to the target storage node according to the current allowable value of the service request throttle, the system service delay and service throughput are guaranteed, and congestion of service requests is avoided, which lays a foundation for improving the storage service quality of the distributed storage system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a distributed storage service request processing method, device, and distributed storage system. Background Art

[0002] With the development of modern information technology and the widespread use of smart devices, the volume and type of data are rapidly increasing, and traditional storage systems can no longer meet the large-scale and diverse data storage needs. The rise of big data and cloud computing technologies has provided more application scenarios for distributed storage systems and promoted the development of distributed storage technology.

[0003] In the related art, a storage node in a distributed storage system usually receives a service request sent by a user, and then transmits the service request to a task queue of a corresponding storage device.

[0004] However, when any storage node in the distributed storage system fails, data recovery and backfilling will be performed after the failure is repaired. During this process, if users send a large number of business requests to the storage node, it will cause business request congestion, which is not conducive to ensuring the storage service quality of the distributed storage system. Summary of the Invention

[0005] The present application provides a distributed storage service request processing method, device and distributed storage system to solve the defects of related technologies that may cause congestion of service requests, which is not conducive to ensuring the storage service quality of the distributed storage system.

[0006] The first aspect of the present application provides a distributed storage service request processing method, comprising:

[0007] Obtain the current total service latency and current service throughput of the distributed storage system;

[0008] Determining a current allowable value of a service request throttle according to a current service total delay and a current service throughput of the distributed storage system;

[0009] When receiving multiple service requests sent by the user terminal, transmitting a corresponding number of service requests to the target task queue of the target storage node according to the current allowed value of the service request throttle;

[0010] In response to the service request in the target task queue, the service data is stored in the corresponding target storage device.

[0011] In an optional implementation, determining the current allowed value of the service request throttle according to the current service total delay and the current service throughput of the distributed storage system includes:

[0012] Determining a target total service delay of the distributed storage system according to a current total service delay and a current service throughput of the distributed storage system;

[0013] A current allowable value of a service request throttle is determined according to the current service total delay of the distributed storage system and the target service total delay.

[0014] In an optional implementation, determining a target total service delay of the distributed storage system according to a current total service delay and a current service throughput of the distributed storage system includes:

[0015] Constructing an objective function based on the current service total delay and current service throughput of the distributed storage system;

[0016] According to the objective function and in accordance with a first preset period, the target total service delay of the distributed storage system is updated.

[0017] In an optional implementation, constructing an objective function based on the current service total delay and current service throughput of the distributed storage system includes:

[0018] Determining a performance measurement indicator of the distributed storage system according to a current service total delay and a current service throughput of the distributed storage system;

[0019] An objective function is constructed according to the calculation formula of the performance measurement indicator.

[0020] In an optional implementation, constructing an objective function according to a calculation formula of the performance measurement indicator includes:

[0021] According to the calculation formula of the performance measurement index, the following objective function is constructed:

[0022]

[0023] in, Indicates the total delay of the current business. Indicates the total delay of the target business before the update. Indicates the current service throughput. Indicates the throughput adjustment parameter.

[0024] In an optional implementation, updating the target total service delay of the distributed storage system according to the objective function includes:

[0025] Update the target total service delay of the distributed storage system based on the following formula:

[0026]

[0027] in, Indicates the updated total delay of the target business, Indicates the total delay of the target business before the update. represents the preset learning rate, represents the partial derivative of the objective function with respect to the total delay of the target service, represents the partial derivative of the objective function with respect to the throughput adjustment parameter.

[0028] In an optional implementation, determining a current allowable value of a service request throttle according to the current total service delay of the distributed storage system and the target total service delay includes:

[0029] determining an adjustment direction of an allowable value of the service request throttle according to a magnitude relationship between a current service total delay of the distributed storage system and the target service total delay;

[0030] The current allowed value of the service request throttle is determined according to the allowed value adjustment direction of the service request throttle and the difference between the current service total delay and the target service total delay.

[0031] In an optional implementation, determining the adjustment direction of the allowable value of the service request throttle according to the magnitude relationship between the current service total delay of the distributed storage system and the target service total delay includes:

[0032] When a magnitude relationship between a current service total delay of the distributed storage system and the target service total delay indicates that the current service total delay is greater than the target service total delay, determining that an adjustment direction of the allowable value of the service request throttle is a decreasing direction;

[0033] When the magnitude relationship between the current total service delay of the distributed storage system and the target total service delay indicates that the current total service delay is not greater than the target total service delay, the adjustment direction of the allowable value of the service request throttle is determined to be an increasing direction.

[0034] In an optional implementation, determining the current allowed value of the service request throttle according to the allowed value adjustment direction of the service request throttle and the difference between the current service total delay and the target service total delay includes:

[0035] determining, according to a second preset period, a current allowed value of the service request throttle according to an allowed value adjustment direction of the service request throttle and a difference between the current service total delay and the target service total delay;

[0036] The second preset period is smaller than the first preset period.

[0037] In an optional embodiment, when receiving multiple service requests sent by the user terminal, transmitting a corresponding number of service requests to the target task queue of the target storage node according to the current allowed value of the service request throttle valve includes:

[0038] Based on the front-end scheduling device, multiple service requests sent by the user end are received and the received service requests are cached in the front-end queue;

[0039] According to the current allowed value of the service request throttle, a corresponding number of service requests in the front-end queue are transmitted to the target task queue of the target storage node.

[0040] In an optional implementation, the transmitting a corresponding number of service requests in the front-end queue to a target task queue of a target storage node according to a current allowed value of the service request throttle includes:

[0041] Determining a target length of a task queue of each storage node in the distributed storage system according to a current allowed value of the service request throttle;

[0042] When multiple service requests are received from the user end, a corresponding number of service requests are scheduled from the front-end queue according to the target length of the task queue and the current capacity occupancy of the target task queue of the target storage node, and the service requests are transmitted to the target task queue of the target storage node.

[0043] In an optional embodiment, storing the business data in a corresponding target storage device in response to the business request in the target task queue includes:

[0044] In response to a business request in the target task queue, split the business request into a data request and a metadata request, and cache the metadata request in a metadata queue;

[0045] According to the data request and metadata request, the data to be stored and the metadata are associated and stored in the corresponding target storage device;

[0046] The business data includes the data to be stored and metadata.

[0047] A second aspect of the present application provides a distributed storage service request processing device, comprising:

[0048] The acquisition module is used to obtain the current total service delay and current service throughput of the distributed storage system;

[0049] A determination module, configured to determine a current allowable value of a service request throttle according to a current total service delay and a current service throughput of the distributed storage system;

[0050] a processing module configured to, upon receiving a plurality of service requests sent by a user terminal, transmit a corresponding number of service requests to a target task queue of a target storage node according to a current allowable value of the service request throttle;

[0051] The response module is used to respond to the service request in the target task queue and store the service data in the corresponding target storage device.

[0052] A third aspect of the present application provides a distributed storage system, including:

[0053] Multiple storage nodes, each storage node includes multiple storage devices;

[0054] Any of the storage nodes processes the service request based on the method described in the first aspect and various possible designs of the first aspect to store the service data of the service request in the corresponding target storage device.

[0055] A fourth aspect of the present application provides an electronic device, comprising: at least one processor and a memory;

[0056] The memory stores computer-executable instructions;

[0057] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method described in the first aspect and various possible designs of the first aspect.

[0058] The fifth aspect of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect is implemented.

[0059] The sixth aspect of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the method described in the first aspect and various possible designs of the first aspect.

[0060] The technical solution of this application has the following advantages:

[0061] The present application provides a distributed storage service request processing method, device and distributed storage system, the method comprising: obtaining the current total service delay and current service throughput of the distributed storage system; determining the current allowed value of the service request throttle according to the current total service delay and current service throughput of the distributed storage system; when receiving multiple service requests sent by the user end, transmitting the corresponding number of service requests to the target task queue of the target storage node according to the current allowed value of the service request throttle; and storing the service data in the corresponding target storage device in response to the service requests in the target task queue. The method provided by the above scheme limits the number of service requests sent to the target storage node according to the current allowed value of the service request throttle, thereby ensuring the system service delay and service throughput, avoiding congestion of service requests, and laying the foundation for improving the storage service quality of the distributed storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0063] Figure 1 A flowchart of a distributed storage service request processing method provided in an embodiment of the present application;

[0064] Figure 2 A flowchart of an exemplary distributed storage service request processing method provided in an embodiment of the present application;

[0065] Figure 3 A network architecture diagram of a distributed storage system provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of the structure of a distributed storage service request processing device provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of the structure of a distributed storage system provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of the structure of an exemplary distributed storage system provided in an embodiment of the present application;

[0069] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0070] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0071] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0072] In addition, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. In the description of the following embodiments, "plurality" means more than two, unless otherwise specifically defined.

[0073] With the development of modern information technology and the prevalence of smart devices, the volume and types of data are rapidly increasing. Traditional storage systems are no longer able to meet the large-scale and diverse data storage needs. The rise of big data and cloud computing has opened up new application scenarios for distributed storage systems, driving the development of distributed storage technology. Traditional file systems cannot compete with distributed storage systems because distributed storage can tolerate common storage failures and scale to thousands of machines. Distributed storage systems aim to provide high-performance, high-reliability, and scalable storage solutions. Distributed storage systems are designed based on scalability and fault tolerance. They utilize a distributed architecture, slicing data and distributing it across multiple nodes as objects. They ensure capacity and load balancing across nodes, enabling data access and management through network connections, while also providing a unified storage interface. High reliability: Data redundancy and automatic fault recovery mechanisms ensure data reliability and persistence. High scalability: Distributed storage system clusters can dynamically scale out to support hundreds or even thousands of nodes. Unified interface: A unified file system, object storage, and block storage interface is provided for convenient data access and management.

[0074] Development trends in distributed storage system technology include more intelligent data management and scheduling, more efficient data balancing and storage technologies, and more secure and reliable data protection and backup technologies. Consequently, distributed storage resource management and I / O scheduling face significant challenges. The storage backend, at the lowest level, is responsible for storing distributed storage client system application data on storage devices, such as disks, solid-state drives, or NVMe SSDs. To more efficiently store data, the storage backend buffers requests in an I / O processing queue and flushes the request data in the queue to the storage device in batches according to certain management and scheduling algorithms. Request scheduling is a key technology for distributed storage systems to meet Quality of Service (QoS) objectives. Request scheduling balances resource allocation, ensures throughput and latency for business applications, and provides performance control isolation.

[0075] With the increasing amount of stored data and the continuous expansion of business scale, it is necessary to ensure high quality of service (QoS) of distributed storage systems. Applying technologies that specify storage performance for distributed storage systems to ensure that specific applications and dynamic workloads always obtain specific access performance has always been the focus of the storage service industry and academic research.

[0076] In related technologies, storage system I / O is typically managed by separating the front-end request scheduling module from the storage back-end module. The front-end request scheduling control module determines how to allocate and process client requests by setting static thresholds during the planning and deployment phases, thereby ensuring the QoS of the storage system. The storage back-end module is responsible for processing data storage I / O and evenly distributing data across storage system devices. When a distributed storage system client business application requests storage data I / O, it first passes through the request scheduling module layer for resource allocation, ensuring the business application's throughput and latency, as well as performance control isolation. The request is then buffered in the I / O processing queue by the storage back-end module, and the request data in the queue is flushed to the storage device in batches.

[0077] With the continuous development of heterogeneous storage devices, storage devices such as disks, solid-state drives, or NVMe SSDs have emerged. They need to be set up in the same distributed storage system cluster through heterogeneous deployment solutions, and different request scheduling strategies need to be formulated based on the performance and capacity characteristics of different storage devices.

[0078] In the dynamic workload requests of a distributed storage system, for example, in the event of a failure, the storage needs to perform data self-recovery and data balancing, which will consume IO resources. At this time, when the client suddenly makes a large number of IO requests, the design of separating the request scheduling module from the storage back-end module will not be able to balance the processing volume of the front-end scheduling and the back-end data storage. Because the two modules are separated and decoupled, it is impossible to monitor the processing status and delay of the two modules in a timely and effective manner. On the other hand, the request scheduling control module needs to set static thresholds in advance to ensure QoS, which has limitations. The front-end refers to the request scheduling module, and the back-end refers to the storage back-end module. Only when the front-end request scheduling processing meets a sufficient number can the scheduling processing be effectively carried out. At the same time, the back-end needs to meet a sufficient processing volume to ensure the IO performance of the data storage. If there is a difference in the number of requests from the front-end and the back-end during this process, the IO delay cannot be automatically adjusted and weighed, which will cause the performance of the distributed storage system to gradually decline.

[0079] When the front-end scheduling process is high, but the back-end data storage queue has few requests, the scheduling mechanism can meet the expected QoS settings. However, due to the low number of requests in the back-end data storage queue, insufficient I / O pressure will reduce the efficiency of distributed storage I / O request processing, and the storage device will remain idle and dormant.

[0080] When the front-end scheduling process has few requests, the back-end data storage queue is overwhelmed. When the back-end storage queue is overwhelmed with requests and a large amount of data is written to the storage system, write requests become congested in the buffer, waiting to be processed. Storage I / O congestion ultimately causes the system to process this data slowly, increasing response time latency and affecting system performance and throughput.

[0081] In response to the above problems, an embodiment of the present application provides a distributed storage service request processing method, device and distributed storage system, the method comprising: obtaining the current total service delay and current service throughput of the distributed storage system; determining the current allowed value of the service request throttle according to the current total service delay and current service throughput of the distributed storage system; when receiving multiple service requests sent by the user end, transmitting the corresponding number of service requests to the target task queue of the target storage node according to the current allowed value of the service request throttle; and storing the service data in the corresponding target storage device in response to the service requests in the target task queue. The method provided by the above scheme limits the number of service requests sent to the target storage node according to the current allowed value of the service request throttle, thereby ensuring the system service delay and service throughput, avoiding congestion of service requests, and laying the foundation for improving the storage service quality of the distributed storage system.

[0082] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following describes the embodiments of the present invention in conjunction with the accompanying drawings.

[0083] The present invention provides a method for processing distributed storage service requests, which is used to process service requests sent by a user terminal to a distributed storage system to avoid service request congestion in the distributed storage system. The embodiment of the present invention is performed by an electronic device, such as a server, a desktop computer, a laptop computer, a tablet computer, or other electronic device capable of processing distributed storage service requests.

[0084] like Figure 1 FIG. 1 is a flow chart of a distributed storage service request processing method provided in an embodiment of the present application, the method comprising:

[0085] Step 101: Obtain the current total service delay and current service throughput of the distributed storage system.

[0086] Specifically, based on a preset data collection device, the total service delay and service throughput of the distributed storage system can be monitored in real time to obtain the current total service delay and current service throughput of the distributed storage system.

[0087] Step 102: Determine the current allowable value of the service request throttle according to the current total service delay and the current service throughput of the distributed storage system.

[0088] It should be noted that the storage front-end sends service requests to the storage back-end (storage node) through the service request throttle. The allowed value of the service request throttle is used to limit the number of service requests sent by the storage front-end to the back-end.

[0089] Specifically, the current allowed value of the service request throttle is determined according to the current total service delay and current service throughput of the distributed storage system, thereby ensuring that the allowed value adopted by the service request throttle complies with the current actual load of the distributed storage system.

[0090] Step 103 : When multiple service requests are received from the user end, a corresponding number of service requests are transmitted to the target task queue of the target storage node according to the current allowed value of the service request throttle.

[0091] It should be noted that the distributed storage system includes storage nodes, each storage node includes multiple storage devices, each storage device corresponds to a task queue, and in the embodiment of the present application, the number of target storage nodes and target task queues is not limited to one.

[0092] Specifically, when multiple service requests are received from the user end, the number of service requests sent to the target task queue of the target storage node is limited according to the current allowed value of the service request throttle.

[0093] Step 104 : In response to the service request in the target task queue, store the service data in the corresponding target storage device.

[0094] Specifically, in response to a service request in the target task queue, the service data indicated by the service request for storage may be stored in the corresponding target storage device.

[0095] Among them, target storage devices include magnetic disks, solid-state disks, and NVMe hard disks.

[0096] On the basis of the above embodiment, in order to ensure the reliability of the current allowed value used by the service request throttle, as an implementable approach, based on the above embodiment, in one embodiment, the current allowed value of the service request throttle is determined based on the current total service delay and current service throughput of the distributed storage system, including:

[0097] Step 1021: determining a target total service delay of the distributed storage system based on the current total service delay and the current service throughput of the distributed storage system;

[0098] Step 1022: Determine the current allowable value of the service request throttle according to the current service total delay and the target service total delay of the distributed storage system.

[0099] Specifically, if Figure 2 As shown, it is a flow chart of an exemplary distributed storage business request processing method provided in an embodiment of the present application, which collects the back-end business throughput and the back-end business total delay in real time, and assigns the obtained data to loop A and loop B. Loop A is used to update the current allowed value of the business request throttle, and loop B is used to update the target business total delay of the distributed storage system, so as to dynamically and adaptively update the double loop according to the load conditions of the distributed storage system, thereby ensuring that the current allowed value adopted by the business request throttle is dynamically matched with the load of the distributed storage system, thereby improving the reliability of the current allowed value adopted by the business request throttle.

[0100] Specifically, in one embodiment, an objective function may be constructed based on the current total service delay and current service throughput of the distributed storage system; and the target total service delay of the distributed storage system may be updated based on the objective function according to a first preset period.

[0101] Specifically, an objective function can be constructed based on the user's performance requirements for the distributed storage system, and then two gradient descent optimization methods can be used to update the target total service delay of the distributed storage system based on the objective function.

[0102] The two-gradient descent optimization method obtains the target parameters in an iterative step and gradually approaches the minimum point of the function by iterating along the negative gradient direction of the objective function. The gradient here refers to the partial derivative of the objective function with respect to the parameter. The two-gradient descent optimization method consists of two loops. One loop optimizes one variable by adjusting the system parameters, and the other loop optimizes the other variable by adjusting the parameters of the first loop.

[0103] Specifically, in one embodiment, a performance measurement index of the distributed storage system may be determined based on the current total service delay and current service throughput of the distributed storage system; and an objective function may be constructed based on a calculation formula of the performance measurement index.

[0104] Specifically, based on the user's performance requirements for the distributed storage system, the current total service delay and current service throughput of the distributed storage system, the performance measurement index of the distributed storage system can be calculated, and then the calculation formula of the performance measurement index can be inferred. Finally, the objective function can be constructed based on the calculation formula of the performance measurement index.

[0105] Among them, the objective function is as follows:

[0106]

[0107] in, Indicates the total delay of the current business. Indicates the total delay of the target business before the update. Indicates the current service throughput. Indicates the throughput adjustment parameter.

[0108] It should be noted that the objective function is a measurement indicator calculated based on the current business total delay and the current business throughput. The square of the current business total delay ( ) represents the excess penalty for delay, which means that the objective function provided by the embodiment of the present application aims to minimize delay. The throughput term ( ) indicates the desire to maximize the throughput of the system, and There are actually two adjustment parameters used to balance the total service delay and service throughput in the optimization objectives.

[0109] Specifically, an embodiment of the present application proposes using a gradient descent algorithm in a distributed storage system to detect the total back-end storage IO delay (total current business delay) and throughput (current business throughput), and adaptively adjusts the calculation formula and derivation process algorithm of the target delay parameter, optimizes the objective function through the gradient descent algorithm, and designs the above objective function in the dynamic load of the distributed storage system so that the system's storage front-end scheduling and storage back-end IO delay and throughput can be effectively balanced.

[0110] Furthermore, in one embodiment, the target total service delay of the distributed storage system may be updated based on the following formula:

[0111]

[0112] in, Indicates the updated total delay of the target business, Indicates the total delay of the target business before the update. represents the preset learning rate, The update step size used to control the total delay of the target business, represents the partial derivative of the objective function with respect to the total delay of the target service, represents the partial derivative of the objective function with respect to the throughput adjustment parameter.

[0113] Specifically, , , the gradient represents the rate of change of the function at a certain point, and is a vector in which each component corresponds to a partial derivative. The gradient descent algorithm uses this gradient to find the minimum point. The embodiment of the present application calculates the gradient of the objective function and then updates the parameters along the negative direction of the gradient. In a distributed storage system, the system load, network conditions, storage medium characteristics, etc. are comprehensively considered, and the gradient of the objective function is calculated by monitoring the actual IO delay (current business total delay) and throughput (current business throughput) of the system, and the target delay parameter (target business total delay) is adaptively adjusted according to the change in the gradient. The adaptive adjustment of the target delay parameter needs to be judged according to the actual situation, and the target delay parameter needs to be dynamically adjusted during the iterative process of gradient descent to adapt to the changes in the dynamic load of the distributed storage system.

[0114] Specifically, if Figure 2As shown, the embodiment of the present application uses a two-variable two-gradient descent optimization method to dynamically and adaptively adjust the target delay parameter under dynamic workloads. The two variables are the storage backend IO delay and IO throughput. The distributed storage client IO request first enters the front-end scheduling device. The embodiment of the present application connects the front-end scheduling device and the back-end storage device through a throttle mechanism to perform forward and backward feedback iteration. Then, the back-end IO delay and throughput distribution are used as input parameters for processing in loop A and loop B, and the operating value of the throttle is adjusted to control and balance the front-end and back-end IO processing, effectively solving the IO congestion problem.

[0115] Specifically, in one embodiment, the allowable value adjustment direction of the business request throttle can be determined based on the size relationship between the current business total delay and the target business total delay of the distributed storage system; the current allowable value of the business request throttle is determined based on the allowable value adjustment direction of the business request throttle and the difference between the current business total delay and the target business total delay.

[0116] Among them, when the size relationship between the current total business delay and the target total business delay of the distributed storage system indicates that the current total business delay is greater than the target total business delay, the allowable value adjustment direction of the business request throttle is determined to be a decreasing direction; when the size relationship between the current total business delay and the target total business delay of the distributed storage system indicates that the current total business delay is not greater than the target total business delay, the allowable value adjustment direction of the business request throttle is determined to be an increasing direction.

[0117] Specifically, an adaptive IO congestion control algorithm can be used to update the current allowed value of the service request throttle. This algorithm records the total delay time allowed for backend storage IO within a certain time interval. At the end of the time interval, if the minimum total delay is higher than the target delay, the allowed value of the queue waiting in the front-end scheduling throttle is reduced. Conversely, if the minimum total delay is lower than the target delay, the allowed value of the queue waiting is increased to accept more IO requests.

[0118] Specifically, if the current total service delay is greater than the target total service delay, it indicates that the distributed storage system has experienced excessive delay. It can be determined that the distributed storage system has experienced service request congestion. Therefore, the permitted value of the service request throttle can be reduced to reduce the number of service requests initiated by the user end to the storage node. If the current total service delay is not greater than the target total service delay, it can be determined that the distributed storage system storage node has relatively few service requests and a small delay. Therefore, the permitted value of the service request throttle can be increased to increase the number of service requests initiated by the user end to the storage node, thereby improving the resource utilization of the storage node.

[0119] Specifically, in one embodiment, the current allowed value of the service request throttle may be determined according to the second preset period, the allowed value adjustment direction of the service request throttle and the difference between the current service total delay and the target service total delay.

[0120] The second preset period is shorter than the first preset period.

[0121] It should be noted that the adaptive IO congestion control algorithm adjusts the allowed value of queue waiting in the front-end scheduling throttle according to the total IO delay of the storage back-end device and the given target delay. The target delay parameter depends on the workload of the distributed storage system. Due to the dynamic business adjustments of the distributed storage system, different target delay values ​​are required under different workloads to perform storage IO operations as expected. Therefore, the embodiment of the present application proposes a dynamic and adaptive adjustment strategy for the target delay parameter under dynamic workloads, which can meet the QoS of distributed storage. Different from the existing related technologies, the use of advance planning, deployment, and setting of static threshold parameters will not solve the current IO congestion management problem of dynamic workloads.

[0122] Specifically, the adaptive IO congestion control algorithm uses the following logic to update the current allowed value of the service request throttle:

[0123] When the time interval ends

[0124] Begin

[0125] If the total delay of the current business is greater than the total delay of the target business, then

[0126] Reduce the allowed value; / / Current service total delay – target service total delay

[0127] else

[0128] Increase the allowed value;

[0129] End

[0130] reset the total delay of the current business

[0131] sleep for interval

[0132] end

[0133] The time interval is the second preset period. Figure 2In this process, Loop A monitors the total storage backend I / O latency in real time and controls the number of requests allowed to be queued for processing based on the target latency parameter. Within short time intervals, Loop A monitors and manages the total backend I / O latency, enabling rapid response to changes in storage I / O latency and workload. Loop B monitors the storage backend I / O throughput at longer intervals. The difference between using longer intervals is that backend I / O throughput is significantly affected by factors such as internal storage data synchronization and reconstruction. Therefore, sampling statistics at lower frequencies over longer intervals is the best approach to ensure accuracy. During these intervals, the target latency parameter in Loop A is adjusted to balance the storage backend I / O throughput. Based on changes in the target latency, as the target latency decreases, Loop A limits the number of I / O requests allowed to the storage backend devices, resulting in lower I / O latency and throughput for the storage backend devices. Loop B updates and optimizes the target latency parameter based on the I / O throughput trade-off to evenly manage and distribute I / O requests between the storage backend and the storage frontend. When I / O throughput reaches maximum capacity, Loop B lowers the target latency parameter to prevent a sudden increase in the buffer. As I / O throughput decreases, Loop B increases the target latency parameter to prevent the storage backend devices from idling and entering a dormant state due to a lack of I / O requests to process.

[0134] In a distributed storage system, when an OSD (object storage) process such as a storage device disk fails in a fault scenario, first, the OSD no longer carries data due to the fault, and the data needs to be balanced to other normal OSD disk devices to achieve data consistency and integrity of the distributed storage. Then, the failed OSD is repaired and restarted, and the data will be restored and backfilled to the OSD, occupying the storage device backend IO throughput. While data recovery is in progress, a high-load request for distributed storage client business is initiated at the same time. However, since the client's high workload request is dynamic and bursty, the storage backend device OSD processing queue has been filled with data recovery IO at this time. When a large number of high-load requests flow into the backend storage device, it is bound to affect the overall IO performance of the storage, resulting in high-latency response requests. In this case, through the continuous iterative parameter adjustment of the dual loops A and B proposed in the embodiment of the present application, the target delay parameters are adjusted, and the front-end load request influx is dynamically controlled to adaptively meet the storage efficient IO processing in the current scenario.

[0135] Based on the above embodiment, as an implementable manner, based on the above embodiment, in one embodiment, when multiple service requests are received from a user terminal, a corresponding number of service requests are transmitted to a target task queue of a target storage node according to a current allowed value of a service request throttle, including:

[0136] Step 1031: Based on the front-end scheduling device, multiple service requests sent by the user end are received, and the received service requests are cached in the front-end queue;

[0137] Step 1032: According to the current allowed value of the service request throttle, a corresponding number of service requests in the front-end queue are transmitted to the target task queue of the target storage node.

[0138] Among them, the distributed storage system in the embodiment of the present application adopts a distributed and decentralized design architecture, and the front-end scheduling device can be any storage node in the distributed storage.

[0139] Specifically, if Figure 2 As shown, the service requests sent by the user end are first cached in the front-end queue. The front-end scheduling device schedules the service requests in the front-end queue according to the current allowed value of the service request throttle to transmit the corresponding number of service requests in the front-end queue to the target task queue of the target storage node (back-end of the storage device).

[0140] Specifically, in one embodiment, the target length of the task queue of each storage node in the distributed storage system can be determined based on the current allowed value of the service request throttle; when multiple service requests sent by the user end are received, the corresponding number of service requests are scheduled from the front-end queue based on the target length of the task queue and the current capacity occupancy of the target task queue of the target storage node, and the service requests are transmitted to the target task queue of the target storage node.

[0141] The target length of the task queue specifically refers to the number of service requests allowed to wait in the task queue.

[0142] Specifically, the number of business requests allowed to wait in each task queue of each storage node in the distributed storage system can be determined based on the current allowed value of the business request throttle. Then, based on the current capacity occupancy of the target task queue, it can be determined how many business requests are currently waiting for response in the target task queue. Finally, based on the difference between the two, it can be determined how many task requests can be added to the target task queue, and then the corresponding number of business requests can be dispatched from the front-end queue to the target task queue.

[0143] Specifically, to ensure load balancing of the distributed storage system, a preset load balancing algorithm can be used to determine the target storage node and target task queue for each task request. The specific load balancing method is not limited in the embodiments of this application.

[0144] Specifically, in one embodiment, in response to a business request in a target task queue, the business request can be split into a data request and a metadata request, and the metadata request can be cached in the metadata queue; based on the data request and the metadata request, the data to be stored and the metadata can be associated and stored in the corresponding target storage device.

[0145] Among them, business data includes data to be stored and metadata.

[0146] Specifically, if Figure 3 The figure shows the network architecture of the distributed storage system provided by an embodiment of the present application. The distributed storage backend is composed of multiple storage device modules, each of which maintains its own processing queue. The distributed storage backend stores data IO. First, the distributed storage system client IO request is sent to the storage device through the IO congestion control algorithm. The operating system's libaio library (target library) performs asynchronous IO operations. The operating system splits the service request into data requests and metadata requests, and places the data request in a kernel-managed asynchronous IO request queue, caching the metadata request in the metadata queue. The kernel then processes these IO requests asynchronously, including copying data from user memory to the kernel buffer, triggering actual hardware IO operations, and handling IO completion events. The distributed storage backend stores metadata IO. First, the metadata needs to be queued in the key-value queue (metadata queue) and then stored in the RocksDB database (metadatabase). Finally, the data and metadata are stored on the disk, solid-state drive, and NVMe hard drive (NVMESSD).

[0147] The distributed storage service request processing method provided by the embodiment of the present application obtains the current total service delay and current service throughput of the distributed storage system; determines the current allowed value of the service request throttle according to the current total service delay and current service throughput of the distributed storage system; when receiving multiple service requests sent by the user end, transmits the corresponding number of service requests to the target task queue of the target storage node according to the current allowed value of the service request throttle; and stores the service data in the corresponding target storage device in response to the service requests in the target task queue. The method provided by the above scheme limits the number of service requests sent to the target storage node according to the current allowed value of the service request throttle, ensures the system service delay and service throughput, avoids congestion of service requests, and lays the foundation for improving the storage service quality of the distributed storage system. In addition, by dynamically adjusting the target delay according to changes in workload, the trade-off between request processing is always maintained within a reasonable range, avoiding serious queuing delays. The performance of the distributed storage system is optimized, serious queuing delays in the storage backend are avoided, and IO congestion is controlled by dynamically adjusting the target delay. Under the same hardware resource conditions, in the case of dynamic high-concurrency requests, the method provided in the embodiment of the present application can quickly respond to dynamic changes in distributed storage client requests, such as sudden growth in requests, thereby preventing excessive accumulation of the distributed storage backend buffer, thereby providing better storage IO transmission performance and stability, improving the competitiveness and maintenance cost of the distributed storage system, and the method is portable and universal on different hardware devices.

[0148] An embodiment of the present application provides a distributed storage service request processing device, which is used to execute the distributed storage service request processing method provided in the above embodiment.

[0149] like Figure 4 FIG. 4 is a schematic diagram of a distributed storage service request processing apparatus according to an embodiment of the present invention. The distributed storage service request processing apparatus 40 comprises: an acquisition module 401 , a determination module 402 , a processing module 403 and a response module 404 .

[0150] Among them, the acquisition module is used to obtain the current total business delay and current business throughput of the distributed storage system; the determination module is used to determine the current allowed value of the business request throttle based on the current total business delay and current business throughput of the distributed storage system; the processing module is used to, when receiving multiple business requests sent by the user end, transmit the corresponding number of business requests to the target task queue of the target storage node according to the current allowed value of the business request throttle; the response module is used to respond to the business requests in the target task queue and store the business data in the corresponding target storage device.

[0151] Regarding the distributed storage service request processing device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method and will not be elaborated on here.

[0152] The distributed storage service request processing device provided in the embodiment of the present application is used to execute the distributed storage service request processing method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0153] An embodiment of the present application provides a distributed storage system for executing the distributed storage service request processing method provided in the above embodiment.

[0154] like Figure 5 As shown, it is a structural diagram of the distributed storage system provided in an embodiment of the present application, the system includes multiple storage nodes, each storage node includes multiple storage devices; any storage node processes a business request based on the distributed storage business request processing method provided in the above embodiment to store the business data of the business request in the corresponding target storage device.

[0155] For example, Figure 6 The figure shows a schematic diagram of the structure of an exemplary distributed storage system provided by an embodiment of the present application. The design concept of the distributed storage system is based on scalability and fault tolerance, eliminating performance bottlenecks such as single-point metadata. It adopts a distributed, decentralized design architecture, slicing data and distributing it across multiple nodes in the form of objects, ensuring load and capacity balance among each node. Data access and management are achieved through network connections, while providing a unified storage interface. Due to its decentralized design architecture, the distributed storage system is highly scalable; the failure of a single storage server node will not cause the entire distributed storage cluster to become unavailable. It supports dynamic expansion, adding or removing storage server nodes, and can automatically relocate data objects in the cluster to balance storage space utilization across storage servers. The distributed storage system architecture includes multiple components, such as an object gateway service, a block device service, a file system service, a unified, self-controlled, and scalable distributed storage consistency management system, a storage pool, a controllable, scalable, distributed data balancing algorithm, and a metadata and monitoring service cluster. In the storage pool, client data is stored in slices, fixed by default to 4MB objects. Objects are grouped, and the group ID is the modulo of Hash(object x) and the number of groups. Finally, the data is stored in the underlying device. The distributed storage system includes multiple storage pools, and each storage pool includes multiple storage nodes.

[0156] The distributed storage system architecture includes the following key components:

[0157] 1) Object gateway service, block device service, and file system service: Provide a unified storage interface to provide access interface services for distributed storage clients.

[0158] 2) A unified, self-controlled, and scalable distributed storage consistency management system: This component is the core component of a distributed storage cluster and provides distributed object-based storage services. In this component, data is stored as objects, each with a unique identifier and associated data. This component is responsible for distributing objects across the nodes of the storage cluster and provides data replication, recovery, and load balancing to ensure data reliability and high-performance access. This component typically implements front-end scheduling policies, using static thresholds to set bandwidth and IOPs performance limits for different service interfaces.

[0159] 3) Storage Pool: A distributed storage cluster consists of multiple storage nodes, each of which can contain multiple hard drives or storage devices. Storage pools divide storage resources into different pools to meet different storage needs.

[0160] 4) Controllable, scalable, distributed data balancing placement algorithm: This component evenly distributes data objects across the nodes of the storage cluster to avoid data hotspots and improve system performance.

[0161] 5) Metadata and Monitoring Service Cluster: The metadata cluster manages file system metadata, including file and directory attributes and locations. The monitoring service cluster monitors cluster status and configuration. Its technical principles include maintaining cluster status, configuration, and health information to ensure cluster consistency and availability.

[0162] Among them, the storage device of each node, namely OSD, will first receive the request for data and metadata, then enter the processing queue of each stage of the storage device, process it in the queue, and finally flush the data and metadata to the storage device.

[0163] The distributed storage system ensures the performance isolation of different business applications by adopting different front-end scheduling algorithms. The front-end scheduling module can ensure high-quality service QoS requirements. These requirements are usually defined in a unified, self-controlled, and scalable distributed storage consistency management system. The storage back-end module is responsible for effectively storing and retrieving data on the underlying storage devices of the distributed storage system. In the distributed storage nodes, the scheduling algorithm rearranges the requests in the queue according to the strategy, and assigns higher priority requests to the back-end storage device processing queue. However, if the back-end storage device has enough buffers, the scheduling algorithm will dispatch more requests, which will cause IO congestion in the buffer. The distributed storage system provided in the embodiment of the present application adopts the method provided in the above embodiment to process business requests to avoid causing IO congestion in the buffer.

[0164] Regarding the distributed storage system in this embodiment, its specific implementation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0165] The distributed storage system provided in the embodiment of the present application is used to execute the distributed storage service request processing method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0166] An embodiment of the present application provides an electronic device for executing the distributed storage service request processing method provided in the above embodiment.

[0167] like Figure 7 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. The electronic device 70 includes at least one processor 71 and a memory 72 .

[0168] The memory stores computer-executable instructions; at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the distributed storage service request processing method provided in the above embodiment.

[0169] The electronic device provided in the embodiment of the present application is used to execute the distributed storage service request processing method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0170] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, a distributed storage service request processing method as provided in any of the above embodiments is implemented.

[0171] The storage medium containing computer-executable instructions provided in the embodiment of the present application can be used to store the computer-executable instructions of the distributed storage service request processing method provided in the aforementioned embodiment. Its implementation method and principle are the same and will not be repeated here.

[0172] An embodiment of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the distributed storage service request processing method provided in the aforementioned embodiment.

[0173] The embodiments of the present application provide a computer program product that can be used to execute computer instructions of the distributed storage service request processing method provided in the aforementioned embodiments. The implementation method and principle are the same and will not be repeated here.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0175] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0176] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0177] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0178] Part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes but is not limited to a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0179] Those skilled in the art will clearly understand that for the sake of convenience and brevity, the division of the above-mentioned functional modules is only used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A distributed storage service request processing method, characterized in that: include: Obtain the current total service delay and current service throughput of the distributed storage system; Determine the current allowable value of the service request throttle according to the current service total delay and the current service throughput of the distributed storage system; the allowable value of the service request throttle is used to limit the number of service requests sent from the storage front end to the back end; When receiving multiple service requests sent by the user end, transmitting a corresponding number of service requests to a target task queue of a target storage node according to a current allowed value of the service request throttle valve; In response to the service request in the target task queue, storing the service data in the corresponding target storage device; The determining, according to the current service total delay and the current service throughput of the distributed storage system, the current allowed value of the service request throttle comprises: Determining a target total service delay of the distributed storage system according to a current total service delay and a current service throughput of the distributed storage system; Determine a current allowed value of a service request throttle according to a current service total delay of the distributed storage system and the target service total delay; The determining, according to the current total service delay and the current service throughput of the distributed storage system, a target total service delay of the distributed storage system comprises: Constructing an objective function according to the current service total delay and the current service throughput of the distributed storage system; According to the objective function, updating the target total service delay of the distributed storage system according to a first preset period; The updating of the target service total delay of the distributed storage system according to the objective function includes: Based on the following formula, the target total service delay of the distributed storage system is updated: in, Indicates the updated total delay of the target service. Indicates the total target service delay before updating. represents the preset learning rate, represents the partial derivative of the objective function with respect to the total delay of the target service, represents the partial derivative of the objective function with respect to the throughput adjustment parameter.

2. The method according to claim 1, characterized in that The constructing of an objective function according to the current service total delay and the current service throughput of the distributed storage system comprises: Determining a performance measurement indicator of the distributed storage system according to a current service total delay and a current service throughput of the distributed storage system; According to the calculation formula of the performance measurement indicator, an objective function is constructed.

3. The method according to claim 2, characterized in that The step of constructing an objective function according to a calculation formula of the performance measurement indicator comprises: According to the calculation formula of the performance measurement index, the following objective function is constructed: in, Indicates the total delay of the current business. Indicates the total target service delay before updating. Indicates the current service throughput. Indicates the throughput adjustment parameter.

4. The method according to claim 1, characterized in that The determining, according to the current total service delay of the distributed storage system and the target total service delay, a current allowable value of the service request throttle valve comprises: Determining the adjustment direction of the allowable value of the service request throttle according to the size relationship between the current service total delay of the distributed storage system and the target service total delay; The current allowed value of the service request throttle is determined according to the allowed value adjustment direction of the service request throttle and the difference between the current service total delay and the target service total delay.

5. The method according to claim 3, characterized in that: The determining, according to the size relationship between the current total service delay of the distributed storage system and the target total service delay, the adjustment direction of the allowable value of the service request throttle valve comprises: In a case where the magnitude relationship between the current service total delay and the target service total delay of the distributed storage system indicates that the current service total delay is greater than the target service total delay, determining that the adjustment direction of the allowable value of the service request throttle is a decreasing direction; When the size relationship between the current total service delay and the target total service delay of the distributed storage system indicates that the current total service delay is not greater than the target total service delay, the adjustment direction of the allowable value of the service request throttle is determined to be an increasing direction.

6. The method according to claim 3, characterized in that The determining the current allowed value of the service request throttle according to the allowed value adjustment direction of the service request throttle and the difference between the current service total delay and the target service total delay includes: According to a second preset period, determining a current allowed value of the service request throttle according to an allowed value adjustment direction of the service request throttle and a difference between the current service total delay and the target service total delay; The second preset period is smaller than the first preset period.

7. The method according to claim 1, characterized in that When receiving multiple service requests sent by the user end, transmitting a corresponding number of service requests to the target task queue of the target storage node according to the current allowed value of the service request throttle, including: Based on the front-end scheduling device, multiple service requests sent by the user end are received, and the received service requests are cached in the front-end queue; According to the current allowed value of the service request throttle, a corresponding number of service requests in the front-end queue are transmitted to the target task queue of the target storage node.

8. The method according to claim 7, characterized in that The transmitting, according to the current allowed value of the service request throttle, a corresponding number of service requests in the front-end queue to the target task queue of the target storage node comprises: Determining a target length of a task queue of each storage node in the distributed storage system according to a current allowed value of the service request throttle; When multiple service requests are received from the user end, a corresponding number of service requests are scheduled from the front-end queue according to the target length of the task queue and the current capacity occupancy of the target task queue of the target storage node, and the service requests are transmitted to the target task queue of the target storage node.

9. The method according to claim 1, characterized in that: The step of storing the business data in a corresponding target storage device in response to the business request in the target task queue includes: In response to a business request in the target task queue, split the business request into a data request and a metadata request, and cache the metadata request in a metadata queue; According to the data request and the metadata request, the data to be stored and the metadata are associated and stored in a corresponding target storage device; The business data includes the data to be stored and metadata.

10. A distributed storage service request processing device, characterized in that: include: An acquisition module is used to obtain the current total service delay and current service throughput of the distributed storage system; A determination module, used to determine a current allowable value of a service request throttle according to a current total service delay and a current service throughput of the distributed storage system; the allowable value of the service request throttle is used to limit the number of service requests sent by the storage front end to the back end; A processing module, configured to, when receiving a plurality of service requests sent by a user terminal, transmit a corresponding number of service requests to a target task queue of a target storage node according to a current allowed value of the service request throttle valve; A response module, used for responding to the service request in the target task queue and storing the service data in the corresponding target storage device; The determination module is specifically used for: Determining a target total service delay of the distributed storage system according to a current total service delay and a current service throughput of the distributed storage system; Determine a current allowed value of a service request throttle according to a current service total delay of the distributed storage system and the target service total delay; The determination module is specifically used for: Constructing an objective function according to the current service total delay and the current service throughput of the distributed storage system; According to the objective function, updating the target total service delay of the distributed storage system according to a first preset period; The determination module is specifically used for: Based on the following formula, the target total service delay of the distributed storage system is updated: in, Indicates the updated total delay of the target service. Indicates the total target service delay before updating. represents the preset learning rate, represents the partial derivative of the objective function with respect to the total delay of the target service, represents the partial derivative of the objective function with respect to the throughput adjustment parameter.

11. A distributed storage system, characterized in that: include: A plurality of storage nodes, each storage node including a plurality of storage devices; Any of the storage nodes processes the service request based on the method according to any one of claims 1 to 9 to store the service data of the service request in the corresponding target storage device.

12. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 9 is implemented.

14. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 9.

Images