Request processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a request processing method and device, equipment and a storage medium. The request processing method comprises the steps that a data operation request of a storage cluster is obtained, the data operation request is distributed to a target process node, a data operation lock matched with a current operation type is requested through the target process node, and the current operation type is the operation type to which the data operation request belongs; the current environment comprises an operation lock state corresponding to the current operation type and operation lock states of other operation types, and the data operation lock is allocated for processing the data operation request in response to the fact that the current environment allows allocation of the corresponding data operation lock. According to the method, the technical problems of low request processing efficiency and unbalanced message distribution are solved, and the technical effects of improving the request processing efficiency, improving the unbalanced message distribution and further improving the overall performance of the cluster are achieved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a request processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] In distributed storage systems, the efficient operation of the monitoring process is a core element in ensuring system stability and performance. With the explosive growth of data volume and the increasing complexity of application scenarios, systems face the severe challenge of maintaining data consistency and reliability under high-concurrency read / write operations, while also needing to balance response speed and resource utilization. Current monitoring processes typically employ a single-process mechanism for handling read / write operations, which is ill-suited to the continuously expanding cluster size. As the number of cluster nodes increases, the amount of process state information that the monitoring process needs to maintain rises dramatically, leading to increasingly prominent performance bottlenecks. The expansion of the cluster size further exacerbates the frequency of service state changes, placing higher demands on the message processing capabilities of the monitoring process. Especially when handling read operations, clients can randomly select any monitoring process node, but the client is completely unaware of the real-time load pressure of the monitoring process node. This blind selection mechanism easily leads to message blocking, resulting in low request processing efficiency, unbalanced message distribution, and ultimately affecting the overall performance of the cluster. Summary of the Invention

[0003] This application provides a request processing method, apparatus, device, and storage medium to at least solve the technical problems of low request processing efficiency and unbalanced message distribution.

[0004] This application provides a request processing method, which includes: obtaining a data operation request from a storage cluster; allocating the data operation request to a target process node; causing the target process node to request a data operation lock matching the current operation type, wherein the current operation type is the operation type to which the data operation request belongs; in response to the request to obtain the data operation lock, verifying whether the current environment allows the allocation of the data operation lock; wherein the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types; and in response to the current environment allowing the allocation of the data operation lock, allocating the data operation lock in response to the data operation request.

[0005] This application also provides a request processing apparatus, which includes:

[0006] The first acquisition module is used to acquire data operation requests from the storage cluster.

[0007] The request module is used to divide the data to be migrated in the current storage unit into multiple hot zones based on resource metrics.

[0008] The second acquisition module is used to verify whether the current environment allows the allocation of a data operation lock in response to a request to acquire a data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types.

[0009] The allocation module is used to allocate data operation locks in response to data operation requests, provided that the current environment allows the allocation of such locks.

[0010] This application also provides an electronic device, which includes:

[0011] Memory, used to store computer programs;

[0012] A processor is used to implement any of the above request processing methods when executing a computer program.

[0013] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described request processing methods.

[0014] This application dynamically optimizes request processing in a distributed environment by acquiring data operation requests from the storage cluster and allocating them to target process nodes. The target node requests a data operation lock matching the data operation request type. Lock allocation verification is then performed based on the current lock state and the states of other locks within the current environment. Finally, upon successful verification, a data operation lock is allocated to respond to the request, achieving dynamic optimization of request processing. This dynamic routing mechanism adapts to the characteristics of different data operation request types, ensuring consistency in write operations while improving the flexibility of read operations. The data operation lock state verification process comprehensively analyzes the global state of multiple types of locks, ensuring resource access security while avoiding read-write conflicts. Finally, the lock allocation mechanism significantly reduces waiting latency in concurrent scenarios through fine-grained control of resource access permissions. The entire process forms a closed-loop optimization chain, effectively improving the throughput and response speed of the storage cluster for high-concurrency requests while maintaining data consistency.

[0015] Therefore, this method can solve the technical problems of low request processing efficiency and message distribution imbalance, thereby improving the technical effects of improving request processing efficiency, mitigating message distribution imbalance, and ultimately enhancing the overall performance of the cluster. Attached Figure Description

[0016] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 An application environment diagram of a request processing method provided in an embodiment of this application;

[0018] Figure 2 A flowchart illustrating a request processing method provided in an embodiment of this application;

[0019] Figure 3 A flowchart illustrating another request processing method provided in an embodiment of this application;

[0020] Figure 4 A schematic diagram of a data operation lock provided in an embodiment of this application;

[0021] Figure 5 A schematic diagram of another data operation lock provided in an embodiment of this application;

[0022] Figure 6 A flowchart illustrating another request processing method provided in an embodiment of this application;

[0023] Figure 7 This is a schematic diagram of the structure of a request processing device provided in an embodiment of this application;

[0024] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0026] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0027] It should be noted that the terms "S1," "S2," etc., are used only for descriptive purposes and do not specifically refer to the order or sequence, nor are they intended to limit this application. They are merely for the convenience of describing the method of this application and should not be construed as indicating the sequential order of the steps. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0028] To address the technical issues of low request processing efficiency and unbalanced message distribution, this application achieves the following technical effects: acquiring data operation requests from the storage cluster; allocating these requests to target process nodes; causing the target process nodes to request a data operation lock matching the current operation type (the operation type to which the data operation request belongs); in response to the data operation lock request, verifying whether the current environment allows the allocation of the data operation lock; wherein the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types; and in response to the current environment allowing the allocation of the data operation lock, allocating the data operation lock to respond to the data operation request. This approach improves request processing efficiency, mitigates message distribution imbalance, and ultimately enhances the overall performance of the cluster.

[0029] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. The request processing method provided in this application can be applied to, for example... Figure 1 , Figure 1 This diagram illustrates an application environment for a request processing method provided in this application embodiment. Terminal 12 communicates with server 14 via a network. Terminal 12 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 14 can be a standalone server or a server cluster consisting of multiple servers.

[0030] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] The embodiments of this application provide a request processing method, and the method is described in detail below in conjunction with the execution flow of the request processing method.

[0032] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a request processing method provided in an embodiment of this application.

[0033] S101: Obtain data operation request from the storage cluster.

[0034] In this embodiment, the storage cluster refers to a distributed system composed of multiple process nodes, responsible for managing data read / write and cluster status. The processes can be MON (Monitors) processes, i.e., parallel service monitoring processes, used to manage the cluster status of the storage cluster. They are collections of physical or virtual nodes that can collaboratively process data operation requests.

[0035] Data operation requests refer to read and write operation instructions initiated by the client, such as reading files or writing configuration files. Data operation types are divided into read requests, which can access shared resources, and write requests, which require exclusive access to modify resources.

[0036] Specifically, the data operation instructions that the client initiates read or write requests are received through the distributed system interface, such as through API (Application Programming Interface) calls or network protocol parsing, to obtain the user's data operation requests to the storage cluster and support high-concurrency request access.

[0037] S102: Assign the data operation request to the target process node; so that the target process node requests a data operation lock that matches the current operation type, where the current operation type is the operation type to which the data operation request belongs.

[0038] In this embodiment, the target process node refers to the MON process instance that executes the data operation request, which is the service node in the distributed system that actually processes the data operation request. A data operation lock is a synchronization mechanism that controls resource access permissions; based on the type of data operation request, it can be divided into read locks and write locks. The current operation type refers to the operation type corresponding to the data operation request. For example, if the data operation request type is a read request, then the data operation lock corresponding to processing the read request is a read lock; if the data operation request type is a write request, then the data operation lock corresponding to processing the write request is a write lock.

[0039] Specifically, upon receiving a data operation request, it needs to be allocated to the target process node for processing. The target process node receives the allocated operation request, takes the received request type as the current operation type, and needs to acquire a data operation lock matching the current operation type. If the current operation type is a read request, a read lock needs to be acquired; if the current operation type is a write request, a write lock needs to be acquired.

[0040] S103: In response to the request to acquire a data operation lock, verify whether the current environment allows the allocation of a data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types.

[0041] In this embodiment, the current environment refers to the real-time state set of all data operation locks in the storage cluster, which is used for dynamic resource occupancy snapshots to determine whether a new lock is available for allocation.

[0042] Specifically, upon receiving a request to acquire a data operation lock, it is necessary to further determine whether a data operation lock matching the request can be granted. This request may include acquiring a read lock or a write lock to process the corresponding read or write request. The state of the data operation lock determines whether a lock can be allocated to the appropriate request. If the current operation type is a read request, the read lock's state must be acquired; if the current operation type is a write request, the write lock's state must be acquired to allocate the corresponding lock.

[0043] S104: In response to the current environment allowing the allocation of a data operation lock, allocate a data operation lock in response to a data operation request.

[0044] In this embodiment, if it is verified that the current environment allows the allocation of data operation locks, then the data operation lock matching the current operation type is allocated to the target process node, and the target process node processes the corresponding data operation request based on the allocated data operation lock.

[0045] In this embodiment, data operation requests from the storage cluster are acquired and dynamically allocated to target process nodes. The target node requests a data operation lock matching the data operation request type. Lock allocation verification is then performed based on the current lock state and the states of other locks within the current environment. Finally, after successful verification, a data operation lock is allocated to respond to the request, achieving dynamic optimization of request processing in a distributed environment. This dynamic routing mechanism adapts to the characteristics of different data operation request types, ensuring consistency in write operations while improving the flexibility of read operations. The data operation lock state verification process comprehensively analyzes the global state of multiple types of locks, ensuring resource access security while avoiding read-write conflicts. Finally, the lock allocation mechanism significantly reduces waiting latency in concurrent scenarios through fine-grained control of resource access permissions. The entire process forms a closed-loop optimization chain, effectively improving the throughput and response speed of the storage cluster for high-concurrency requests while maintaining data consistency.

[0046] In one embodiment, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating another request processing method provided in an embodiment of this application. The request processing method further includes:

[0047] S201: Determine the request type of the data operation request.

[0048] In this embodiment, it is necessary to determine the request type of the data operation request, which includes read requests and write requests, and to allocate the data operation request to different process nodes according to the request type.

[0049] Specifically, if the request type is a read request, then step S202 is executed to allocate a target process node for the write request; if the request type is a write request, then step S203 is executed to allocate a target process node for the read request based on the process node's load score. This ensures that data operation requests are allocated to suitable process nodes by determining the request type.

[0050] S202: In response to a write request, the master process node is selected as the target process node to route the write request to the master process node.

[0051] In this embodiment, the process nodes include a master process node and slave process nodes. The master process node is the only monitoring process node in the storage cluster that handles write requests, while the slave process nodes are all other process nodes besides the master process node, and the slave process nodes can only handle read requests.

[0052] Specifically, when a data operation request is identified as a write request, the write request is routed to the master process node, which is the target process node for the current write request. Since only the master process node can handle write requests, when a data operation request is a write request, the address of the master process node in the storage cluster is queried. Using a network protocol such as TCP (Transmission Control Protocol), the write request is sent to the master process node based on its address to ensure the sequential processing of write requests.

[0053] S203: In response to a read request, select the process node that executes the read request as the target process node based on the process node's load score.

[0054] In this embodiment, when a data operation request is identified as a read request, a suitable process node for handling the read request needs to be selected as the target process node based on the load score of each process node in the storage cluster. However, in a distributed storage system, when a client connection process node of a monitoring process sends read / write operation requests, a random node selection mechanism is used by default. In this embodiment, for read requests, a process node needs to be selected as the target process node based on its load score. The process node selected based on the load score is used to handle the read request; this process node can be either the master process node (the same process node as the write request) or a slave process node. Sending the read request to the appropriate process node based on the load score ensures that the read request is sent to the monitoring process node with lower load pressure, balancing the load pressure of each monitoring process and avoiding resource waste.

[0055] In the above embodiments, by distinguishing the request types of data operation requests and dynamically allocating target process nodes based on these types, write requests are fixedly routed to the main process node to ensure the order of operations and data consistency. Meanwhile, read requests are dynamically selected for execution based on the load scores of each process node. This mechanism can avoid conflicts and state chaos caused by concurrent writes from multiple nodes, thereby ensuring the reliability of core state changes in the cluster. Read operations, through dynamic routing driven by load scores, respond in real time to changes in node pressure, prioritizing requests to low-load nodes, effectively distributing read pressure and preventing single-point overload risks, thereby optimizing the resource utilization, concurrent throughput, and overall stability of the storage cluster.

[0056] S204: In response to a request to acquire a data operation lock, verify whether the current environment allows the allocation of a data operation lock.

[0057] In this embodiment, data operation requests are allocated to the corresponding target process nodes based on their request types, and the target process nodes request to acquire the corresponding data operation locks according to the received data operation request types.

[0058] Data operation locks include shared locks and exclusive locks. Shared locks record the number of times a read lock is acquired, representing the number of read requests currently being processed. Shared locks allow multiple read requests to execute concurrently, improving the throughput of read-intensive tasks. Typically, request processing uses a single lock, such as... Figure 4 , Figure 4 This is a schematic diagram of a data operation lock provided in an embodiment of this application. When a data operation request is being processed, the data operation lock is held, and other data operation requests must wait until the previous data operation request is completed and the data operation lock is released before the released data operation lock can be held to process the data operation request. This single lock uses serial processing logic, resulting in low efficiency in processing data operation requests.

[0059] In this embodiment, as Figure 5 , Figure 5 This is a schematic diagram of another data operation lock provided in an embodiment of this application. As long as a shared lock is in use, the allocation of an exclusive lock is prohibited. Read requests do not block other read requests; that is, even with a shared lock in use, a read lock can still be allocated to process read requests, achieving concurrent processing of read requests. An exclusive lock is used to mark the acquisition status of a write lock. An exclusive lock indicates that the resource is occupied, ensuring that write requests exclusively occupy the resource and avoiding data inconsistency.

[0060] and

[0061] Specifically, if the data operation request type is a write request, a write lock needs to be acquired to process the write request; if the data operation request type is a read request, a read lock needs to be acquired to process the read request. When acquiring the corresponding data operation lock, the status of the data operation lock in the storage cluster needs to be verified, and the status of the data operation lock determines whether a read lock or a write lock can be allocated. If the lock to be acquired is a read lock, step S205 is executed; if the lock to be acquired is a write lock, step S206 is executed.

[0062] S205: When the data operation lock is a read lock, the number of times the read lock is acquired is accumulated using the shared lock; in response to the completion of the read request, the read lock is released and the acquisition count is released.

[0063] In this embodiment, when the monitoring process starts, the data operation lock is initialized, the shared lock counter used to record the number of read lock acquisitions is set to 0, and the status flag of the exclusive lock is set to the unacquired state.

[0064] When a read request arrives at the monitoring process, it first attempts to acquire the read lock. An atomic operation is used to check and increment the shared lock counter. If the write lock is not currently held, the read request is allowed, and the shared lock counter is incremented. Multiple read threads, after acquiring the read lock, can simultaneously perform read operations on shared resources, such as storing cluster state information and configuration parameters. After a read request completes its read operation, an atomic operation decrements the shared lock counter.

[0065] S206: When the data operation lock is a write lock, determine whether the number of read lock acquisitions is zero and whether the exclusive lock is in an unacquired state.

[0066] In this embodiment, when the data operation lock is a write lock, it is necessary to determine whether the current counter is 0 to determine whether a read request is currently being processed. If the number of read lock acquisitions is zero and the exclusive lock is not acquired, then step S207 is executed; if the number of read lock acquisitions is not zero or the exclusive lock is not not acquired, then step S208 is executed.

[0067] S207: In response to the read lock acquisition count being zero and the exclusive lock being unacquired, acquire the write lock and enter the batch queue.

[0068] In this embodiment, when a write request arrives, it attempts to acquire the write lock. If the current exclusive lock is in an unacquired state, it means the write lock has not been acquired, and the read lock acquisition count is 0, indicating that all read requests have released their read locks. At this point, the write request can attempt to acquire the write lock. Once the write request successfully acquires the write lock, it enters an exclusive execution state. At this time, other read and write requests cannot acquire the corresponding lock. The write request enters the batch processing queue to perform modification operations on the shared resource, such as updating cluster status information or modifying configuration parameters. After completing the write operation, the write request releases the write lock. At this point, waiting read and write requests are notified, allowing them to attempt to acquire the corresponding lock.

[0069] S208: In response to the read lock acquisition count being non-zero and / or the exclusive lock being in an acquired state, the write request enters a waiting state.

[0070] In this embodiment, if the current exclusive lock is in the acquire state, it means that the write lock has been acquired; or if the number of times the read lock has been acquired is not 0, it means that a read request is being executed, and the write request enters the waiting state until the counter is 0, and then tries to acquire the write lock again.

[0071] In the above embodiments, a collaborative operation mechanism between shared locks and exclusive locks is defined. When processing read requests, the shared lock allows for safe incrementing of its counter when there is no write lock interference, enabling multiple read operations to execute in parallel without blocking each other. When processing write requests, the dual conditions of the read lock counter reaching zero and the exclusive lock being idle are verified to ensure that write operations acquire exclusive access to resources only in a safe environment. This decision-making mechanism based on real-time lock state feedback enables the cluster to perform data synchronization, state maintenance, and request processing more efficiently when handling high-concurrency read and write operations, thereby improving the overall storage and read / write performance of the cluster and meeting the high performance and reliability requirements of large-scale distributed storage scenarios.

[0072] In this embodiment, a preset timeout threshold is also included to determine whether the waiting time of a write request entering the waiting state exceeds the preset timeout threshold. If the waiting time exceeds the preset timeout threshold, the allocation of new read locks is suspended, and the backlog of write requests is processed first. This waiting time determination mechanism can prevent write requests from waiting indefinitely and ensure the real-time performance of critical operations.

[0073] S209: Dynamically adjust the actual window duration.

[0074] In this embodiment, the actual window duration refers to the aggregation time of batch processing requests. The process involves: obtaining the total number of process nodes in the storage cluster; evaluating the storage cluster size that matches the total number of nodes; obtaining a first factor that matches the storage cluster size; monitoring the number of process changes with state changes; evaluating the change scale that matches the number of process changes; obtaining a second factor that matches the change scale; summing the first and second factors to obtain a comprehensive factor; matching a preset time window duration with a time window duration associated with the storage cluster size as the target time window duration; using the ratio of the target time window duration to the comprehensive factor as the actual window duration; obtaining data operation requests within the actual window duration and adding the data operation requests to the batch processing queue.

[0075] Specifically, the monitoring process, acting as a server, can distinguish whether there are frequent changes in the storage cluster size and cluster process status through scenario identification. Cluster size can be divided into clusters, large clusters, and super-large clusters. Clusters with fewer than 10 nodes are considered clusters, those with 10 to 100 nodes are large clusters, and those with more than 100 nodes are super-large clusters. Process status changes are divided into node-level changes and cluster-level changes. Node-level changes refer to changes in the status of more than 30% of cluster processes, while cluster-level changes refer to changes in the status of more than 30% of the nodes. The purpose of scenario identification is to assign different weights to the dynamic batch processing strategy. The larger the storage cluster size or the greater the number of process changes, the higher the weight. The time window duration is dynamically adjusted based on the scenario identification results. In scenarios with large-scale changes, the time window duration is shortened to speed up request processing and reduce latency.

[0076] Specifically, if the weights for cluster, large cluster, and super-large cluster are 1, 2, and 3 respectively, and the total number of process nodes in the storage cluster is 80, then the storage cluster is a large cluster, with a weight of 2, which serves as the first factor. If the weights for node-level changes and cluster-level changes are 1 and 2 respectively, and the number of process changes in the cluster process state is 50%, then the change scale is a cluster-level change, with a corresponding weight of 2, which serves as the second factor. Summing the first and second factors yields 4, which is used as the comprehensive factor.

[0077] Furthermore, if the preset time window duration corresponding to the cluster size is 500ms, the preset time window duration corresponding to the large cluster size is 300ms, and the preset time window duration corresponding to the ultra-large cluster size is 100ms, then the target time window duration for the current large cluster is 300ms, and the actual window duration is adjusted according to the target time window duration. That is, the actual window duration is the ratio of the target time window duration to the comprehensive factor, i.e., 300ms / 4 = 75ms. Data operation requests within the actual window duration are then acquired and added to the batch processing queue for merging processing.

[0078] In the above embodiments, the cluster size is assessed by obtaining the total number of nodes in the storage cluster to generate a first factor, and the change scale is assessed by monitoring the number of process changes to generate a second factor. These two factors are then weighted and fused to obtain a comprehensive factor. The actual window duration is then calculated based on a preset time window duration. Finally, data operation requests within this duration are aggregated and added to the batch processing queue. This allows the system to automatically shorten the time window to accelerate processing and reduce latency when the number of nodes increases or changes are frequent, and to automatically extend the window to improve throughput efficiency when the cluster size is stable or changes are sparse. Simultaneously, it avoids resource waste or response delays caused by deviations in a single metric, thereby enhancing the efficiency of the batch processing process.

[0079] S210: Determine if there are conflicting requests in the batch processing queue.

[0080] In this embodiment, a conflicting request refers to a mutually exclusive operation on the same resource, that is, two operation requests that update the same attribute of the same object in a contradictory manner. Specifically, the conflicting request rules include no cross-object conflict, that is, a conflict is only considered when the objects and attributes are exactly the same, such as the OSD-01 (Object-based Storage Device) state change and the OSD-02 state change not conflicting; it also includes no cross-attribute conflict, that is, changes to different attributes of the same object do not conflict, such as the OSD-01 state change and the OSD-01 capacity change can be performed in parallel. If a conflicting request exists, step S211 is executed; if no conflicting request exists, step S212 is executed.

[0081] S211: In response to a conflicting request, process the conflicting request.

[0082] In this embodiment, the most recent data operation request is obtained based on the timestamp as the target operation request; the target operation request in the conflicting requests is retained in the batch processing queue, and other data operation requests in the conflicting requests are moved to the batch processing queue corresponding to the next actual window duration.

[0083] Specifically, if request A sets the OSD status to "up" with a timestamp of 10ms, request B sets the OSD capacity to 90% with a timestamp of 30ms, request C sets the OSD status to "down" with a timestamp of 50ms, and request D sets the OSD status to "up" with a timestamp of 70ms, then request C contradicts request A, and request D contradicts request C. Based on timestamp order, request D takes precedence over request C, which in turn takes precedence over request A. The result of these contradictory requests is that request D is retained in the current window's batch queue, requests C and A are moved to the batch queue corresponding to the next actual window duration, and request B is retained in the current window's batch queue. Further, when processing the batch queue corresponding to the next actual window duration, it is necessary to again check for contradictory requests and process them according to the contradictory request rules.

[0084] S212: In response to the absence of conflicting requests, merge and process data operation requests.

[0085] In this embodiment, if there are no conflicting requests in the batch processing queue, the data operation requests in the batch processing queue can be merged, and a transaction mechanism can be used to commit the complete change operation. For example, merging request B and request D in processing step S211.

[0086] In the above embodiments, conflicting requests in the batch processing queue are detected to resolve conflicts. Simultaneously, timestamp-based priority ensures that the latest operations are executed promptly, eliminating the risk of inconsistencies caused by outdated instructions and guaranteeing the temporal integrity of cluster changes. Delaying non-latest requests to subsequent windows prevents current processing flow from being blocked and maintains the continuity of the batch processing queue. In scenarios without conflicting requests, merging processing improves transaction throughput and reduces system overhead.

[0087] In one embodiment, such as Figure 6 As shown, Figure 6 This is a flowchart illustrating another request processing method provided in an embodiment of this application, which selects the process node executing the read request as the target process node based on the load score of the process node, including:

[0088] S301: Evaluate the load score of the process node.

[0089] In this embodiment, the current latency, historical average latency, and number of connections of a process node are obtained, and the current latency, historical average latency, and number of connections of the process node are weighted and fused to obtain its load score.

[0090] Specifically, the monitoring process sends the current latency for processing the current client request, the historical average latency for processing the request, and the number of client connections to the client. The client maintains a data structure to cache the load metrics of the process nodes used.

[0091] Furthermore, the current latency, historical average latency, and number of connections are normalized, and a load score is calculated based on the processed values. The load score is calculated as shown in Equation 1-1:

[0092] m score =cur latency *w1+avg latency *w2+con*w3 formula 1-1

[0093] Where, m score Indicates load score, cur latency Indicates the current latency, avg latency The historical average latency is represented by w1, the number of connections is represented by w2, and the weights of the current latency, the historical average latency, and the number of connections are represented by w1, w2, and w3, respectively.

[0094] Specifically, different weights are assigned to the current latency, historical average latency, and number of connections. For example, w1 is 0.5, w2 is 0.3, and w3 is 0.2. Usually, the most recent latency information is the most valuable, so it is given the largest weight.

[0095] S302: Probability factor for evaluating process nodes.

[0096] In this embodiment, to prevent all data operation requests from flooding the same low-load process node instantly, a probability calculation is used to assign a different selection probability value to each process node, thus distinguishing the selection probabilities of different process nodes. The lower the load of a process node, the higher its selection probability. The load score of the process node and a preset smoothing factor are fitted as attenuation factors. A transformation value matching the attenuation factor of the process node is calculated, and the transformation values ​​of all process nodes are summed to obtain the total transformation value. The ratio of the transformation value of a process node to the total transformation value is used as its probability factor.

[0097] Specifically, the probability factor is calculated as shown in Equation 1-2:

[0098]

[0099] Where P represents the probability factor, i.e., the probability of being selected, e is a natural number, k represents the preset smoothing factor, such as 3, and m score This indicates the load score.

[0100] For example, when the load score is 0.25, the smoothing factor is 3, and the total conversion value is 1.34, the calculated probability factor is 35.2%; when the load score is 0.45, the smoothing factor is 3, and the total conversion value is 1.34, the calculated probability factor is 19.3%. That is, the higher the load score, the greater the load pressure on the process node, and the lower the probability value of being selected.

[0101] S303: Select the target process node from the process nodes according to the probability factor.

[0102] In this embodiment, data operation requests are randomly assigned to the target process node based on the probability value of the process node being selected.

[0103] In the above embodiments, a load score is generated by weighting and merging the current latency, historical average latency, and number of connections of the process nodes. A differentiated design, assigning the highest weight to current latency and the lowest weight to the number of connections, accurately quantifies the real-time stress state of the nodes. Furthermore, a decay function is constructed by introducing a smoothing factor, non-linearly mapping the load score to a probability factor. The selection probability is dynamically allocated based on the ratio of each node's converted value to the total converted value, forming a dual optimization mechanism. The load score dynamically captures the changing trend of node processing capacity from multiple dimensions, avoiding misjudgments caused by single-indicator bias. The probability factor conversion effectively suppresses the risk of instantaneous overload of low-load nodes through non-linear decay, ensuring that request allocation responds to both real-time load status and long-term balance. This reduces the pressure peak of high-load nodes while avoiding waste of cluster resource utilization, thereby improving the overall stability and throughput efficiency of the storage cluster.

[0104] In one embodiment, such as Figure 7 As shown, Figure 7 This is a schematic diagram of a request processing device provided in an embodiment of this application. The request processing device may include a first acquisition module 21, a request module 22, a second acquisition module 23, and an allocation module 24.

[0105] The first acquisition module 21 is used to acquire data operation requests from the storage cluster.

[0106] Request module 22 is used to allocate data operation requests to target process nodes, so that the target process node requests a data operation lock that matches the current operation type, where the current operation type is the operation type to which the data operation request belongs.

[0107] The second acquisition module 23 is used to verify whether the current environment allows the allocation of a data operation lock in response to the request to acquire a data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types.

[0108] The allocation module 24 is used to allocate a data operation lock in response to a data operation request, provided that the current environment allows the allocation of the data operation lock.

[0109] Each module in the aforementioned request processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the electronic device, or stored in the memory of the electronic device in software form, so that the processor can call and execute the operations corresponding to each module.

[0110] Embodiments of this application also provide an electronic device, including a memory for storing computer programs;

[0111] A processor, when executing a computer program, can perform at least the following steps:

[0112] Obtain a data operation request from the storage cluster; allocate the data operation request to the target process node; cause the target process node to request a data operation lock matching the current operation type, where the current operation type is the operation type to which the data operation request belongs; in response to the request to acquire a data operation lock, verify whether the current environment allows the allocation of a data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types; in response to the current environment allowing the allocation of a data operation lock, allocate a data operation lock in response to the data operation request.

[0113] In one embodiment, the electronic device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores software management data. The network interface is used for communication with external terminals via a network connection.

[0114] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0115] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, can perform at least the following steps:

[0116] Obtain a data operation request from the storage cluster; allocate the data operation request to the target process node; cause the target process node to request a data operation lock matching the current operation type, where the current operation type is the operation type to which the data operation request belongs; in response to the request to acquire a data operation lock, verify whether the current environment allows the allocation of a data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types; in response to the current environment allowing the allocation of a data operation lock, allocate a data operation lock in response to the data operation request.

[0117] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0118] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, can perform at least the following steps:

[0119] Obtain a data operation request from the storage cluster; allocate the data operation request to the target process node; cause the target process node to request a data operation lock matching the current operation type, where the current operation type is the operation type to which the data operation request belongs; in response to the request to acquire a data operation lock, verify whether the current environment allows the allocation of a data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types; in response to the current environment allowing the allocation of a data operation lock, allocate a data operation lock in response to the data operation request.

[0120] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0121] It will also be appreciated that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0122] The foregoing has provided a detailed description of a request processing method, system, device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A request processing method, characterized in that, The request processing method includes: Retrieve data operation requests from the storage cluster; The data operation request is assigned to a target process node, so that the target process node requests a data operation lock that matches the current operation type, where the current operation type is the operation type to which the data operation request belongs. In response to a request to acquire a data operation lock, the system verifies whether the current environment allows the allocation of the data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types; In response to the current environment allowing the allocation of the data operation lock, the data operation lock is allocated in response to the data operation request.

2. The request processing method according to claim 1, characterized in that, The process node includes a master process node and slave process nodes; the step of allocating the data operation request to the target process node includes: Determine the request type of the data operation request; In response to the request type being a write request, the main process node is designated as the target process node to route the write request to the main process node; In response to the request type being a read request, the process node executing the read request is selected as the target process node based on the load score of the process node.

3. The request processing method according to claim 2, characterized in that, The step of selecting the process node that executes the read request as the target process node based on the load score of the process node includes: Obtain the current latency, historical average latency, and number of connections of the process node; The load score of a process node is obtained by weighted fusion of its current latency, historical average latency, and number of connections; wherein the weights of the current latency, the historical average latency, and the number of connections decrease in that order. The probability value of the process node that matches the load score is evaluated as a probability factor; The target process node is selected from the process nodes according to the probability factor.

4. The request processing method according to claim 3, characterized in that, The probability value of selecting the process node that matches the load score, as a probability factor, includes: The load score of the process node and a preset smoothing factor are used as the decay factor. Calculate the conversion value that matches the decay factor of the process node; The total conversion value is obtained by summing the conversion values ​​of each process node. The ratio of the transformation value of the process node to the total transformation value is used as its probability factor.

5. The request processing method according to claim 1, characterized in that, The data operation lock includes a shared lock and an exclusive lock. The shared lock is used to record the number of times the read lock is acquired, and the number of acquisitions represents the number of read requests being processed. The exclusive lock is used to mark the acquisition status of the write lock. The step of verifying whether the current environment allows the allocation of the data operation lock includes: When the data operation lock is a read lock, the number of times the read lock is acquired is accumulated using the shared lock; in response to the completion of the read request, the read lock is released and the number of acquisitions is released. When the data operation lock is a write lock, determine whether the number of times the read lock has been acquired is zero and whether the exclusive lock is in an unacquired state; in response to the number of times the read lock has been acquired being zero and the exclusive lock is in an unacquired state, acquire the write lock and enter the batch processing queue; in response to the number of times the read lock has been acquired being non-zero and / or the exclusive lock is in an acquired state, the write request enters a waiting state.

6. The request processing method according to claim 5, characterized in that, The step of acquiring the write lock and entering the batch processing queue includes: Obtain the total number of process nodes in the storage cluster, evaluate the storage cluster size that matches the total number of nodes, and obtain a first factor that matches the storage cluster size. Monitor the number of process changes that have occurred, assess the change scale that matches the number of process changes, and obtain a second factor that matches the change scale; The combined factor is obtained by summing the first factor and the second factor. The target time window duration is determined by matching a preset time window duration with a time window duration associated with the storage cluster size. The ratio of the target time window duration to the comprehensive factor is taken as the actual window duration; Obtain the data operation requests within the actual window duration and add the data operation requests to the batch processing queue.

7. The request processing method according to claim 6, characterized in that, The process of adding the data operation request to the batch processing queue also includes: Determine whether there are conflicting requests in the batch processing queue; In response to the existence of the conflicting request, the most recent data operation request is obtained based on the timestamp as the target operation request; the target operation request in the conflicting request is retained in the batch processing queue, and other data operation requests in the conflicting request are moved into the batch processing queue corresponding to the next actual window duration; In response to the absence of the conflicting request, the data operation requests in the batch queue are merged.

8. A request processing apparatus, characterized in that, include: The first acquisition module is used to acquire data operation requests from the storage cluster. The request module is used to allocate the data operation request to the target process node; The target process node is instructed to request a data operation lock that matches the current operation type, where the current operation type is the operation type to which the data operation request belongs. The second acquisition module is used to verify whether the current environment allows the allocation of the data operation lock in response to the request to acquire the data operation lock; wherein, the current environment includes the operation lock status of the current operation type and the operation lock status of other operation types; An allocation module is configured to allocate the data operation lock in response to the current environment allowing the allocation of the data operation lock, in response to the data operation request.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the request processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the request processing method as described in any one of claims 1 to 7.

Images