Data processing method, device, equipment and medium

Abstract

The present invention discloses a data processing method, apparatus, device and medium. When it is determined that the target storage space cannot be allocated for the obtained target request at present, the sleep time of the target request is determined, and the state of the target request is determined to be a sleep state, that is, the task of allocating the target storage space for the target request is suspended. Subsequently, when it is determined that the duration of the target request in the sleep state reaches the sleep duration, the task of allocating the target storage space for the target request is resumed. Since the task of allocating storage space to some obtained requests can be suspended when the storage space is insufficient, the number of tasks of allocating storage space to the requests is reduced, the pressure of allocating storage space to all obtained requests is reduced, and the request process is avoided from frequently notifying the elimination thread, resulting in the request thread and the elimination thread and the GC thread competing for resources, thereby ensuring the working efficiency of the elimination thread and the GC thread, and thus improving the efficiency of processing the target request.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a data processing method, apparatus, device and medium. Background Art

[0002] Block Cache (BCache) is a Linux kernel-level block cache that allows high-speed hard drives (such as solid-state drives (SSDs)) to act as cache for one or more low-speed hard drives (such as mechanical hard drives (HDDs)). It is one of the most common SSD cache solutions.

[0003] In actual use, if block caching technology is used to allocate storage space for a target request (e.g., an I / O request), the requesting thread first determines the storage space to allocate for the target request from the storage space contained in the free list queue. If there is insufficient storage space available in the free list queue, the requesting thread can notify the elimination thread, which is responsible for eliminating storage space, to work, so that the elimination thread can add the reclaimed storage space to the free list queue. If the elimination thread is unable to reclaim storage space, the elimination thread needs to notify the garbage collection (GC) thread to work, so that the GC thread can mark some storage space as reclaimable, thereby facilitating the elimination thread to reclaim the marked reclaimable storage space. The reclaimed storage space is then added to the free list queue, and the requesting thread can allocate storage space for the target request from the added free list queue. During this process, if the storage space in the electronic device is largely occupied or severely insufficient, the requesting thread will frequently notify the elimination thread, causing the requesting thread to compete with the elimination thread and the GC thread for resources, significantly reducing the efficiency of the elimination and GC threads, and thus reducing the efficiency of processing the target request. Summary of the Invention

[0004] The embodiments of the present invention provide a data processing method, apparatus, device and medium for solving the problem that existing request threads, elimination threads and GC threads seize resources, thereby affecting the efficiency of processing target requests.

[0005] An embodiment of the present invention provides a data processing method, the method comprising:

[0006] If it is determined that the target storage space cannot be allocated for the obtained target request, determining that the state of the target request is a sleep state and the sleep duration corresponding to the target request; wherein the sleep duration is used to indicate the length of time the target request is in the sleep state, and the sleep state is used to indicate suspending the task of allocating the target storage space for the target request;

[0007] If it is determined that the duration of the target request being in the sleep state reaches the sleep duration, the task of allocating the target storage space to the target request is resumed.

[0008] Furthermore, the determining that target storage space cannot be allocated for the obtained target request includes one or more of the following:

[0009] The capacity of the free storage space is smaller than the capacity of the target storage space;

[0010] The storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information;

[0011] The obtained backend device status information is congested.

[0012] Further, determining the sleep time of the target request includes:

[0013] The sleep time of the target request is determined according to the ratio between the number of requests currently being processed and the preset number of read and write times per unit time.

[0014] Furthermore, after determining that the target storage space cannot be allocated for the acquired target request, and before determining that the state of the target request is a sleep state, and before determining the sleep duration corresponding to the target request, the method further includes:

[0015] Make sure the dynamic flow control switch is in the on state.

[0016] An embodiment of the present invention provides a data processing device, comprising:

[0017] a first processing unit, configured to, if it is determined that the target storage space cannot be allocated for the obtained target request, determine that the state of the target request is a sleep state and a sleep duration corresponding to the target request; wherein the sleep duration is used to indicate a length of time the target request is in the sleep state, and the sleep state is used to indicate a suspension of a task of allocating the target storage space for the target request;

[0018] The second processing unit is configured to resume the task of allocating the target storage space to the target request if it is determined that the duration of the target request being in the sleep state reaches the sleep duration.

[0019] Furthermore, the first processing unit is specifically configured to determine that target storage space cannot be allocated for the obtained target request by one or more of the following methods:

[0020] The capacity of the free storage space is smaller than the capacity of the target storage space;

[0021] The storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information;

[0022] The obtained backend device status information is congested.

[0023] Furthermore, the first processing unit is specifically configured to determine the sleep time of the target request according to a ratio between the number of requests currently being processed and a preset number of read and write operations per unit time.

[0024] Furthermore, the first processing unit is also used to determine that after determining that the target storage space cannot be allocated for the obtained target request, the state of the target request is determined to be a sleep state, and before the sleep time corresponding to the target request, determine that the switch state of the dynamic flow control switch is an open state.

[0025] An embodiment of the present invention provides an electronic device, comprising a processor, wherein the processor is configured to implement the steps of any of the above-mentioned data processing methods when executing a computer program stored in a memory.

[0026] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the steps of any of the above-mentioned data processing methods.

[0027] During the process of allocating target storage space for target requests, if it is determined that target storage space cannot be allocated for the acquired target requests at present, the sleep time of the target request is determined, and the state of the target request is determined to be a sleep state, that is, the task of allocating target storage space for the target request is suspended. Subsequently, when it is determined that the target request has been in a sleep state for a certain length of time, the task of allocating target storage space for the target request is resumed. Since the task of allocating storage space for some acquired requests can be suspended when storage space is insufficient, the number of tasks of allocating storage space for requests is reduced, the pressure of allocating storage space for all acquired requests is reduced, and the request process is avoided from frequently notifying the elimination thread, which causes the request thread to compete with the elimination thread and the GC thread for resources. This ensures the working efficiency of the elimination thread and the GC thread, thereby improving the efficiency of processing the target requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 A schematic diagram of a data processing process provided by an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a principle for determining sleep duration provided by an embodiment of the present invention;

[0031] Figure 3 A schematic diagram of a specific data processing flow provided by an embodiment of the present invention;

[0032] Figure 4 A schematic diagram of a specific sleep process provided by an embodiment of the present invention;

[0033] Figure 5 A schematic structural diagram of a dynamic flow control component provided by an embodiment of the present invention;

[0034] Figure 6 A structural diagram of a data processing device is provided for an embodiment of the present invention;

[0035] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0037] Those skilled in the art will appreciate that the embodiments of the present invention may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0038] It should be understood herein that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0039] In order to improve the efficiency of processing target requests, embodiments of the present invention provide a data processing method, apparatus, device, and medium.

[0040] Example 1:

[0041] Figure 1 A schematic diagram of a data processing process provided by an embodiment of the present invention includes:

[0042] S101: If it is determined that the target storage space cannot be allocated for the obtained target request, determine that the state of the target request is a sleep state, and the sleep duration corresponding to the target request; wherein the sleep duration is used to indicate the length of time the target request is in the sleep state, and the sleep state is used to indicate the suspension of the task of allocating the target storage space for the target request.

[0043] The data processing method provided by the embodiment of the present invention is applied to an electronic device, which may be a smart device such as a mobile terminal, a laptop computer, etc., or a server, a cache, etc.

[0044] In the embodiment of the present invention, the storage space allocated for the target request may be determined as the target storage space.

[0045] In an example, the target request may be an input / output (IO) request, that is, the target request may be a read request or a write request.

[0046] When an electronic device obtains a target request, for example, a write request, it can process the target request, thereby allocating target storage space for the target request. In the process of allocating target storage space for the target request, there may be insufficient storage space of the electronic device, resulting in a situation where target storage space cannot be allocated for the obtained target request in a short period of time. Therefore, in an embodiment of the present invention, if it is determined that target storage space cannot be allocated for the obtained target request, indicating that the storage space of the current electronic device is insufficient, the task of allocating target storage space for the target request can be suspended, so that when the storage space of the subsequent electronic device is sufficient, the target storage space can be allocated for the target request, thereby reducing the number of tasks for allocating storage space for requests, reducing the pressure of allocating storage space for all obtained requests, avoiding the request process frequently notifying the elimination thread, resulting in the request thread and the elimination thread and GC thread competing for resources, ensuring the working efficiency of the elimination thread and GC thread, and thus improving the efficiency of processing the target request.

[0047] In an embodiment of the present invention, a state can be configured for a target request. The state includes at least a sleep state, and the sleep state is used to identify the task of suspending the allocation of the target storage space for the target request. It can also be understood that the sleep state is used to identify the thread that allocates the target storage space for the target request to enter sleep. If the electronic device determines that it cannot allocate the target storage space for the obtained target request, it can determine that the state of the target request is a sleep state. When the subsequent electronic device processes the target request, it can determine whether to suspend the task of allocating the target storage space for the target request by determining whether the state of the target request is a sleep state, thereby accurately processing the target request and improving the accuracy of control over the electronic device.

[0048] When the electronic device determines that it cannot allocate target storage space for the acquired target request, it can also determine the duration of suspending the task of allocating target storage space for the target request. For example, it can determine the sleep duration corresponding to the target request, that is, the length of time the target request is in the sleep state. This can further accurately control the electronic device and facilitate subsequent electronic devices to allocate target storage space for the target request in a timely manner, thereby improving the efficiency of processing the target request.

[0049] In a possible implementation, the sleep duration may be a preset duration value, or may be determined based on the capacity of the current free storage space. For example, based on a pre-configured correspondence between the capacity of the free storage space and the duration value, the duration value corresponding to the capacity of the current free storage space is determined, and the duration value is determined as the sleep duration.

[0050] In another possible implementation, considering that the target request can be processed in a timely manner after the sleep state is terminated, in this embodiment of the present invention, the sleep duration can be determined based on the number of read and write operations per unit time of the electronic device and the number of requests currently being processed. Figure 2 A schematic diagram of a principle for determining sleep duration provided by an embodiment of the present invention is shown as follows: Figure 2 As shown, assuming that the average processing time of one request is T, and the number of requests currently being processed is N, in the process of processing these N requests, if the electronic device uses a completely parallel processing method to complete the N requests, it only takes T time, but if the electronic device uses a completely serial processing method to complete the N requests, it takes NT time. In the process of the electronic device processing the N requests, once a new request is received, it can be determined that the newly received request is in a dormant state, and after the electronic device processes a request, it resumes the task of allocating storage space for the newly received request to ensure that the electronic device can allocate storage space for the newly received request in a timely manner.

[0051] The unit time may be 1 second, 1 minute, etc. For example, if the unit time is 1 second, the number of read and write operations per second (IOPS) is the number of read and write operations per second.

[0052] In an example, the sleep time of the target request may be determined according to a ratio between the number of requests currently being processed and a preset number of read and write times per unit time.

[0053] For example, the average time it takes for the electronic device to process each request may be determined based on the number of requests currently being processed and the preset number of read and write times per unit time, and the average time may be determined as the sleep duration.

[0054] Based on the number of requests currently being processed and the preset number of read and write times per unit time, the average time it takes for the electronic device to process each request can be expressed by the following formula:

[0055]

[0056] Where TIME represents the average time it takes for an electronic device to process N requests, and T represents the average processing time it takes for an electronic device to process one request. Expected_throughput_iops is the preset number of read and write times per unit time. It is understandable that the average time for an electronic device to process N requests is

[0057] In the embodiment of the present invention, situations where the electronic device cannot allocate target storage space for the acquired target request may include, but is not limited to, one or more of the following:

[0058] Case 1: The capacity of the free storage space is smaller than the capacity of the target storage space.

[0059] When an electronic device obtains a target request, it can allocate a target storage space for the target request. For example, the target storage space is allocated for the target request by running the request thread (such as an IO request thread) corresponding to the target request. Exemplarily, the request thread corresponding to the target request obtains the capacity of the target storage space required by the target request and the capacity of the current free storage space. If it is determined that the capacity of the current free storage space is not less than the capacity of the target storage space, it means that the storage space in the current electronic device is relatively sufficient, and space can be allocated for the target request, then the target storage space is determined from the current free storage space.

[0060] In one possible implementation, the electronic device may have insufficient free storage space. Therefore, in an embodiment of the present invention, if the electronic device determines that the capacity of the current free storage space is less than the capacity of the target storage space, it means that the storage space in the current electronic device is insufficient, and it can be determined that the target storage space cannot be allocated for the obtained target request.

[0061] For example, the request thread corresponding to the target request obtains the capacity of free storage space from the free list queue and then determines whether the current capacity of free storage space is not less than the capacity of the target storage space. If it is determined that the current capacity of free storage space is not less than the capacity of the target storage space, the target storage space is determined from the free storage space included in the free list queue. If it is determined that the current capacity of free storage space is less than the capacity of the target storage space, it is determined that the target storage space cannot be allocated for the obtained target request.

[0062] Case 2: the capacity of the free storage space is smaller than the capacity of the target storage space, and the storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any address information of an incomplete flush.

[0063] In actual application, considering that when the data in the storage space corresponding to a certain address information is flushed back to the backend device, for example, other SSDs, HDDs, etc. may receive a request to access the data in the storage space corresponding to the address information again. If the request is allowed to continue to access the data in the storage space corresponding to the address information at this time, the data in the storage space corresponding to the address information may be changed, resulting in the data flushed back to the backend device being contaminated, causing the data flushed back to the backend device to be inconsistent with the actual data, and other problems. Therefore, in an embodiment of the present invention, the electronic device can determine whether the target storage space can be allocated for the target request based on whether the storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information. This can avoid the target request from contaminating the data in the storage space corresponding to any unfinished flushed address information, and ensure the accuracy of the data flushed back to the backend device. Exemplary, after obtaining a target request, the electronic device can determine the storage space corresponding to the address information carried in the target request and the storage space corresponding to the unfinished flushed address information. If it is determined that the storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information, it means that the target request may cause pollution to the data in the storage space corresponding to the unfinished flushed address information, and it is determined that the target storage space cannot be allocated for the obtained target request.

[0064] For example, when a request thread in an electronic device determines that there is insufficient free storage space in the free list queue to serve as the target storage space, the request thread can notify the elimination thread responsible for eliminating storage space, which can then add the reclaimed storage space to the free list queue. For example, the elimination thread can reclaim storage space from the reclaimable queue and add the reclaimed storage space to the free list queue. If the elimination thread is unable to reclaim storage space, meaning the reclaimable queue is empty, the elimination thread can notify the garbage collection thread, which can then mark some storage space as reclaimable. After the electronic device triggers the GC thread, it can determine the storage space corresponding to the address information carried in the target request and the storage space corresponding to the address information of any incomplete flushing. It can then determine whether the storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any incomplete flushing address information, thereby determining whether storage space can be allocated for the received target request. Specifically, if it is determined that the storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information, it means that the target request may cause pollution to the data in the storage space corresponding to the unfinished flushed address information, then it is determined that the target storage space cannot be allocated for the obtained target request.

[0065] Case 3: It is determined that the capacity of the free storage space is smaller than the capacity of the target storage space, and the acquired status information of the backend device is congestion.

[0066] In actual application, when the load on the backend device is too large, if the request thread of the electronic device frequently notifies the elimination thread, the load on the electronic device will be further increased, and the request thread will compete with the elimination thread, the GC thread, and the thread interacting with the backend device for resources, which will seriously reduce the working efficiency of the elimination thread, the GC thread, and the thread interacting with the backend device, thereby reducing the efficiency of processing the target request. Therefore, in an embodiment of the present invention, the electronic device can determine whether the backend device is in an overloaded state based on whether the status information of the backend device is congested. When it is determined that the backend device is in an overloaded state, it will promptly process data to avoid the request process frequently notifying the elimination thread, which will cause the request thread to compete with the elimination thread, the GC thread, and the thread interacting with the backend device for resources, thereby ensuring the working efficiency of the elimination thread, the GC thread, and the thread interacting with the backend device, thereby improving the efficiency of processing the target request. For example, after obtaining a target request, the electronic device can obtain the status information of the backend device. If it is determined that the obtained status information of the backend device is congested, it means that the backend device is in an overloaded state, and then it is determined that the target storage space cannot be allocated for the obtained target request.

[0067] For example, after the electronic device triggers the GC thread to work based on the above embodiment, the electronic device can determine whether the storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished back-flushing address information. If the storage space corresponding to the address information carried in the target request does not overlap with the storage space corresponding to any unfinished back-flushing address information, the status information of the back-end device can be obtained. If it is determined that the status information of the back-end device is congested, indicating that the back-end device is in an overloaded state, it is determined that the target storage space cannot be allocated for the obtained target request.

[0068] In a possible embodiment, a dynamic flow control switch is also pre-configured, and the dynamic flow control switch corresponds to a switch state, which includes at least an open state and a closed state. After the electronic device determines that it cannot allocate target storage space for the acquired target request, before determining that the state of the target request is a sleep state and the sleep duration corresponding to the target request, it can determine whether the switch state of the dynamic flow control switch is an open state. When it is determined that the switch state of the dynamic flow control switch is an open state, the step of determining that the state of the target request is a sleep state and the sleep duration corresponding to the target request is performed. In this way, by adjusting the switch state of the dynamic flow control switch, it can be determined whether to execute the solution of suspending the task of allocating storage space for a part of the requests when the storage space of the electronic device is insufficient. In this way, in a scenario where only the processing rate of the electronic device for the target request is required, the processing efficiency of the electronic device for the target request can be guaranteed, and the electronic device can be accurately controlled according to the scenario requirements, thereby improving the flexibility of the data processing solution provided by the present application.

[0069] Specifically, when the electronic device determines that it cannot allocate target storage space for the acquired target request, it can obtain the switch state of the dynamic flow control switch and determine whether the switch state is on. If it is determined that the switch state of the dynamic flow control switch is on, indicating that data processing is required, then the subsequent steps of determining that the state of the target request is a sleep state and the corresponding sleep duration of the target request are executed; if it is determined that the switch state of the dynamic flow control switch is off, indicating that data processing is not required, then the subsequent steps of determining that the state of the target request is a sleep state and the corresponding sleep duration of the target request are not executed, and the target storage space is directly re-allocated for the target request.

[0070] S102: If it is determined that the duration of the target request being in the sleep state reaches the sleep time, resuming the task of allocating the target storage space to the target request.

[0071] When the electronic device suspends the task of allocating target storage space for the target request based on the above embodiment, it starts to record the duration of the target request being in the sleep state. When it is determined that the duration of the target request being in the sleep state reaches the determined sleep duration, the task of allocating target storage space for the target request can be resumed, thereby achieving timely allocation of target storage space for the target request.

[0072] In one example, when the electronic device suspends the task of allocating target storage space to the target request based on the above embodiment, the electronic device can initialize and start a preconfigured timer. When it is determined that the time recorded in the timer is equal to the determined sleep duration, it is determined that the duration of the target request in the sleep state reaches the determined sleep duration, and the task of allocating the target storage space to the target request is restarted.

[0073] During the process of allocating target storage space for target requests, if it is determined that target storage space cannot be allocated for the acquired target requests at present, the sleep time of the target request is determined, and the state of the target request is determined to be a sleep state, that is, the task of allocating target storage space for the target request is suspended. Subsequently, when it is determined that the target request has been in a sleep state for a certain length of time, the task of allocating target storage space for the target request is resumed. Since the task of allocating storage space for some acquired requests can be suspended when storage space is insufficient, the number of tasks of allocating storage space for requests is reduced, the pressure of allocating storage space for all acquired requests is reduced, and the request process is avoided from frequently notifying the elimination thread, which causes the request thread to compete with the elimination thread and the GC thread for resources. This ensures the working efficiency of the elimination thread and the GC thread, thereby improving the efficiency of processing the target requests.

[0074] Example 2:

[0075] The data processing method provided by the embodiment of the present invention is described below through specific embodiments. Figure 3 A schematic diagram of a specific data processing flow provided by an embodiment of the present invention, the flow includes:

[0076] S301: The requesting thread obtains an IO request.

[0077] S302: The requesting thread obtains the capacity of the current free storage space and the capacity of the target storage space required by the target request, and determines whether the capacity of the free storage space is not less than the capacity of the target storage space. If so, execute S303; otherwise, execute S304.

[0078] S303: The requesting thread determines a target storage space from the current free storage space.

[0079] S304: The requesting thread triggers the elimination thread.

[0080] The specific process of triggering the elimination thread includes: the request thread notifies the elimination thread responsible for eliminating storage space, which then adds the reclaimed storage space to the free list. If the elimination thread is unable to reclaim storage space, meaning the reclaimable queue is empty, the elimination thread notifies the garbage collection thread, which then marks some storage space as reclaimable.

[0081] S305: Eliminating the thread triggers the GC thread.

[0082] After the GC thread is triggered, some storage spaces are marked as recyclable through the GC thread.

[0083] S306: Determine whether the storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information. If so, execute S310; otherwise, execute S307.

[0084] S307: Obtain status information of the backend device.

[0085] S308: Determine whether the status information of the backend device is congested. If so, execute S310; otherwise, execute S309.

[0086] S309: Execute the Bypass IO instruction.

[0087] The specific steps of executing the Bypass IO instruction belong to the existing technology and will not be detailed here.

[0088] S310: Execute the sleep step.

[0089] The specific steps for executing sleep can be found in Figure 4 , Figure 4 A specific sleep process diagram provided in an embodiment of the present invention includes:

[0090] S401: Obtain the switch state of the dynamic flow control switch and determine whether the switch state is open. If it is determined that the switch state of the dynamic flow control switch is open, then execute S402; otherwise, execute the end.

[0091] S402: Determine the sleep time of the target request according to the ratio between the number of requests currently being processed and the preset number of read and write times per unit time.

[0092] S403: Determine that the state of the target request is a sleep state.

[0093] S404: If it is determined that the duration of the target request being in the sleep state reaches the sleep duration, the task of allocating target storage space to the target request is resumed.

[0094] Based on the above description, the above S301 to S308 can be concentrated in the dynamic throttle component. Figure 5 A structural diagram of a dynamic flow control component provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the component may include a sleep (Delay) submodule, a state statistics (Statistic) submodule, a calculation (Calculate) submodule, and a configuration (Config) submodule.

[0095] The Statistic submodule is used to implement statistics of information required during data processing, such as the capacity of the target storage space required for obtaining the target request, the capacity of the current free storage space, and information about the free list queue.

[0096] The Delay submodule is used to suspend the task of allocating target storage space for the target request when it is determined that the target storage space cannot be allocated for the obtained target request, and to resume the task of allocating target storage space for the target request when it is determined that the target request has been in a sleep state for a period of time that reaches the sleep duration.

[0097] The Config module is implemented through a memory-based file system (sysfs), mainly including the preset number of read and write times per unit time, the maximum number of requests allowed to be processed, the on / off status of the dynamic flow control switch, etc.

[0098] Calculate module, used for sleep duration calculation module.

[0099] Example 3:

[0100] An embodiment of the present invention provides a data processing device, Figure 6 A schematic structural diagram of a data processing device is provided for an embodiment of the present invention, the device comprising:

[0101] The first processing unit 61 is configured to, if it is determined that the target storage space cannot be allocated for the obtained target request, determine that the state of the target request is a sleep state and a sleep duration corresponding to the target request; wherein the sleep duration is used to indicate a length of time the target request is in the sleep state, and the sleep state is used to indicate a suspension of a task of allocating the target storage space for the target request;

[0102] The second processing unit 62 is configured to resume the task of allocating the target storage space to the target request if it is determined that the duration of the target request being in the sleep state reaches the sleep duration.

[0103] Furthermore, the first processing unit 61 is specifically configured to determine that target storage space cannot be allocated for the obtained target request by one or more of the following methods:

[0104] The capacity of the free storage space is smaller than the capacity of the target storage space;

[0105] The storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information;

[0106] The obtained backend device status information is congested.

[0107] Furthermore, the first processing unit 61 is specifically configured to determine the sleep time of the target request according to a ratio between the number of requests currently being processed and a preset number of read and write operations per unit time.

[0108] Furthermore, the first processing unit 61 is also used to determine that after it is determined that the target storage space cannot be allocated for the obtained target request, the state of the target request is determined to be a sleep state, and before the sleep time corresponding to the target request, the switch state of the dynamic flow control switch is determined to be an open state.

[0109] During the process of allocating target storage space for target requests, if it is determined that target storage space cannot be allocated for the acquired target requests at present, the sleep time of the target request is determined, and the state of the target request is determined to be a sleep state, that is, the task of allocating target storage space for the target request is suspended. Subsequently, when it is determined that the target request has been in a sleep state for a certain length of time, the task of allocating target storage space for the target request is resumed. Since the task of allocating storage space for some acquired requests can be suspended when storage space is insufficient, the number of tasks of allocating storage space for requests is reduced, the pressure of allocating storage space for all acquired requests is reduced, and the request process is avoided from frequently notifying the elimination thread, which causes the request thread to compete with the elimination thread and the GC thread for resources. This ensures the working efficiency of the elimination thread and the GC thread, thereby improving the efficiency of processing the target requests.

[0110] Example 4:

[0111] Based on the above embodiments, the present application also provides an electronic device, Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, it includes: a processor 71, a communication interface 72, a memory 73 and a communication bus 74, wherein the processor 71, the communication interface 72, and the memory 73 communicate with each other through the communication bus 74;

[0112] The memory 73 stores a computer program. When the program is executed by the processor 71, the processor 71 performs the following steps:

[0113] If it is determined that the target storage space cannot be allocated for the obtained target request, determining that the state of the target request is a sleep state and the sleep duration corresponding to the target request; wherein the sleep duration is used to indicate the length of time the target request is in the sleep state, and the sleep state is used to indicate suspending the task of allocating the target storage space for the target request;

[0114] If it is determined that the duration of the target request being in the sleep state reaches the sleep duration, the task of allocating the target storage space to the target request is resumed.

[0115] Furthermore, the processor 71 is specifically configured to determine that target storage space cannot be allocated for the obtained target request by one or more of the following methods:

[0116] The capacity of the free storage space is smaller than the capacity of the target storage space;

[0117] The storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information;

[0118] The obtained backend device status information is congested.

[0119] Furthermore, the processor 71 is specifically configured to determine the sleep time of the target request according to a ratio between the number of requests currently being processed and a preset number of read and write operations per unit time.

[0120] Furthermore, the processor 71 is also used to determine that after determining that the target storage space cannot be allocated for the obtained target request, determine that the state of the target request is a sleep state, and before the sleep time corresponding to the target request, determine that the switch state of the dynamic flow control switch is an open state.

[0121] Since the principle of solving the problem by the above electronic device is similar to that of the data processing method, the implementation of the above electronic device can refer to the embodiment of the method, and the repeated parts will be omitted.

[0122] The communication bus mentioned in the above-mentioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 72 is used for communication between the above-mentioned electronic device and other devices. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0123] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processing processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0124] During the process of allocating target storage space for target requests, if it is determined that target storage space cannot be allocated for the acquired target requests at present, the sleep time of the target request is determined, and the state of the target request is determined to be a sleep state, that is, the task of allocating target storage space for the target request is suspended. Subsequently, when it is determined that the target request has been in a sleep state for a certain length of time, the task of allocating target storage space for the target request is resumed. Since the task of allocating storage space for some acquired requests can be suspended when storage space is insufficient, the number of tasks of allocating storage space for requests is reduced, the pressure of allocating storage space for all acquired requests is reduced, and the request process is avoided from frequently notifying the elimination thread, which causes the request thread to compete with the elimination thread and the GC thread for resources. This ensures the working efficiency of the elimination thread and the GC thread, thereby improving the efficiency of processing the target requests.

[0125] Example 5:

[0126] Based on the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program executable by a processor. When the program runs on the processor, the processor implements the following steps:

[0127] If it is determined that the target storage space cannot be allocated for the obtained target request, determining that the state of the target request is a sleep state and the sleep duration corresponding to the target request; wherein the sleep duration is used to indicate the length of time the target request is in the sleep state, and the sleep state is used to indicate suspending the task of allocating the target storage space for the target request;

[0128] If it is determined that the duration of the target request being in the sleep state reaches the sleep duration, the task of allocating the target storage space to the target request is resumed.

[0129] Furthermore, the determining that target storage space cannot be allocated for the obtained target request includes one or more of the following:

[0130] The capacity of the free storage space is smaller than the capacity of the target storage space;

[0131] The storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information;

[0132] The obtained backend device status information is congested.

[0133] Further, determining the sleep time of the target request includes:

[0134] The sleep time of the target request is determined according to the ratio between the number of requests currently being processed and the preset number of read and write times per unit time.

[0135] Furthermore, after determining that the target storage space cannot be allocated for the acquired target request, and before determining that the state of the target request is a sleep state, and before determining the sleep duration corresponding to the target request, the method further includes:

[0136] Make sure the dynamic flow control switch is in the on state.

[0137] Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to that of the data processing method, the implementation of the above-mentioned computer-readable storage medium can refer to the embodiment of the method, and the repeated parts will be omitted.

[0138] During the process of allocating target storage space for target requests, if it is determined that target storage space cannot be allocated for the acquired target requests at present, the sleep time of the target request is determined, and the state of the target request is determined to be a sleep state, that is, the task of allocating target storage space for the target request is suspended. Subsequently, when it is determined that the target request has been in a sleep state for a certain length of time, the task of allocating target storage space for the target request is resumed. Since the task of allocating storage space for some acquired requests can be suspended when storage space is insufficient, the number of tasks of allocating storage space for requests is reduced, the pressure of allocating storage space for all acquired requests is reduced, and the request process is avoided from frequently notifying the elimination thread, which causes the request thread to compete with the elimination thread and the GC thread for resources. This ensures the working efficiency of the elimination thread and the GC thread, thereby improving the efficiency of processing the target requests.

[0139] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0140] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0143] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: The method comprises: If it is determined that the target storage space cannot be allocated for the obtained target request, determining that the state of the target request is a sleep state and the sleep duration corresponding to the target request; wherein the sleep duration is used to indicate the length of time the target request is in the sleep state, and the sleep state is used to indicate suspending the task of allocating the target storage space for the target request; If it is determined that the duration of the target request being in the sleep state reaches the sleep duration, resuming the task of allocating the target storage space to the target request; Determining the sleep time of the target request includes: According to the number of requests currently being processed and the preset number of read and write times per unit time, the average time for the electronic device to process each request is determined, and the average time is determined as the sleep duration; the average time is Where N is the number of requests currently being processed, and expected_throughput_iops is the preset number of reads and writes per unit time; or, Determine the duration value corresponding to the capacity of the current free storage space according to the pre-configured correspondence between the capacity of the free storage space and the duration value, and determine the duration value corresponding to the capacity of the current free storage space as the sleep duration; After determining that the target storage space cannot be allocated for the acquired target request, and before determining that the state of the target request is a sleep state and the sleep duration corresponding to the target request, the method further includes: Make sure the dynamic flow control switch is in the on state.

2. The method according to claim 1, characterized in that Determining that target storage space cannot be allocated for the obtained target request includes one or more of the following: The capacity of the free storage space is smaller than the capacity of the target storage space; The storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information; The obtained backend device status information is congested.

3. A data processing device, characterized in that: The device comprises: a first processing unit, configured to, if it is determined that the target storage space cannot be allocated for the obtained target request, determine that the state of the target request is a sleep state and a sleep duration corresponding to the target request; wherein the sleep duration is used to indicate a length of time the target request is in the sleep state, and the sleep state is used to indicate a suspension of a task of allocating the target storage space for the target request; a second processing unit, configured to resume the task of allocating the target storage space to the target request if it is determined that the duration of the target request being in the sleep state reaches the sleep duration; The first processing unit is specifically configured to determine the average time it takes for the electronic device to process each request based on the number of requests currently being processed and the preset number of read and write times per unit time, and to determine the average time as the sleep duration; the average time is Where N is the number of requests currently being processed, and expected_throughput_iops is the preset number of reads and writes per unit time; or, Determine the duration value corresponding to the capacity of the current free storage space according to the pre-configured correspondence between the capacity of the free storage space and the duration value, and determine the duration value corresponding to the capacity of the current free storage space as the sleep duration; The first processing unit is also used to determine that after determining that the target storage space cannot be allocated for the obtained target request, the state of the target request is determined to be a sleep state, and before the sleep time corresponding to the target request, determine that the switch state of the dynamic flow control switch is an open state.

4. The device according to claim 3, characterized in that The first processing unit is specifically configured to determine that a target storage space cannot be allocated for the obtained target request by one or more of the following methods: The capacity of the free storage space is smaller than the capacity of the target storage space; The storage space corresponding to the address information carried in the target request overlaps with the storage space corresponding to any unfinished flushed address information; The obtained backend device status information is congested.

5. An electronic device, characterized in that: The electronic device includes a processor, and the processor is configured to implement the steps of the data processing method according to any one of claims 1 to 2 when executing a computer program stored in a memory.

6. A computer-readable storage medium, characterized in that It stores a computer program, which, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 2.

Images