Data processing method and device

Abstract

The embodiments of the present specification provide a data processing method and device, wherein the data processing method includes: receiving a data processing request for a target project, and determining a target waiting time for waiting for the target data to be returned when it is determined that the target data returned by the target process based on the data processing request is empty; determining a current waiting time for waiting for the target data to be returned, and when the current waiting time is less than the target waiting time, allocating a processor corresponding to the execution of the data processing request to a data processing request for the next target project. This method will not occupy the processor for a long time due to a long waiting time for the target data, thereby achieving a higher utilization rate of the processor and improving the user experience.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular, to a data processing method. Background Art

[0002] In the process of data calling, using user-space file systems can reduce the complexity of development to reduce dependence on the underlying operating system kernel. However, due to the need for additional coordination between the operating system kernel and user-space (usually through event waiting and notification), user-space file systems usually incur more latency costs than running in the kernel and have lower performance. Part of the performance overhead is due to the existing waiting mechanism in FUSE (Filesystem in Userspace), which the user-space file system uses to coordinate with the kernel to obtain file system access requests, and the kernel uses this mechanism to wait for the request to complete. This waiting mechanism will cause the system to take a long time to access files and have low CPU utilization. Summary of the invention

[0003] In view of this, an embodiment of this specification provides a data processing method. One or more embodiments of this specification also relate to a data processing device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a data processing method is provided, including:

[0005] receiving a data processing request for a target project, and determining a target waiting time for waiting for returning the target data when it is determined that the target data returned by the target process based on the data processing request is empty;

[0006] A current waiting time for waiting to return the target data is determined, and when the current waiting time is less than the target waiting time, a processor corresponding to the data processing request is allocated to the data processing request of the next target item.

[0007] According to a second aspect of an embodiment of this specification, there is provided a data processing device, including:

[0008] A time determination module is configured to receive a data processing request of a target project, and determine a target waiting time for waiting to return the target data when it is determined that the target data returned by the target process based on the data processing request is empty;

[0009] The processing module is configured to determine a current waiting time for returning the target data, and when the current waiting time is less than the target waiting time, allocate the processor corresponding to the data processing request to the data processing request of the next target item.

[0010] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0011] Memory and processor;

[0012] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the processor implements the steps of the data processing method when executing the computer-executable instructions.

[0013] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of any one of the data processing methods are implemented.

[0014] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0015] An embodiment of the present specification receives a data processing request for a target project, and when it is determined that the target data returned by the target process based on the data processing request is empty, determines a target waiting time for waiting to return the target data; determines a current waiting time for waiting to return the target data, and when the current waiting time is less than the target waiting time, allocates a processor corresponding to the data processing request to execute the next data processing request for the target project.

[0016] Specifically, when the target process does not return the target data, the preset waiting time for waiting for the target data to be returned is determined, and if the preset waiting time is not exceeded, the processor corresponding to the data processing request is allocated to the data processing request of the next target project. This method will not occupy the processor for a long time due to the long waiting time for the target data, thereby achieving higher utilization of the processor and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a system structure diagram of a data processing method provided by an embodiment of this specification;

[0018] Figure 2 is a flow chart of a data processing method provided by an embodiment of this specification;

[0019] Figure 3 It is a flowchart of a process waiting process of a data processing method provided by an embodiment of this specification;

[0020] Figure 4 is a flow chart of a data processing method for calculating a target waiting time provided by an embodiment of the present specification;

[0021] Figure 5 It is a flow chart of an event waiting process of a data processing method provided by an embodiment of the present specification;

[0022] Figure 6 is a structural schematic diagram of a data processing device provided by an embodiment of this specification;

[0023] Figure 7 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0024] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0025] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] First, the terms involved in one or more embodiments of this specification are explained.

[0028] FUSE (Filesystem in Userspace): is a module in Linux used to mount certain network spaces, such as SSH, to the local file system.

[0029] User state: refers to the non-privileged state. In this state, the executed code is limited by hardware and cannot perform certain operations, such as writing to the storage space of other processes, to prevent security risks to the operating system.

[0030] The data processing method provided in the embodiment of this specification can estimate the time required to wait for the target data this time based on the historical waiting time for the target data when determining that the target data is empty, and determine multiple waiting methods for waiting for the required target data based on the estimated waiting time this time. Method one takes into account the low utilization rate of the processor but the high latency itself; method two takes into account the high utilization rate of the processor but the low latency itself. The above two methods require system self-adjustment, which can also be called adaptive waiting, to achieve a balance between pursuing low latency and avoiding wasting processors. Furthermore, this method dynamically judges the effectiveness of the above two methods to decide whether to use method two in a short period of time to improve the user experience with low latency, or to use method one directly.

[0031] Based on this, in this specification, a data processing method is provided. This specification also involves a data processing device, a computing device, a computer-readable storage medium and a computer program, which are described in detail one by one in the following embodiments.

[0032] Figure 1 A schematic diagram of the system structure of a data processing method provided according to an embodiment of the present specification is shown.

[0033] Figure 1 The FUSE kernel module can be understood as the processing process of the application. Figure 1The user space file system process in can be understood as a user state file system process. Specifically, after the application sends a file system call request to the file system processing code module of the FUSE kernel module, the file system processing code packages and encapsulates the file system call request (FUSE request), and places the processed FUSE request in the pending queue. When it is determined that there is target data that needs to be obtained from the user control file system, the FUSE request is placed in the processing queue. At the same time, the user space file system process determines whether there is target data that needs to be obtained in the processing queue, obtains the FUSE request from the device processing code module, and converts the data format of the FUSE request through the FUSE library, and then obtains the corresponding target data based on the FUSE interface, and formats it again through the FUSE library, and returns the processed target data to the FUSE kernel module, and the target data is returned to the application as the reply data of the FUSE request.

[0034] It should be noted that the process of the FUSE kernel module obtaining target data from the user space file system requires a certain amount of processing time, so the waiting time for this event is an important factor in determining the execution time of the entire task.

[0035] In actual applications, when an application makes a file call to a FUSE-based user control file system, the FUSE request is transmitted from the application thread to the user space file system thread through the FUSE kernel module. After the file system processes the request, the FUSE reply returns to the application thread in the opposite direction. In general, for synchronous FUSE requests, its completion involves two event waits in the kernel. One is that the user file system thread waits for a new FUSE request in the kernel; the other is that the application thread waits for a reply to its FUSE request in the kernel. The kernel event wait and subsequent notification take several microseconds to complete. For requests that can be quickly processed by the user space file system, the overhead of two event waits is high, and its end-to-end delay basically comes from the delay of the two event waits. The delay of the event wait method itself also masks the low latency advantage that high-performance storage devices (such as those based on persistent memory technology) can provide.

[0036] Based on this, a data processing method provided in an embodiment of this specification is intended to reduce the delay of the existing waiting scheme between the kernel and the user space file system, see Figure 2 , Figure 2 A flowchart of a data processing method provided in an embodiment of the present specification is shown, which specifically includes the following steps.

[0037] Step 202: Receive a data processing request for a target project, and when it is determined that the target data returned by the target process based on the data processing request is empty, determine a target waiting time for waiting for the target data to be returned.

[0038] Among them, the data processing request for the target project can be understood as a data query request, a data editing request, a data processing request, etc. for the data required by the target project.

[0039] In specific implementation, after the FUSE kernel module receives a data processing request, it will receive the target data returned by the target process based on the data processing request. After the FUSE kernel module obtains the corresponding target data, it can perform subsequent data processing operations accordingly. However, the process of receiving the target data returned by the target process requires a waiting time. In actual applications, this waiting time affects the efficiency of server data processing and also affects the user experience. Therefore, the waiting time for this time is the key consideration of the data processing method provided in the embodiments of this specification.

[0040] Based on this, after the FUSE kernel module receives a data processing request, when it is determined that the target data returned by the target process is empty, or in other words, when it is determined that the target process has not yet returned the corresponding target data, the FUSE kernel module can first determine the target waiting time for waiting for the return of the target data, and then realize the processing of subsequent target data.

[0041] Further, in the case where it is determined that the target data returned by the target process based on the data processing request is empty, before determining the target waiting time for waiting to return the target data, the method further includes:

[0042] The data processing request is encapsulated and processed, and the processed data processing request is placed in a queue to be processed.

[0043] In actual applications, after the FUSE kernel module receives the data processing request of the target project, the file system processing code in the FUSE kernel module packages and encapsulates the received data processing request to generate an encapsulated data processing request, which may include information such as data size, time of requesting data, and length of data, and places the encapsulated data processing request in a queue to be processed.

[0044] The data processing method provided in the embodiments of this specification places a data processing request in a queue to be processed, so that a subsequent target process can check whether there is required data in the queue to be processed.

[0045] Since the time when the target process returns the target data is uncertain, the processing process of the FUSE kernel module determines a target waiting time when it is not configured to be in a sleep state and has not received the target data; specifically, when it is determined that the target data returned by the target process based on the data processing request is empty, determining the target waiting time for waiting to return the target data includes:

[0046] When it is determined that the data processing request obtained by the target process from the pending queue is not configured as a sleep state and the target data returned by the target process based on the data processing request is empty, a target waiting time for waiting to return the target data is determined.

[0047] It should be noted that in the traditional data processing process, when it is determined that the target data returned by the target process has not been received, the data processing process can be put into a sleep state, that is, an event waiting state. When it is determined that the target data returned by the target process has been received, the data processing process will be awakened to continue the subsequent data processing process. For specific implementation steps, please refer to Figure 3 , Figure 3 A flowchart of the process waiting process of the data processing method provided in the embodiment of this specification is shown in FIG.

[0048] Step 302: The data processing process waits for target data.

[0049] Step 304: The data processing process determines whether the target data is received, and if so, executes step 308, and if not, executes step 306.

[0050] Step 306: The data processing process enters event waiting and continues to execute step 304.

[0051] Step 308: The data processing process is awakened and returns the target data.

[0052] Based on this, in the above embodiment, when the data processing process enters the sleep state, the corresponding waiting time for sleeping, waiting, and waking up will be generated. If the target data is received when the data processing process just enters the sleep state, the data processing process needs to be woken up before processing the data. The wake-up process will still take a certain amount of time, usually several microseconds, which will generate some unnecessary waiting time. Therefore, the data processing method provided in the embodiment of this specification can pre-set a target waiting time to reduce some unnecessary waiting time of the process.

[0053] In practical applications, when the processing process of the FUSE kernel module determines that the data processing request has not been configured as a sleep state, and the target data returned by the target process is empty, the target waiting time that the data processing process can wait for the target process to return the target data can be calculated. It should be noted that the target process can search in real time from the pending queue in the FUSE kernel module to see if there is a data processing request that needs to be executed. If there is a data processing request that the target process needs to process in the pending queue, the target process will obtain the data processing request from the pending queue, and the target process will format the data processing request through the FUSE library, and process the required target data based on the processed data processing request. Since the target process may take a long time to process the target data due to various reasons in the process of obtaining the target data, the time to return the target data to the FUSE kernel module will also be relatively long.

[0054] The data processing method provided in the embodiments of this specification reduces the data processing delay caused by waiting too long for the target data by determining the target waiting time for the data processing process to wait for the target data to be returned when the data processing process has not entered the sleep state and the returned target data is empty.

[0055] Furthermore, the preset target waiting time is determined based on which factors, and different target waiting times can be determined with reference to different requirements of the target project; specifically, the determination of the target waiting time for waiting to return the target data includes:

[0056] A historical waiting time and item attribute information of the target item are determined based on the target item, and a target waiting time of the target data is determined based on the historical waiting time and the item attribute information.

[0057] Among them, the historical waiting time of the target project can be understood as the historical waiting time of the data processing process during the historical data processing operations of the target project; the project attribute information can be understood as the basic attribute information such as the type of the target project and the complexity of processing. Based on the project attribute information, the processing requirements of the data processing request can be determined.

[0058] In actual applications, the data processing process can determine the target waiting time for the data processing process to wait for the target data based on the historical waiting time of the target project and the project attribute information. Figure 4 , Figure 4 A flow chart of calculating the target waiting time by the data processing method provided in the embodiment of this specification is shown.

[0059] Step 402: Determine whether the target waiting time has been calculated for this data processing process. If so, execute step 412; if not, execute step 404.

[0060] Step 404: Estimate the waiting time G required this time based on the historical waiting time of the data processing request; the waiting time P configured according to the project attribute information; the net overhead time D for event waiting of the data processing process.

[0061] Step 406: Determine whether to enter event waiting, that is, G < P + D. If so, execute Step 408; if not, execute Step 410.

[0062] Step 408: The data processing process determines that the waiting time P is the target waiting time.

[0063] Step 410: The data processing process determines that the target waiting time is 0.

[0064] Step 412: The data processing process returns the target waiting time.

[0065] For example, configure the busy waiting time (the waiting time configured according to the project attribute information) P to 10 microseconds, and the net overhead time for event waiting is 5 microseconds. If the historical target data arrival time G for the previous time is 20 microseconds, it can also be considered that the target data arrival time this time may also be 20 microseconds. Therefore, under the prediction of the waiting time this time, it is meaningless to busy wait for 10 microseconds, and event waiting can be directly performed; if the historical target data arrival time G for the previous time is 12 microseconds, and the event waiting itself consumes 5 microseconds of overhead, therefore, the target data may arrive between 7 and 12 microseconds, and thus it is valuable to busy wait for 10 microseconds, and the data processing process can be set to the busy waiting stage.

[0066] It should be noted that if G >= P + D, it may be meaningless to configure the data processing process for busy waiting, and the busy waiting time should be set to 0. Otherwise, the busy waiting time required this time can be set to the configured busy waiting time P, where the configured busy waiting time can also be configured with different waiting times according to the deployment scenario, system characteristics, required data type, etc.

[0067] The embodiment of this specification determines the target waiting time of the target data through the historical waiting time of the target project and the project attribute information, and can accurately preset the adaptive process waiting time of the target project, which is convenient for subsequently determining whether the data processing process needs to enter the sleep state through the target waiting time to improve the processing efficiency of the CPU.

[0068] In addition, for the data processing method provided by another embodiment of this specification, the target waiting time of the target can also be determined based on the data type of the target data; specifically, in the case where it is determined that the target data returned by the target process based on the data processing request is empty, determining the target waiting time for waiting to return the target data includes:

[0069] When it is determined that the target data returned by the target process based on the data processing request is empty, the data type of the target data is determined based on the data attribute information carried in the data processing request, and the target waiting time for waiting for the target data to be returned is determined based on the data type.

[0070] During specific implementation, when the data processing process determines that the target data returned by the target process based on the data processing request is empty, the data processing process determines the data type of the target data based on the data attribute information carried in the data processing request, and determines whether the target data required by the data processing request is a specific data type. In actual applications, different waiting times can be configured based on different data types. The waiting time set according to different scenarios facilitates the subsequent precise control of the maximum waiting time of the data processing request, thereby reducing unnecessary delays and improving CPU processing efficiency.

[0071] It should be noted that the method for determining the target waiting time provided in the above two embodiments is not limited in this specification, and the target waiting time may also be determined in different ways, such as by considering other deployment scenarios and system characteristics.

[0072] The data processing method provided in the embodiments of this specification can also determine different target waiting times according to the data type, and thus can accurately preset the adaptive process waiting time of the target project, so as to facilitate the subsequent determination of whether the data processing process needs to enter a sleep state through the target waiting time, so as to improve the processing efficiency of the CPU.

[0073] Step 204: Determine the current waiting time for returning the target data, and when the current waiting time is less than the target waiting time, allocate the processor corresponding to the data processing request to the data processing request of the next target item.

[0074] In specific implementation, after determining the target waiting time that the data processing process can wait, it is also necessary to determine the current waiting time of the data processing process. When it is determined that the current waiting time is less than the target waiting time, the processor corresponding to the data processing request executed by the data processing process is allocated to the next data processing request for executing the target project.

[0075] In actual applications, allocating the processor corresponding to the execution of the data processing request to the data processing request of the next target item can be understood as calling the operating system kernel scheduling (such as schedule), which means asking the kernel scheduler to select the next process / thread to run. When using this method, if the thread finds that there is no required target data, it will temporarily give up the CPU to other threads for execution. This is a certain degree of cyclic polling of data availability, and it will not continue to occupy CPU resources. If the data arrives within the thread cycle, the thread can process the data faster and avoid the multi-microsecond delay required to enter sleep and then be awakened. Similarly, it usually only makes sense when the data demand thread and the providing thread run on different CPUs. The reason is that if the data demand thread and the providing thread are scheduled to run on the same CPU in succession, the end-to-end delay will be relatively long.

[0076] The embodiments of this specification provide a detailed description of the above-mentioned data processing method, indicating that when the current waiting time of the data processing process is less than the target waiting time, the processor executing the data processing request can be allocated to the next thread for use to avoid unnecessary occupation of CPU resources, thereby improving CPU utilization.

[0077] Furthermore, in another embodiment of the present specification, the data processing method provided by the data processing process may be configured to be in a sleep state when the current waiting time of the data processing process is greater than or equal to the target waiting time; specifically, after determining the current waiting time for waiting to return the target data, the method further includes:

[0078] When the current waiting time is greater than or equal to the target waiting time, the data processing request is configured as a sleep state.

[0079] In actual applications, when the current waiting time of the data processing process waiting for the target data to be returned is greater than or equal to the target waiting time, it indicates that the waiting time is long, and the processing process corresponding to the data processing request can be configured to sleep to reduce the waste of processing resources. This method can also be called the waiting queue method (such as thread sleep and wake-up based on wait_queue_head_t). When the target data can only be reached after a considerable period of time, the sleep waiting method is a better choice. For example, after a certain period of time, when the process has not waited for the target data using other methods, the process can switch to the waiting queue method, that is, sleep on the corresponding event until the target data arrives and is woken up by the process.

[0080] The data processing method provided in the embodiments of the present specification is to reconfigure the data processing request to the sleep state when the current waiting time is greater than or equal to the target waiting time. This method does not enter the sleep state when the waiting starts, but configures the data processing request to the sleep state only when subsequent conditions are met. This can avoid receiving the target data when the process just enters the sleep state, and waste extra time in the process of waking up the process, thereby reducing the waiting time of the process.

[0081] After the data processing process is configured to be in a sleep state, the data processing process will not be awakened to process the target data until the target process returns the target data; specifically, after the data processing request is configured to be in a sleep state, the method further includes:

[0082] In a case where it is determined that the target process returns the target data based on the data processing request, the data processing request is configured to be in a wake-up state, and the data processing request is processed based on the target data.

[0083] In actual applications, the data processing process will repeatedly check whether the target data has been received and returned. When it is determined that the target process has returned the target data based on the data processing request, the data processing process will be configured to wake up and continue with the subsequent process of processing the data processing request. The data processing process processes the target data, including returning the target data directly to the application, and performing other processing on the received target data. The embodiments of this specification do not impose any limitations on this.

[0084] The data processing method provided in the embodiments of this specification can wake up the data processing request from the sleep state and execute subsequent data processing requests when the data processing process determines that the target process has returned the target data.

[0085] In addition, during the waiting process of the data processing process, when the processor corresponding to the current data processing request is allocated to the next data processing request, it is necessary to continuously check whether the data processing process has received the target data, so as to execute the subsequent process; specifically, after allocating the processor corresponding to the data processing request to the data processing request of the next target item, it also includes:

[0086] determining whether the target process returns target data based on the data processing request,

[0087] If so, processing the data processing request based on the target data;

[0088] If not, the target waiting time for waiting for the target data to be returned is re-determined.

[0089] In actual applications, after the data processing process has allocated the processor corresponding to the data processing request to the data processing request of the next target item, it is still necessary to continuously check whether the target data is returned from the target process. After determining that the returned target data has been received, the data processing process continues. After the returned target data has not been received, the target waiting time can be recalculated, and the recalculated target waiting time may be adjusted to adapt to the current process execution status.

[0090] The data processing method provided in the embodiment of this specification can quickly re-determine the new waiting time of the target data by continuously polling in the data processing process of the FUSE kernel module to determine whether the target data is received, thereby realizing a method for dynamically adjusting the waiting time of the data processing process and balancing the CPU usage.

[0091] The following combination Figure 5 , Figure 5 A flow chart of an event waiting process of a data processing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0092] Step 502: The data processing process waits for target data.

[0093] Step 504: The data processing process determines whether the target process returns the target data. If so, execute step 514; if not, execute step 506.

[0094] Step 506: The data processing process calculates the target waiting time.

[0095] Step 508: The data processing process determines whether the current waiting time is less than the target waiting time. If so, step 510 is executed; if not, step 512 is executed.

[0096] Step 510: The data processing process allocates the processor corresponding to the data processing request to the next data processing request.

[0097] Step 512: The data processing process is configured to be in a sleep state and enters an event waiting phase.

[0098] Step 514: The data processing process records the current waiting time.

[0099] Step 516: The data processing process returns the target data.

[0100] In practical applications, in order to avoid the overhead of event waiting, this embodiment introduces busy waiting in event waiting, which is also called the initial stage of busy polling. It should be noted that the busy waiting can be understood as the time waiting process of the above embodiment. If the condition sought by the process has been met during the busy waiting period, the process will continue to execute the subsequent logic, so there will be no cost of event waiting. But if the condition sought during the busy waiting period cannot be met, the method will fall back to the original event waiting, so the method can be called busy event waiting.

[0101] Furthermore, when the kernel runs the device processing code, the user space file system thread of the method will continuously check whether there are new pending FUSE requests in a busy loop. If a new FUSE request arrives during the busy loop, it will be discovered by the user space file system process and processed immediately; if a new FUSE request arrives during the busy loop, the process will fall back to wait for an event bound to a pending queue, and the process waiting for the event will be suspended by the operating system kernel. When the next FUSE request arrives at the pending queue, the kernel will wake up the user space file system thread waiting on the corresponding event. After the user space file system processes a FUSE request, it will send the corresponding FUSE reply to the kernel. The kernel's device processing code then sends the reply back to the file system processing code in the kernel in the same way. If the user-space file system can process the request quickly enough, the file system processing code may be in a busy loop waiting for the FUSE reply. In this idealized case, both original event waits are replaced by busy loops, and the end-to-end latency observed by the application process (data processing process) can be as low as 3-4 microseconds. In contrast, if the two original event waits are used for both FUSE requests and replies, the end-to-end latency will be 8-9 microseconds. Of course, the specific latency value will vary depending on the system configuration, but busy-event wait can greatly reduce the latency of event wait.

[0102] The effectiveness of busy waiting depends to a certain extent on whether the application process and the user space file system process are running on the same or different CPUs. When setting the busy waiting period, it is necessary to balance the request delay and CPU utilization to achieve the ideal IO throughput. The busy waiting time should be set according to the performance characteristics of the file system and the underlying storage system, but as a reference value, for systems based on persistent memory technology and systems based on SSDs, a busy waiting time of about 10 microseconds can achieve a good balance. In order to further reduce CPU consumption and potentially increase throughput, the busy waiting stage can also be dynamically adjusted according to the delay observed when the system runs. It should be noted that if the actual time required to serve the FUSE request exceeds the busy waiting stage, busy waiting will only waste CPU resources. Therefore, the embodiment of this specification further improves this. In each method of waiting for data, the system records the delay from the beginning of the need for data to the arrival of data last time. Then, each time you wait for data, first compare the delay observed last time with the configured busy waiting time plus the net overhead of an event waiting (as a reference value, the net overhead of event waiting will reach 5 microseconds). If the value of the former is larger, busy waiting is expected to have no effect, and the system can skip the busy waiting stage and do event waiting directly. Otherwise, busy waiting is re-enabled. This busy event waiting method with the ability to dynamically enable and disable the busy waiting stage is called adaptive busy-event wait, also known as adaptive waiting.

[0103] In summary, the data processing method provided in the embodiments of this specification uses an adaptive waiting scheme to reduce the delay of kernel event waiting between the kernel and the FUSE-based user space file system. This method introduces an initial busy waiting stage before the event waiting, and predicts the effectiveness of the busy waiting period based on the observed historical data delay value, so as to dynamically enable or disable the busy waiting period.

[0104] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises:

[0105] The time determination module 602 is configured to receive a data processing request of a target project, and determine a target waiting time for waiting to return the target data when it is determined that the target data returned by the target process based on the data processing request is empty;

[0106] The processing module 604 is configured to determine a current waiting time for returning the target data, and when the current waiting time is less than the target waiting time, allocate the processor corresponding to the data processing request to the data processing request of the next target item.

[0107] Optionally, the device further comprises:

[0108] The encapsulation module is configured to encapsulate the data processing request and place the processed data processing request in a queue to be processed.

[0109] Optionally, the time determination module 602 is further configured to:

[0110] When it is determined that the data processing request obtained by the target process from the pending queue is not configured as a sleep state and the target data returned by the target process based on the data processing request is empty, a target waiting time for waiting to return the target data is determined.

[0111] Optionally, the time determination module 602 is further configured to:

[0112] A historical waiting time and item attribute information of the target item are determined based on the target item, and a target waiting time of the target data is determined based on the historical waiting time and the item attribute information.

[0113] Optionally, the device further comprises:

[0114] The configuration module is configured to configure the data processing request to a sleep state when the current waiting time is greater than or equal to the target waiting time.

[0115] Optionally, the processing module 604 is further configured to:

[0116] In a case where it is determined that the target process returns the target data based on the data processing request, the data processing request is configured to be in a wake-up state, and the data processing request is processed based on the target data.

[0117] Optionally, the device further comprises:

[0118] a determination module, configured to determine whether the target process returns target data based on the data processing request,

[0119] If so, processing the data processing request based on the target data;

[0120] If not, the target waiting time for waiting for the target data to be returned is re-determined.

[0121] Optionally, the time determination module 602 is further configured to:

[0122] When it is determined that the target data returned by the target process based on the data processing request is empty, the data type of the target data is determined based on the data attribute information carried in the data processing request, and the target waiting time for waiting for the target data to be returned is determined based on the data type.

[0123] The data processing device provided in the embodiments of the present specification determines the preset waiting time for waiting for the target data to be returned when the target process does not return the target data, and allocates the processor corresponding to the execution of the data processing request to the occupancy of the data processing request of the next target project if the preset waiting time is not exceeded. This method will not occupy the processor for a long time due to the long waiting time for the target data, thereby achieving a higher utilization rate of the processor and improving the user experience.

[0124] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0125] Figure 7 The block diagram of a computing device 700 according to an embodiment of the present specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0126] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0127] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0128] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 700 may also be a mobile or stationary server.

[0129] The processor 720 is used to execute the following computer executable instructions, which implement the steps of the above data processing method when executed by the processor.

[0130] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data processing method.

[0131] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned data processing method when executed by a processor.

[0132] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data processing method.

[0133] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0134] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above data processing method.

[0135] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0136] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0137] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0138] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0139] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Receiving a data processing request for a target project, and when it is determined that the target data returned by the target process based on the data processing request is empty, determining a target waiting time for waiting to return the target data, where the target waiting time is the time that the data processing process can wait for the target process to return the target data. The determining the target waiting time for waiting to return the target data includes: estimating the time G that needs to be waited this time according to the historical waiting time of the data processing request; the waiting time P configured according to the attribute information of the target project; the net overhead time D for event waiting of the data processing process; if G < P + D, determining the waiting time P as the target waiting time; if G >= P + D, determining the target waiting time as 0; Determining the current waiting time for waiting to return the target data, and when the current waiting time is less than the target waiting time, allocating the processor corresponding to the data processing request to the data processing request of the next target project.

2. The data processing method according to claim 1, before determining the target waiting time for waiting to return the target data when it is determined that the target data returned by the target process based on the data processing request is empty, further comprising: Encapsulating the data processing request, and placing the processed data processing request in a to-be-processed queue.

3. The data processing method according to claim 2, the determining the target waiting time for waiting to return the target data when it is determined that the target data returned by the target process based on the data processing request is empty, includes: When it is determined that the data processing request obtained by the target process from the to-be-processed queue is not configured to be in a sleep state and the target data returned by the target process based on the data processing request is empty, determining the target waiting time for waiting to return the target data.

4. The data processing method according to claim 1, after determining the current waiting time for waiting to return the target data, further comprising: When the current waiting time is greater than or equal to the target waiting time, configuring the data processing request to be in a sleep state.

5. The data processing method according to claim 4, after configuring the data processing request to be in a sleep state, further comprising: When it is determined that the target process returns the target data based on the data processing request, configuring the data processing request to be in a wake-up state, and processing the data processing request based on the target data.

6. The data processing method according to claim 1, after allocating the processor corresponding to the data processing request to the data processing request of the next target project, further comprising: Determining whether the target process returns target data based on the data processing request, If so, processing the data processing request based on the target data; If not, re-determining the target waiting time for waiting to return the target data.

7. The data processing method according to claim 1, wherein in the case where it is determined that the target data returned by the target process based on the data processing request is empty, determining a target waiting time for waiting for the return of the target data includes: In the case where it is determined that the target data returned by the target process based on the data processing request is empty, determining the data type of the target data based on the data attribute information carried in the data processing request, and determining the target waiting time for waiting for the return of the target data based on the data type.

8. A data processing device, comprising: A time determination module, configured to receive a data processing request for a target project, and in the case where it is determined that the target data returned by the target process based on the data processing request is empty, determine a target waiting time for waiting for the return of the target data, wherein the target waiting time is the time for which the data processing process can wait for the target process to return the target data, and the time determination module is further configured to: estimate the time G to be waited for this time according to the historical waiting time of the data processing request; the waiting time P configured according to the attribute information of the target project; the net overhead time D for event waiting of the data processing process; if G < P + D, determine the waiting time P as the target waiting time; if G >= P + D, determine the target waiting time as 0; A processing module, configured to determine the current waiting time for waiting for the return of the target data, and in the case where the current waiting time is less than the target waiting time, allocate the processor corresponding to the execution of the data processing request to the data processing request of the next target project.

9. A computing device, comprising: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the data processing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 7 are implemented.

11. A computer program, wherein: When the computer program is executed on a computer, the computer is made to execute the steps of the data processing method according to any one of claims 1 to 7.

Images