Business data processing method, device and electronic equipment

Abstract

The present invention provides a method, device and electronic device for processing business data, which obtains a data processing request corresponding to business data; processes the data processing request through multiple sub-request processing modules corresponding to a preset request processing method, and determines a target data storage server among multiple data storage servers; the sub-request processing module is used to: simultaneously process multiple data processing requests according to the preset sub-request processing method; and send processing parameters of at least part of the data to the target data storage server, so that the target data storage server performs data processing on at least part of the data based on the processing parameters. In this method, the client performs pipeline processing on the data processing request by setting multiple sub-request processing modules, and can process multiple data processing requests at the same time, thereby improving the processing speed of the data processing request and reducing the request processing pressure of the system.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and electronic equipment for processing business data. Background Art

[0002] When online games have many scenes or more complex gameplay, cloud gaming is usually used to run the game. In the cloud gaming scenario, the game's running files are usually stored in a distributed system. In the process of playing games, players usually trigger data read or write operations of the cloud gaming system. In related technologies, data read or write operations triggered by multiple players usually wait for execution synchronously in the form of a queue in a distributed storage system. Usually, the client in the distributed storage system processes the data read or write operation through the main thread and sends it to the data storage server, which then performs the data read or write operation. This method causes the input and output pressure of the devices in the distributed storage system to be high, occupies more central processing unit resources, and responds slowly to data read or write operations, resulting in a poor gaming experience for players. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a method, device and electronic device for processing business data, so as to improve the processing speed of data processing requests and reduce the request processing pressure of the system.

[0004] In the first aspect, an embodiment of the present invention provides a method for processing business data, wherein the business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system also includes a client; the method is applied to the client; the method includes: obtaining a data processing request corresponding to the business data; the data processing request includes processing parameters for at least part of the data in the business data; processing the data processing request through a plurality of sub-request processing modules corresponding to a preset request processing method, and determining a target data storage server among a plurality of data storage servers; the sub-request processing module is used to: process a plurality of data processing requests simultaneously according to a preset sub-request processing method; the target data storage server is used to: store at least part of the data in the business data corresponding to the data processing request; and send the processing parameters for at least part of the data to the target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters.

[0005] In a second aspect, an embodiment of the present invention provides a method for processing business data, wherein the business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system also includes a client; the method is applied to a first server among the plurality of data storage servers; the method includes: obtaining processing parameters of at least part of the data in the business data corresponding to a preset data processing request; the processing parameters are sent after the client processes the data processing request through a plurality of sub-request processing modules corresponding to a preset request processing method; the data processing request includes processing parameters of at least part of the data in the business data; the sub-request processing module is used to: simultaneously process a plurality of data processing requests according to a preset sub-request processing method; at least part of the data corresponding to the data processing request is stored in the first server; at least part of the data is processed based on the processing parameters by the sub-data processing module corresponding to the preset data processing method to obtain a processing result corresponding to the data processing request; the sub-data processing module is used to: simultaneously process at least part of the data stored in the first server according to the preset sub-data processing method.

[0006] In the third aspect, an embodiment of the present invention provides a device for processing business data, wherein the business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system also includes a client; the device is arranged on the client; the device includes: a request acquisition module, used to obtain a data processing request corresponding to the business data; the data processing request includes processing parameters of at least part of the data in the business data; a request processing module, used to process the data processing request through a plurality of sub-request processing modules corresponding to a preset request processing method, and determine a target data storage server among a plurality of data storage servers; the sub-request processing module is used to: process a plurality of data processing requests simultaneously according to a preset sub-request processing method; the target data storage server is used to: store at least part of the data in the business data corresponding to the data processing request; a processing parameter sending module, used to send the processing parameters of at least part of the data to the target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters.

[0007] In a fourth aspect, an embodiment of the present invention provides a device for processing business data, wherein the business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system also includes a client; the device is arranged on a first server among the plurality of data storage servers; the device includes: a processing parameter acquisition module, used to acquire processing parameters of at least part of the data in the business data corresponding to a preset data processing request; the processing parameters are sent after the client processes the data processing request through a plurality of sub-request processing modules corresponding to a preset request processing method; the data processing request includes processing parameters of at least part of the data in the business data; the sub-request processing module is used to: simultaneously process a plurality of data processing requests according to a preset sub-request processing method; at least part of the data corresponding to the data processing request is stored on the first server; a data processing module, used to process at least part of the data based on the processing parameters through a sub-data processing module corresponding to the preset data processing method, and obtain a processing result corresponding to the data processing request; the sub-data processing module is used to: simultaneously process at least part of the data stored in the first server according to the preset sub-data processing method.

[0008] In a fifth aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned business data processing method.

[0009] In a sixth aspect, an embodiment of the present invention provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned business data processing method.

[0010] The embodiments of the present invention bring the following beneficial effects:

[0011] The above-mentioned method, device and electronic device for processing business data obtain a data processing request corresponding to the business data; process the data processing request through multiple sub-request processing modules corresponding to the preset request processing method, and determine the target data storage server among multiple data storage servers; the sub-request processing module is used to: process multiple data processing requests simultaneously according to the preset sub-request processing method; send the processing parameters of at least part of the data to the target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters. In this method, the client performs pipeline processing on the data processing request by setting multiple sub-request processing modules, and can process multiple data processing requests at the same time, thereby improving the processing speed of the data processing request and reducing the request processing pressure of the system.

[0012] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0013] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A schematic diagram of the data storage relationship between a cloud gaming system and a distributed storage system provided by an embodiment of the present invention;

[0016] Figure 2 A schematic diagram of the structure of an MFS system provided by an embodiment of the present invention;

[0017] Figure 3 A flowchart of responding to the data read and write request in the MFS system provided by an embodiment of the present invention;

[0018] Figure 4 A flowchart of a method for processing business data provided by an embodiment of the present invention;

[0019] Figure 5 A flowchart of another method for processing business data provided by an embodiment of the present invention;

[0020] Figure 6 A schematic diagram of the structure of a device for processing business data provided by an embodiment of the present invention;

[0021] Figure 7 A schematic diagram of the structure of another device for processing business data provided by an embodiment of the present invention;

[0022] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] Cloud games usually require a large number of games to be pre-installed. During the game update process and the user's access to cloud games (also called "cloud game instances"), multiple data read and write requests that need to access the same storage data are often generated. This situation requires that the cloud game data storage capacity must be large, can be managed uniformly, and have high read and write performance. The characteristics of distributed storage naturally match this demand.

[0025] Figure 1 The following is a diagram of the data storage relationship between the cloud gaming system and the distributed storage system. Among them, cloud gaming uses a virtual machine storage method, specifically an open source system virtualization module, Kernel-based VirtualMachine (referred to as "KVM" virtual machine). The distributed storage system uses the MooseFS (referred to as "MFS") distributed storage system. Correspondingly, the data corresponding to the disk in the virtual machine is stored in the qcow2 format of the virtual operating system emulator (Quick EMUlator, QEMU). The file is stored in the MFS system as a normal file. A virtual machine can have multiple disks, and there may be multiple virtual machine instances in a cluster corresponding to a cloud game. For larger-scale cloud gaming clusters, storage is required to improve higher performance. The performance of the storage directly affects the IO performance of the virtual machine, thereby affecting the gaming experience.

[0026] The above MFS system mainly includes client, management server (master server), data storage server (chunk servers), such as Figure 2 .

[0027] The client uses the Filesystem in Userspace (FUSE) to behave like a local file system and provide file system functions to the outside world.

[0028] In MFS, files are divided into fixed-size chunks (also called "shards"), each of which is fixed at 64MB. The data storage server stores these shard data in the local Linux file system. A shard data can be stored in multiple data storage servers, which constitutes a data copy to improve data availability.

[0029] The management server maintains the metadata of the file system, including the namespace (namespace, i.e. the file tree in a regular file system), access control information, etc. There is only one management server.

[0030] Taking the read operation as an example, the following briefly introduces the general process of responding to a data read / write request in the MFS system after the cloud game initiates a data read / write request. Figure 3 .

[0031] 1. The cloud game sends a data read request. After the data read request is processed by the client of the user space file system (mfsmount), the corresponding processing function registered by the MFS system is called. The read operation is mfs_read.

[0032] 2. mfs_read converts the offset and other information from the data read request into shard information, splits the request (such as across multiple shards), generates pre-read requests, etc., and then puts these requests into the read request queue. At this time, the request is still in the FUSE thread synchronization waiting process.

[0033] 3. The read worker class that performs read operations takes these requests out of the read request queue.

[0034] 4. Get the shard distribution (i.e. the data storage server where the shard is located) from the cache. If the cache does not have this information, get it from the management server and cache it.

[0035] 5. The main thread of the data storage server receives the read request and then queues the request.

[0036] 6. The io worker class that performs read and write operations calls hdd_read (that is, the actual shard data) to read the data and return.

[0037] 7. The mfsmount client ends the read request after receiving the data returned by the data storage server.

[0038] In the above method, since the client's data read / write (Input / Output, referred to as "IO") request is in a synchronous waiting state after entering the queue, the concurrency is limited by the number of processing threads of the libfuse library data read and write operations. At the same time, there is serious competition for IO locks. Under high IO pressure conditions, the central processing unit occupancy is high, which leads to a reduction in the central processing unit available for cloud games, resulting in poor game performance and a decline in user experience.

[0039] Since all request events on the data storage server are located in the main thread, and the data storage server needs to provide services to multiple clients. When there are many cloud game instances, there may be a delay in the data storage server main thread accepting requests, resulting in poor performance. At the same time, after receiving the IO request, the data storage server uses synchronous thread processing for each IO. Under high pressure, IO consumes a large number of threads, the system resource overhead is large, and there is serious resource competition between threads. Therefore, under the current client and data storage server architecture, it is difficult to support a large-scale cloud game cluster.

[0040] Based on this, an embodiment of the present invention provides a method, device and electronic device for processing business data. This technology can be applied to the processing of business data in various distributed storage systems.

[0041] See also Figure 4 First, a method for processing business data provided by an embodiment of the present invention is introduced. The business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system also includes a client; the method is applied to the client; the method includes the following steps:

[0042] Step S402, obtaining a data processing request corresponding to the business data; the data processing request includes processing parameters of at least part of the data in the business data.

[0043] The above data processing request can be sent by the player to the game server through the terminal device. The game server then parses the game-related operation request sent by the server and generates a data processing request and sends it to the distributed storage system.

[0044] When the above-mentioned business data is stored in the data storage server, it is usually stored in the form of multiple data slices. All data slices of the business data can be stored in multiple data storage servers, or part of the data slices can be stored in each data storage server according to certain storage rules. The storage information of the data slices in the data storage server is usually stored in the management server. When the client receives a data processing request, it usually obtains the storage information of the data slices required in the data processing request from the management server, and caches the storage information in the client.

[0045] The data shard required for the data processing request is usually determined based on at least part of the business data, that is, the data shard storing at least part of the data is the data shard required for the data processing request.

[0046] The above data processing request usually includes a read operation or a write operation, or both. Correspondingly, the processing parameters may also include a read position, a read data quantity, etc. corresponding to the read operation, or a write position, a write content, etc. corresponding to the write operation.

[0047] Step S404, processing the data processing request through multiple sub-request processing modules corresponding to the preset request processing method, and determining the target data storage server among the multiple data storage servers; the sub-request processing module is used to: process multiple data processing requests at the same time according to the preset sub-request processing method; the target data storage server is used to: store at least part of the data in the business data corresponding to the data processing request.

[0048] The above-mentioned preset request processing methods can be divided into a processing method for read operations and a processing method for write operations. Then the processing method for read operations is divided into multiple processing steps, and a sub-request processing module is established for each processing step. The sub-request processing method corresponding to the sub-request processing module can implement the corresponding processing steps. For example, the client's request for data processing can generally include steps such as decapsulation, determining the data shards corresponding to the data processing request, and determining the data storage server that stores the data shards. The above-mentioned sub-request processing module can be used to implement the above steps. Similarly, a corresponding sub-request processing module can also be established for the processing method for write operations.

[0049] The method of establishing a sub-request processing module is usually to establish a thread pool and set multiple threads in the thread pool to process the data processing request accordingly. In the specific implementation process, a thread pool can be established for each sub-request processing module, or one thread pool can correspond to multiple sub-request processing modules.

[0050] When a data processing request is received, a currently idle thread can be called from the thread pool corresponding to the first sub-request processing module to process the data processing request in accordance with the processing order of multiple sub-request processing modules. Then the processing results are passed to other sub-request processing modules in turn, forming a processing method similar to an assembly line. Since the thread pool includes multiple threads, each sub-request processing module can process multiple data processing requests at the same time, thereby improving the processing efficiency of data processing requests.

[0051] Step S406: sending processing parameters of at least part of the data to a target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters.

[0052] When sending the processing parameters of at least part of the data to the target data storage server, if there are multiple target data storage servers, that is, the data slices corresponding to the data processing request are stored in different data storage servers, a corresponding data read or write request can be generated based on the processing parameters corresponding to each target data storage server, and then the request can be sent to the target data storage server.

[0053] The above process of generating corresponding data read or write requests based on the processing parameters corresponding to each target data storage server and sending the requests to the target data storage server can also establish multiple sub-communication modules for sending processing parameters in steps, which can also improve the processing efficiency of data processing requests.

[0054] The above-mentioned method for processing business data obtains a data processing request corresponding to the business data; processes the data processing request through multiple sub-request processing modules corresponding to the preset request processing method, and determines the target data storage server among multiple data storage servers; the sub-request processing module is used to: process multiple data processing requests simultaneously according to the preset sub-request processing method; send the processing parameters of at least part of the data to the target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters. In this method, the client performs pipeline processing on the data processing request by setting multiple sub-request processing modules, and can process multiple data processing requests at the same time, thereby improving the processing speed of the data processing request and reducing the request processing pressure of the system.

[0055] The following embodiment provides an implementation method for processing a data processing request through multiple sub-request processing modules corresponding to a preset request processing method.

[0056] The above-mentioned sub-request processing module includes multiple first candidate threads, which can be regarded as placed in a "thread pool". For each sub-request processing module, a first target thread can be determined from multiple first candidate threads, and the first target thread can perform a first process on the data processing request. Corresponding to the process of data request processing, the first process can be a request segmentation process, a data analysis process, or a data storage server search process corresponding to at least part of the data. These processes are implemented in sequence, and after being processed by the first target thread of a sub-request processing module, the processing result is passed to the first target thread of the next sub-request processing module.

[0057] In the process of determining the first target thread from multiple first candidate threads, it is usually necessary to allocate threads based on characteristic parameters of the data processing request and a preset load balancing algorithm. The above characteristic parameters can be multiple, such as storage information of at least part of the data in the data processing request, sending time and sending address of the data processing request, etc., to achieve load balancing of thread allocation.

[0058] The following embodiment provides an implementation method for sending processing parameters of at least part of the data to a target data storage server.

[0059] When sending the processing parameters of at least part of the data to the target data storage server, the above-mentioned pipeline processing method can also be used. In specific implementation, the processing parameters of at least part of the data can be processed by multiple first sub-communication modules corresponding to the preset data sending method, and the processed processing parameters of at least part of the data can be sent to the target data storage server; wherein the first sub-communication module is used to: simultaneously process multiple processing parameters of at least part of the data according to the preset sub-communication method.

[0060] Specifically, a corresponding thread pool may be established for each first sub-communication module, and each thread pool includes a plurality of second candidate threads. In the process of processing the processing parameters of at least part of the data through the first sub-communication module, it is necessary to determine the second target thread from the plurality of second candidate threads; and then perform a second processing on the processing parameters of at least part of the data through the second target thread. Corresponding to the data sending process, the second processing may be a packaging processing or a data sending processing or other processing methods.

[0061] When determining the second target thread from multiple second candidate threads, it is usually necessary to obtain the load information of the target data storage server and the delay information between the client and the target data storage server; then based on the load information of the target data storage server and the delay information between the client and the target data storage server, determine the second target thread from the multiple second candidate threads. When there are multiple target data storage servers and there is redundancy, a target data storage server that can respond quickly can be selected based on the load information of the target data storage server and the above-mentioned delay information.

[0062] The embodiment of the present invention also provides another method for processing business data. Figure 5 The business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system also includes a client; the method is applied to a first server among the plurality of data storage servers; Figure 5 , the method comprises the following steps:

[0063] Step S502, obtaining processing parameters of at least part of the data in the business data corresponding to the preset data processing request; the processing parameters are sent after the client processes the data processing request through multiple sub-request processing modules corresponding to the preset request processing method; the data processing request includes processing parameters of at least part of the data in the business data; the sub-request processing module is used to: process multiple data processing requests simultaneously according to the preset sub-request processing method; at least part of the data corresponding to the data processing request is stored in the first server.

[0064] The processing parameters of at least part of the above data are as follows: Figure 4 At least part of the data is sent to the first server. Figure 4 At least part of the data in the method may be the same, or may be part of the at least part of the data mentioned above, which is related to the storage method of the business data and the specific content of at least part of the data corresponding to the data processing request.

[0065] The processing parameters received from the client are usually packaged and compressed based on the communication protocol, communication mode, etc. Therefore, after receiving the processing parameters, certain processing is also required to obtain the processing parameters that the data storage server can use to process the data. In the process of obtaining the processing parameters of at least part of the data corresponding to the preset data processing request in the business data, the above-mentioned pipeline processing method can also be used, that is, the target threads in multiple submodules are used to process the received processing parameters in turn.

[0066] Step S504, the sub-data processing module corresponding to the preset data processing method processes at least part of the data based on the processing parameters to obtain a processing result corresponding to the data processing request; the sub-data processing module is used to: simultaneously process at least part of the multiple data stored in the first server according to the preset sub-data processing method.

[0067] The above-mentioned preset data processing method can be divided into a processing method for reading data and a processing method for writing data. Then the processing method for reading data and the processing method for writing data are respectively divided into multiple processing steps, and a corresponding sub-data processing module is established for each processing step. The sub-data processing method corresponding to the sub-data processing module can implement the corresponding processing step. For example, the data storage server can generally include searching the storage location of the data in the server and performing a reading process on the stored business data based on the processing parameters. The above-mentioned sub-request processing module can be used to implement the above steps. Similarly, a corresponding sub-data processing module can also be established for the method of writing data.

[0068] When the data processing request received by the client is a request for data reading, after the first server obtains the processing result corresponding to the data processing request, it needs to send the processing result to the client, so that the client sends the processing result to the cloud game server. The process of the first server sending the processing result corresponding to the data processing request to the client can also adopt the above-mentioned pipeline processing method, which is not repeated here.

[0069] The above-mentioned method for processing business data obtains processing parameters of at least part of the data in the business data corresponding to the preset data processing request; the processing parameters are sent by the client after processing the data processing request through multiple sub-request processing modules corresponding to the preset request processing method; the sub-data processing module corresponding to the preset data processing method processes at least part of the data based on the processing parameters to obtain the processing result corresponding to the data processing request. In this method, the data storage server performs pipeline processing on at least part of the data in the business data by setting multiple sub-data processing modules, and can perform data processing of multiple business data at the same time, thereby improving the processing speed of the data processing request and reducing the request processing pressure of the system.

[0070] The following embodiment provides an implementation method for obtaining processing parameters of at least part of data in business data corresponding to a preset data processing request.

[0071] In the specific implementation process, multiple second sub-communication modules corresponding to the preset data receiving mode can be established, wherein the second sub-communication modules are used to process the processing parameters of at least part of the multiple business data stored in the first server according to the preset sub-receiving mode. And the processing parameters of at least part of the data corresponding to the preset data processing request in the business data sent by the client are processed by the multiple second sub-communication modules, so as to obtain the processing parameters of at least part of the processed data;

[0072] Specifically, a corresponding thread pool can be established for each second sub-communication module, and each thread pool includes multiple third candidate threads. For each second sub-communication module, a third target thread can be determined from multiple third candidate threads; and the processing parameters of at least part of the business data are subjected to third processing through the third target thread; the third processing includes data receiving processing, decapsulation processing or parsing processing.

[0073] The following embodiment provides an implementation method for processing at least part of the data based on processing parameters by a sub-data processing module corresponding to a preset data processing method.

[0074] Specifically, a corresponding thread pool can be established for the sub-data processing module, and each thread pool includes multiple fourth candidate threads. For each sub-data processing module, a fourth target thread needs to be determined from multiple fourth candidate threads; the fourth target thread performs a fourth process on at least part of the data based on the processing parameters; the fourth process includes pre-reading processing, storage location search processing, or data processing corresponding to the processing parameters. Corresponding to the data processing process, the data processing includes read processing and / or write processing.

[0075] The embodiment of the present invention also provides another method for processing business data. Figure 4 The method is implemented on the basis of the method. This method introduces a pipeline model to process the data reading and writing process, making the data reading and writing process lock-free and parallel, improving the concurrency of data reading and writing and reducing the competition between data reading and writing, improving the data reading and writing performance while reducing resource consumption.

[0076] In the above method, the MFS client (usually implemented by the mfsmount module) includes the following steps:

[0077] 1. When the cloud game data processing request is passed to the MFS system through the libfuse library, the request will be sent to the IO pipeline according to the file number of the data required in the request, and then returned. This makes the files independent and there is no competition problem. At the same time, the request processing is asynchronous and no longer occupies the libfuse thread, so that the Fuse thread can continue to accept upper-level requests.

[0078] 2. After receiving the request, the data processing pipeline continues to process the data processing request, such as slicing the data processing request, finding the data shard location, etc. Here, it may be further sent to the next layer of the pipeline depending on whether the operation is blocked.

[0079] 3. After the data processing pipeline has processed the data processing request, it will then initiate a read request to the corresponding data storage server, which will be sent to the network pipeline for processing. The network pipeline will send the request to the corresponding data storage server based on the latency between itself and the data storage server, the load of the shard, and other conditions.

[0080] 4. After the data storage server processes the data processing request, it returns the data processing request to the network pipeline layer, and the pipeline layer returns layer by layer to complete the data processing operation.

[0081] The data storage server side includes the following steps:

[0082] 1. All request events on the data storage server are separated from the main thread as an independent network pipeline layer. At the same time, the number of network processing threads is increased so that the cluster can carry a larger number of clients.

[0083] 2. After receiving the data processing request, the network pipeline layer sends it to the data processing pipeline layer, which processes it, such as looking up the corresponding backend storage, whether to pre-read, etc.

[0084] 3. After being processed by the data processing layer, the data processing request enters the back-end pipeline layer. This layer no longer uses the synchronous data processing model, but the asynchronous data processing model to reduce system overhead and thus improve system throughput.

[0085] 4. After the backend data processing returns, the pipeline returns layer by layer, and finally returns the data processing request to the client through the network.

[0086] The above pipelines include the corresponding submodules, such as sub-request processing module, sub-communication module, etc. A thread pool is set at each stage of the pipeline, and the thread pool includes multiple threads to realize simultaneous processing of multiple data processing requests or related data.

[0087] The beneficial effects brought by the technical solution of the present invention are as follows:

[0088] 1. When the client processes the same number of data processing requests, the CPU resource usage is significantly reduced, leaving more idle system resources for cloud games, which improves the performance of cloud games. At the same time, the maximum performance is significantly improved.

[0089] 2. The performance of the data storage server is improved, which improves the cluster data processing request processing capability and can support larger-scale cloud gaming clusters.

[0090] 3. Different data processing stages are in different data processing pipeline layers, which divides data processing into several stages that are independent of each other, greatly improving data processing concurrency.

[0091] By sharding files and index nodes (inodes), corresponding requests are distributed to different pipelines. The same request is always executed in a specific thread, eliminating competition within the file. At the same time, the pipeline model is implemented through a lock-free queue, so that requests can be transmitted between different pipelines without competing for resources, improving overall concurrency.

[0092] For the above method embodiments, see Figure 6 The embodiment of the present invention provides a device for processing business data, wherein the business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system further includes a client; the device is arranged on the client; the device includes:

[0093] The request acquisition module 602 is used to acquire a data processing request corresponding to the business data; the data processing request includes processing parameters of at least part of the business data;

[0094] The request processing module 604 is used to process the data processing request through multiple sub-request processing modules corresponding to the preset request processing mode, and determine the target data storage server among the multiple data storage servers; the sub-request processing module is used to: process multiple data processing requests simultaneously according to the preset sub-request processing mode; the target data storage server is used to: store at least part of the data corresponding to the data processing request in the business data;

[0095] The processing parameter sending module 606 is used to send the processing parameters of at least part of the data to the target data storage server, so that the target data storage server performs data processing on at least part of the data based on the processing parameters.

[0096] The above-mentioned business data processing device obtains a data processing request corresponding to the business data; processes the data processing request through multiple sub-request processing modules corresponding to the preset request processing method, and determines the target data storage server among multiple data storage servers; the sub-request processing module is used to: process multiple data processing requests simultaneously according to the preset sub-request processing method; send the processing parameters of at least part of the data to the target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters. In this device, the client performs pipeline processing on the data processing request by setting multiple sub-request processing modules, and can process multiple data processing requests at the same time, thereby improving the processing speed of the data processing request and reducing the request processing pressure of the system.

[0097] The above-mentioned sub-request processing module includes multiple first candidate threads; the request processing module is also used to: determine the first target thread from multiple first candidate threads; perform first processing on the data processing request through the first target thread; the first processing includes request segmentation processing, data analysis processing or data storage server search processing corresponding to at least part of the data.

[0098] The request processing module is also used to determine the first target thread from multiple first candidate threads based on characteristic parameters of the data processing request and a preset load balancing algorithm; the characteristic parameters include storage information of at least part of the data, sending time and / or sending address of the data processing request.

[0099] The above-mentioned processing parameter sending module is also used to: process the processing parameters of at least part of the data through multiple first sub-communication modules corresponding to the preset data sending method, and send the processed processing parameters of at least part of the data to the target data storage server; the first sub-communication module is used to: simultaneously process multiple processing parameters of at least part of the data according to the preset sub-communication method.

[0100] The first sub-communication module is used to include multiple second candidate threads; the processing parameter sending module is also used to: determine a second target thread from multiple second candidate threads; perform a second processing on the processing parameters of at least part of the data through the second target thread; the second processing includes encapsulation processing or data sending processing.

[0101] The processing parameter sending module is also used to: obtain the load information of the target data storage server and the delay information between the client and the target data storage server; and determine the second target thread from multiple second candidate threads based on the load information of the target data storage server and the delay information between the client and the target data storage server.

[0102] For the above method embodiments, see Figure 7 The embodiment of the present invention provides a device for processing business data, wherein the business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; the distributed storage system further comprises a client; the device is arranged in a first server among the plurality of data storage servers; the device comprises:

[0103] The processing parameter acquisition module 702 is used to acquire processing parameters of at least part of the data in the business data corresponding to the preset data processing request; the processing parameters are sent by the client after processing the data processing request through multiple sub-request processing modules corresponding to the preset request processing method; the data processing request includes the processing parameters of at least part of the data in the business data; the sub-request processing module is used to: process multiple data processing requests simultaneously according to the preset sub-request processing method; at least part of the data corresponding to the data processing request is stored in the first server;

[0104] The data processing module 704 is used to process at least part of the data based on processing parameters through a sub-data processing module corresponding to a preset data processing method to obtain a processing result corresponding to the data processing request; the sub-data processing module is used to: simultaneously process at least part of the data stored in the first server according to the preset sub-data processing method.

[0105] The above-mentioned business data processing device obtains processing parameters of at least part of the data in the business data corresponding to the preset data processing request; the processing parameters are sent by the client after processing the data processing request through multiple sub-request processing modules corresponding to the preset request processing method; the sub-data processing module corresponding to the preset data processing method processes at least part of the data based on the processing parameters to obtain the processing result corresponding to the data processing request. In this device, the data storage server performs pipeline processing on at least part of the data in the business data by setting multiple sub-data processing modules, and can perform data processing of multiple business data at the same time, thereby improving the processing speed of data processing requests and reducing the request processing pressure of the system.

[0106] The above-mentioned processing parameter acquisition module is also used to: process the processing parameters of at least part of the data corresponding to the preset data processing request in the business data sent by the client through multiple second sub-communication modules corresponding to the preset data receiving method, and obtain the processing parameters of at least part of the data after processing; the second sub-communication module is used to: process the processing parameters of at least part of the data in the multiple business data stored in the first server according to the preset sub-receiving method.

[0107] The above-mentioned second sub-communication module is used to include multiple third alternative threads; the above-mentioned processing parameter acquisition module is also used to: determine a third target thread from multiple third alternative threads; perform third processing on the processing parameters of at least part of the business data through the third target thread; the third processing includes data receiving processing, decapsulation processing or parsing processing.

[0108] The above-mentioned sub-data processing module is used to include multiple fourth alternative threads; the above-mentioned data processing module is also used to: determine a fourth target thread from multiple fourth alternative threads; perform fourth processing on at least part of the data based on processing parameters through the fourth target thread; the fourth processing includes pre-reading processing, storage location search processing or data processing corresponding to the processing parameters; data processing includes read processing and / or write processing.

[0109] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned business data processing method. When the electronic device is used as a client in a distributed storage system, the method is implemented as follows:

[0110] A data processing request corresponding to business data is obtained; the data processing request includes processing parameters for at least part of the data in the business data; the data processing request is processed by multiple sub-request processing modules corresponding to a preset request processing method, and a target data storage server among multiple data storage servers is determined; the sub-request processing module is used to: process multiple data processing requests simultaneously according to a preset sub-request processing method; the target data storage server is used to: store at least part of the data in the business data corresponding to the data processing request; and send the processing parameters for at least part of the data to the target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters.

[0111] In the above method, the client performs pipeline processing on the data processing request by setting multiple sub-request processing modules, and can process multiple data processing requests at the same time, thereby improving the processing speed of the data processing request and reducing the request processing pressure of the system.

[0112] Optionally, the above-mentioned sub-request processing module includes multiple first alternative threads; the step of processing the data processing request through multiple sub-request processing modules corresponding to the preset request processing method includes: determining the first target thread from the multiple first alternative threads; performing a first processing on the data processing request through the first target thread; the first processing includes request segmentation processing, data parsing processing or data storage server search processing corresponding to at least part of the data.

[0113] Optionally, the above-mentioned step of determining the first target thread from multiple first candidate threads includes: determining the first target thread from multiple first candidate threads based on characteristic parameters of the data processing request and a preset load balancing algorithm; the characteristic parameters include storage information of at least part of the data, sending time and\or sending address of the data processing request.

[0114] Optionally, the above-mentioned step of sending the processing parameters of at least part of the data to the target data storage server includes: processing the processing parameters of at least part of the data through multiple first sub-communication modules corresponding to the preset data sending method, and sending the processed processing parameters of at least part of the data to the target data storage server; the first sub-communication module is used to: simultaneously process multiple processing parameters of at least part of the data according to the preset sub-communication method.

[0115] Optionally, the above-mentioned first sub-communication module is used to include multiple second alternative threads; the step of processing the processing parameters of at least part of the data through multiple first sub-communication modules corresponding to the preset data sending method includes: determining the second target thread from the multiple second alternative threads; performing a second processing on the processing parameters of at least part of the data through the second target thread; the second processing includes encapsulation processing or data sending processing.

[0116] Optionally, the above-mentioned step of determining the second target thread from multiple second candidate threads includes: obtaining the load information of the target data storage server and the delay information between the client and the target data storage server; based on the load information of the target data storage server and the delay information between the client and the target data storage server, determining the second target thread from multiple second candidate threads.

[0117] When the electronic device is used as the first server in the data storage server, the method is implemented as follows:

[0118] Acquire processing parameters of at least part of the data in the business data corresponding to a preset data processing request; the processing parameters are sent after the client processes the data processing request through multiple sub-request processing modules corresponding to a preset request processing method; the data processing request includes processing parameters of at least part of the data in the business data; the sub-request processing module is used to: process multiple data processing requests simultaneously according to a preset sub-request processing method; at least part of the data corresponding to the data processing request is stored on a first server; at least part of the data is processed based on the processing parameters by a sub-data processing module corresponding to the preset data processing method to obtain a processing result corresponding to the data processing request; the sub-data processing module is used to: process at least part of the multiple data stored in the first server simultaneously according to the preset sub-data processing method.

[0119] In the above method, the data storage server performs pipeline processing on at least part of the business data by setting up multiple sub-data processing modules, and can simultaneously process multiple business data, thereby improving the processing speed of data processing requests and reducing the request processing pressure of the system.

[0120] Optionally, the above-mentioned step of obtaining processing parameters of at least part of the data in the business data corresponding to the preset data processing request includes: processing the processing parameters of at least part of the data in the business data sent by the client corresponding to the preset data processing request through multiple second sub-communication modules corresponding to the preset data receiving method, and obtaining the processing parameters of at least part of the data after processing; the second sub-communication module is used to: process the processing parameters of at least part of the data in the multiple business data stored in the first server according to the preset sub-receiving method.

[0121] Optionally, the above-mentioned second sub-communication module is used to include multiple third alternative threads; the step of processing the processing parameters of at least part of the data in the business data through multiple second sub-communication modules corresponding to the preset data receiving method includes: determining a third target thread from multiple third alternative threads; performing a third processing on the processing parameters of at least part of the data in the business data through the third target thread; the third processing includes data receiving processing, decapsulation processing or parsing processing.

[0122] Optionally, the above-mentioned sub-data processing module is used to include multiple fourth alternative threads; the step of processing at least part of the data based on processing parameters by the sub-data processing module corresponding to the preset data processing method includes: determining a fourth target thread from multiple fourth alternative threads; performing fourth processing on at least part of the data based on processing parameters through the fourth target thread; the fourth processing includes pre-reading processing, storage location search processing or data processing corresponding to the processing parameters; data processing includes read processing and / or write processing.

[0123] See also Figure 8 The electronic device includes a processor 100 and a memory 101. The memory 101 stores machine executable instructions that can be executed by the processor 100. The processor 100 executes the machine executable instructions to implement the above-mentioned business data processing method.

[0124] Further, Figure 8 The electronic device further includes a bus 102 and a communication interface 103 , and the processor 100 , the communication interface 103 and the memory 101 are connected via the bus 102 .

[0125] The memory 101 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0126] The processor 100 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 100. The above processor 100 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101 and completes the steps of the method of the above embodiment in combination with its hardware.

[0127] This embodiment also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned business data processing method.

[0128] The embodiments of the present invention provide a method, device and electronic device for processing business data, including a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the method in the above method embodiment. When the electronic device is used as a client in a distributed storage system, the method is implemented as follows:

[0129] A data processing request corresponding to business data is obtained; the data processing request includes processing parameters for at least part of the data in the business data; the data processing request is processed by multiple sub-request processing modules corresponding to a preset request processing method, and a target data storage server among multiple data storage servers is determined; the sub-request processing module is used to: process multiple data processing requests simultaneously according to a preset sub-request processing method; the target data storage server is used to: store at least part of the data in the business data corresponding to the data processing request; and send the processing parameters for at least part of the data to the target data storage server, so that the target data storage server processes at least part of the data based on the processing parameters.

[0130] In the above method, the client performs pipeline processing on the data processing request by setting multiple sub-request processing modules, and can process multiple data processing requests at the same time, thereby improving the processing speed of the data processing request and reducing the request processing pressure of the system.

[0131] Optionally, the above-mentioned sub-request processing module includes multiple first alternative threads; the step of processing the data processing request through multiple sub-request processing modules corresponding to the preset request processing method includes: determining the first target thread from the multiple first alternative threads; performing a first processing on the data processing request through the first target thread; the first processing includes request segmentation processing, data parsing processing or data storage server search processing corresponding to at least part of the data.

[0132] Optionally, the above-mentioned step of determining the first target thread from multiple first candidate threads includes: determining the first target thread from multiple first candidate threads based on characteristic parameters of the data processing request and a preset load balancing algorithm; the characteristic parameters include storage information of at least part of the data, sending time and\or sending address of the data processing request.

[0133] Optionally, the above-mentioned step of sending the processing parameters of at least part of the data to the target data storage server includes: processing the processing parameters of at least part of the data through multiple first sub-communication modules corresponding to the preset data sending method, and sending the processed processing parameters of at least part of the data to the target data storage server; the first sub-communication module is used to: simultaneously process multiple processing parameters of at least part of the data according to the preset sub-communication method.

[0134] Optionally, the above-mentioned first sub-communication module is used to include multiple second alternative threads; the step of processing the processing parameters of at least part of the data through multiple first sub-communication modules corresponding to the preset data sending method includes: determining the second target thread from the multiple second alternative threads; performing a second processing on the processing parameters of at least part of the data through the second target thread; the second processing includes encapsulation processing or data sending processing.

[0135] Optionally, the above-mentioned step of determining the second target thread from multiple second candidate threads includes: obtaining the load information of the target data storage server and the delay information between the client and the target data storage server; based on the load information of the target data storage server and the delay information between the client and the target data storage server, determining the second target thread from multiple second candidate threads.

[0136] When the electronic device is used as the first server in the data storage server, the method is implemented as follows:

[0137] Acquire processing parameters of at least part of the data in the business data corresponding to a preset data processing request; the processing parameters are sent after the client processes the data processing request through multiple sub-request processing modules corresponding to a preset request processing method; the data processing request includes processing parameters of at least part of the data in the business data; the sub-request processing module is used to: process multiple data processing requests simultaneously according to a preset sub-request processing method; at least part of the data corresponding to the data processing request is stored on a first server; at least part of the data is processed based on the processing parameters by a sub-data processing module corresponding to the preset data processing method to obtain a processing result corresponding to the data processing request; the sub-data processing module is used to: process at least part of the multiple data stored in the first server simultaneously according to the preset sub-data processing method.

[0138] In the above method, the data storage server performs pipeline processing on at least part of the business data by setting up multiple sub-data processing modules, and can simultaneously process multiple business data, thereby improving the processing speed of data processing requests and reducing the request processing pressure of the system.

[0139] Optionally, the above-mentioned step of obtaining processing parameters of at least part of the data in the business data corresponding to the preset data processing request includes: processing the processing parameters of at least part of the data in the business data sent by the client corresponding to the preset data processing request through multiple second sub-communication modules corresponding to the preset data receiving method, and obtaining the processing parameters of at least part of the data after processing; the second sub-communication module is used to: process the processing parameters of at least part of the data in the multiple business data stored in the first server according to the preset sub-receiving method.

[0140] Optionally, the above-mentioned second sub-communication module is used to include multiple third alternative threads; the step of processing the processing parameters of at least part of the data in the business data through multiple second sub-communication modules corresponding to the preset data receiving method includes: determining a third target thread from multiple third alternative threads; performing a third processing on the processing parameters of at least part of the data in the business data through the third target thread; the third processing includes data receiving processing, decapsulation processing or parsing processing.

[0141] Optionally, the above-mentioned sub-data processing module is used to include multiple fourth alternative threads; the step of processing at least part of the data based on processing parameters by the sub-data processing module corresponding to the preset data processing method includes: determining a fourth target thread from multiple fourth alternative threads; performing fourth processing on at least part of the data based on processing parameters through the fourth target thread; the fourth processing includes pre-reading processing, storage location search processing or data processing corresponding to the processing parameters; data processing includes read processing and / or write processing.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0143] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0144] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0145] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate positions or positional relationships based on the positions or positional relationships described in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0146] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for processing business data, characterized in that: The business data is stored in a plurality of preset data storage servers; the business data is stored in the data storage servers in the form of file slices; The multiple data storage servers belong to a preset distributed storage system; The distributed storage system further includes a client; the method is applied to the client; the method includes: Obtaining a data processing request corresponding to the business data; the data processing request includes processing parameters of at least part of the business data; The data processing request is processed by a plurality of sub-request processing modules corresponding to a preset request processing method, and a target data storage server among the plurality of data storage servers is determined; the sub-request processing module is used to: process a plurality of data processing requests simultaneously according to a preset sub-request processing method; the target data storage server is used to: store at least part of the data in the business data corresponding to the data processing request; Sending the processing parameters of at least part of the data to the target data storage server, so that the target data storage server performs data processing on at least part of the data based on the processing parameters; The sub-request processing module includes a plurality of first candidate threads; The step of processing the data processing request by using a plurality of sub-request processing modules corresponding to a preset request processing method comprises: Determine a first target thread from the plurality of first candidate threads; Performing a first process on the data processing request through the first target thread; the first process includes request segmentation processing, data parsing processing, or data storage server search processing corresponding to at least part of the data; The step of determining a first target thread from the plurality of first candidate threads comprises: Determining at least one file corresponding to the at least portion of the data; For each file in the at least one file, based on the serial number of the file, a first target thread corresponding to the file is determined from the multiple first candidate threads.

2. The method according to claim 1, characterized in that The step of determining a first target thread from the plurality of first candidate threads comprises: Based on characteristic parameters of the data processing request and a preset load balancing algorithm, a first target thread is determined from the multiple first candidate threads; the characteristic parameters include storage information of at least part of the data, sending time and / or sending address of the data processing request.

3. The method according to claim 1, characterized in that The step of sending the processing parameters of at least part of the data to the target data storage server comprises: The processing parameters of at least part of the data are processed by multiple first sub-communication modules corresponding to the preset data sending method, and the processed processing parameters of at least part of the data are sent to the target data storage server; the first sub-communication module is used to: simultaneously process multiple processing parameters of at least part of the data according to the preset sub-communication method.

4. The method according to claim 3, characterized in that The first sub-communication module includes a plurality of second candidate threads; The step of processing the processing parameters of at least part of the data by using a plurality of first sub-communication modules corresponding to the preset data sending mode includes: Determine a second target thread from the plurality of second candidate threads; The second target thread performs a second processing on the processing parameters of at least part of the data; the second processing includes encapsulation processing or data sending processing.

5. The method according to claim 4, characterized in that The step of determining the second target thread from the plurality of second candidate threads comprises: Acquire load information of the target data storage server and delay information between the client and the target data storage server; Based on the load information of the target data storage server and the delay information between the client and the target data storage server, a second target thread is determined from the plurality of second candidate threads.

6. A method for processing business data, characterized in that: The business data is stored in a plurality of preset data storage servers; the plurality of data storage servers belong to a preset distributed storage system; The distributed storage system further includes a client; the method is applied to a first server among the plurality of data storage servers; the method includes: Acquire processing parameters of at least part of the data in the business data corresponding to a preset data processing request; the processing parameters are sent after the client processes the data processing request through multiple sub-request processing modules corresponding to a preset request processing method; the data processing request includes processing parameters of at least part of the data in the business data; the sub-request processing module is used to: process multiple data processing requests simultaneously according to a preset sub-request processing method; at least part of the data corresponding to the data processing request is stored in the first server; Processing the at least part of the data based on the processing parameters by a sub-data processing module corresponding to a preset data processing method to obtain a processing result corresponding to the data processing request; the sub-data processing module is used to: simultaneously process the at least part of the data stored in the first server according to the preset sub-data processing method; The sub-request processing module includes a plurality of first candidate threads; The step of processing the data processing request by using a plurality of sub-request processing modules corresponding to a preset request processing method comprises: Determine a first target thread from the plurality of first candidate threads; Performing a first process on the data processing request through the first target thread; the first process includes request segmentation processing, data parsing processing, or data storage server search processing corresponding to at least part of the data; The step of determining a first target thread from the plurality of first candidate threads comprises: Determining at least one file corresponding to the at least portion of the data; For each file in the at least one file, based on the serial number of the file, a first target thread corresponding to the file is determined from the multiple first candidate threads.

7. The method according to claim 6, characterized in that The step of obtaining processing parameters of at least part of the data in the business data corresponding to the preset data processing request includes: The processing parameters of at least part of the data corresponding to the preset data processing request in the business data sent by the client are processed by multiple second sub-communication modules corresponding to the preset data receiving method to obtain the processing parameters of at least part of the data after processing; the second sub-communication module is used to: process the processing parameters of at least part of the data in the multiple business data stored in the first server according to the preset sub-receiving method.

8. The method according to claim 7, characterized in that The second sub-communication module includes a plurality of third candidate threads; The step of processing the processing parameters of at least part of the business data by using a plurality of second sub-communication modules corresponding to the preset data receiving mode includes: Determine a third target thread from the plurality of third candidate threads; The third target thread performs a third processing on the processing parameters of at least part of the business data; the third processing includes data receiving processing, decapsulation processing or parsing processing.

9. The method according to claim 6, characterized in that The sub-data processing module is used to include a plurality of fourth candidate threads; The step of processing at least part of the data based on the processing parameters by a sub-data processing module corresponding to a preset data processing method comprises: Determine a fourth target thread from the plurality of fourth candidate threads; The fourth target thread performs a fourth processing on at least part of the data based on the processing parameters; the fourth processing includes pre-reading processing, storage location search processing or data processing corresponding to the processing parameters; the data processing includes read processing and / or write processing.

10. A device for processing business data, characterized in that: The business data is stored in a plurality of preset data storage servers; the business data is stored in the data storage servers in the form of file slices; The multiple data storage servers belong to a preset distributed storage system; The distributed storage system further includes a client; the device is arranged on the client; The device comprises: A request acquisition module, used to acquire a data processing request corresponding to the business data; the data processing request includes processing parameters of at least part of the business data; A request processing module, used to process the data processing request through a plurality of sub-request processing modules corresponding to a preset request processing method, and determine a target data storage server among the plurality of data storage servers; the sub-request processing module is used to: simultaneously process a plurality of data processing requests according to a preset sub-request processing method; the target data storage server is used to: store at least part of the data in the business data corresponding to the data processing request; A processing parameter sending module, used for sending the processing parameters of at least part of the data to the target data storage server, so that the target data storage server performs data processing on at least part of the data based on the processing parameters; The sub-request processing module includes a plurality of first candidate threads; The request acquisition module is also used for: Determine a first target thread from the plurality of first candidate threads; Performing a first process on the data processing request through the first target thread; the first process includes request segmentation processing, data parsing processing, or data storage server search processing corresponding to at least part of the data; The request acquisition module is also used for: Determining at least one file corresponding to the at least portion of the data; For each file in the at least one file, based on the serial number of the file, a first target thread corresponding to the file is determined from the multiple first candidate threads.

11. A device for processing business data, characterized in that: The business data is stored in a plurality of preset data storage servers; the business data is stored in the data storage servers in the form of file slices; the plurality of data storage servers belong to a preset distributed storage system; The distributed storage system further comprises a client; the device is arranged on a first server among the plurality of data storage servers; The device comprises: a processing parameter acquisition module, used for acquiring processing parameters of at least part of the data in the business data corresponding to a preset data processing request; the processing parameters are sent by the client after processing the data processing request through a plurality of sub-request processing modules corresponding to a preset request processing method; the data processing request includes processing parameters of at least part of the data in the business data; the sub-request processing module is used for: processing a plurality of data processing requests simultaneously according to a preset sub-request processing method; at least part of the data corresponding to the data processing request is stored in the first server; a data processing module, configured to process the at least part of the data based on the processing parameters by a sub-data processing module corresponding to a preset data processing method, and obtain a processing result corresponding to the data processing request; the sub-data processing module is configured to simultaneously process the at least part of the data stored in the first server according to the preset sub-data processing method; The sub-request processing module includes a plurality of first candidate threads; The step of processing the data processing request by using a plurality of sub-request processing modules corresponding to a preset request processing method comprises: Determine a first target thread from the plurality of first candidate threads; Performing a first process on the data processing request through the first target thread; the first process includes request segmentation processing, data parsing processing, or data storage server search processing corresponding to at least part of the data; The step of determining a first target thread from the plurality of first candidate threads comprises: Determining at least one file corresponding to the at least portion of the data; For each file in the at least one file, based on the serial number of the file, a first target thread corresponding to the file is determined from the multiple first candidate threads.

12. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the business data processing method described in any one of claims 1-9.

13. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the business data processing method described in any one of claims 1-9.

Images