A data processing method and device

Abstract

The present invention discloses a data processing method and device, wherein the method is as follows: a data middleware receives task data from a user host; the data middleware pre-stores a mapping relationship between a script language type and a computing engine type; the data middleware determines at least one computing engine mapped with the script language type of the script to be executed according to the mapping relationship; if there is a target computing engine that meets the computing engine type of the script to be executed in the at least one computing engine, the data middleware submits the script to be executed to the target computing engine for processing. When the above method is applied to financial technology (Fintech), since the computing engine type is specified in the task data and the script to be executed of the specified script language type is passed in, the flexibility of the data middleware processing is increased.

Description

Technical Field

[0001] The present invention relates to the fields of financial technology (Fintech) and computer software, and in particular to a data processing method and device. Background Art

[0002] With the development of computer technology, more and more technologies (big data, distributed computing, blockchain, artificial intelligence, etc.) are being applied in the financial field, and the traditional financial industry is gradually transforming into financial technology (Fintech). At present, in the field of financial technology, due to the limited data processing capabilities of the client, the client's script is often transferred through data middleware, and then the data is processed by the computing engine.

[0003] However, in the existing technology, data middleware does not support the processing of multiple computing engines, nor does it support the processing of multiple scripting languages. It can only connect specific scripts to specific computing engines for data services. For example, HiveServer2 and SparkThriftserver can only connect to data in hive, which reduces the flexibility of data processing. This is a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and device, which solve the problem of reduced flexibility of data processing in data middleware in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a data processing method: a data middleware receives task data from a user host; the task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the data middleware pre-stores a mapping relationship between a script language type and a computing engine type; the data middleware determines at least one computing engine mapped to the script language type of the script to be executed based on the mapping relationship; if there is a target computing engine that matches the computing engine type of the script to be executed among the at least one computing engine, the data middleware submits the script to be executed to the target computing engine for processing.

[0006] Under the above method, the data middleware receives task data from the user host. Since the data middleware pre-stores the mapping relationship between multiple scripting languages ​​and multiple computing engines, the computing engine type can be specified in the task data, and the script to be executed of the specified script language type can be passed in, which increases the flexibility of the data middleware processing. The data middleware can determine at least one computing engine according to the script language type of the script to be executed. If there is a target computing engine, the script to be executed is submitted to the target computing engine for processing, thereby completing the processing of the task data.

[0007] In an optional implementation, the data middleware has multiple entrances, and the entrances are used to store data received by the data middleware; the data middleware receives task data from the client, including: the data middleware selects a target entrance from the multiple entrances; the probability of each entrance in the multiple entrances being selected as the target entrance is negatively correlated with the amount of data in the entrance; the data middleware receives the task data through the target entrance.

[0008] In the above method, the data middleware sets up multiple entrances, and selects a target entrance from the multiple entrances. Since the probability of each entrance among the multiple entrances being selected as the target entrance is negatively correlated with the amount of data in the entrance, the probability of the entrance with a higher load being selected is lower, and the probability of the entrance with a lower load being selected is higher, thereby achieving load balancing of each entrance.

[0009] In an optional implementation, for any one of the multiple entrances, there are multiple producer queues and multiple consumer queues in the entrance; the multiple producer queues are used to store data received by the data middleware; the multiple consumer queues are used to obtain data from the multiple producer queues; the method also includes: the data middleware processes the data in the consumer queues in parallel.

[0010] In the above method, there are multiple producer queues and multiple consumer queues in the entry; the data received by the data middleware is temporarily stored through the multiple producer queues, and data is obtained from the multiple producer queues through multiple consumer queues; the data middleware processes the data in the consumer queues in parallel, thereby improving the processing efficiency of the data middleware.

[0011] In an optional implementation, before the data middleware submits the script to be executed to the target computing engine for processing, it also includes: the data middleware receives the host resource information of the user host; the data middleware determines whether the host resources of the user host are sufficient based on the host resource information; if so, the data middleware submits the script to be executed to the target computing engine for processing.

[0012] Under the above method, the data middleware receives the host resource information of the user host, and only when the host resources of the user host are sufficient, the data middleware submits the script to be executed to the target computing engine for processing. If the host resources of the user host are insufficient, a failure may occur in the target computing engine during the execution of the script to be executed. Therefore, early judgment can ensure the execution stability of the script to be executed.

[0013] In a second aspect, the present application provides a data processing method, comprising: a target computing engine obtains task data submitted by a data middleware; the task data comprises: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the target computing engine is a computing engine determined by the data middleware according to a mapping relationship between the script language type and the computing engine type to conform to the script language type of the script to be executed and the computing engine type of the script to be executed; the target computing engine executes the script to be executed according to the script language type of the script to be executed.

[0014] In the above manner, the target computing engine is a computing engine that the data middleware determines, based on the mapping relationship between the script language type and the computing engine type, to conform to the script language type of the script to be executed and the computing engine type of the script to be executed; thereby ensuring that the target computing engine is capable of processing the script to be executed of the script language type; therefore, the target computing engine executes the script to be executed according to the script language type of the script to be executed, thereby improving the flexibility of data processing.

[0015] In an optional implementation, the task data in the data middleware comes from the user host; the method further includes: the target computing engine obtains the user name submitted by the data middleware; the user name submitted by the data middleware is submitted by the user host to the data middleware; the target computing engine determines the data record range of the user name, and executes the script to be executed within the data record range.

[0016] In the above manner, the target computing engine obtains the user name submitted by the data middleware; and executes the script to be executed within the data record range, thereby isolating the data record processing of different users and improving the security of data processing.

[0017] In a third aspect, the present application provides a data processing device, comprising: a receiving module, used to receive task data from a user host; the task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the data middleware pre-stores a mapping relationship between the script language type and the computing engine type; a processing module, used to determine at least one computing engine mapped to the script language type of the script to be executed according to the mapping relationship; if there is a target computing engine that matches the computing engine type of the script to be executed among the at least one computing engine, the script to be executed is submitted to the target computing engine for processing.

[0018] In an optional embodiment, the device has multiple entrances, and the entrances are used to store data received by the data middleware; the processing module is specifically used to: select a target entrance from the multiple entrances; the probability of each entrance in the multiple entrances being selected as the target entrance is negatively correlated with the amount of data in the entrance; the receiving module is specifically used to: receive the task data through the target entrance.

[0019] In an optional implementation, for any one of the multiple entrances, there are multiple producer queues and multiple consumer queues in the entrance; the multiple producer queues are used to store data received by the data middleware; the multiple consumer queues are used to obtain data from the multiple producer queues; and the processing module is also used to: process the data in the consumer queues in parallel.

[0020] In an optional implementation, the receiving module is further used to: receive host resource information of the user host; the processing module is further used to: determine whether the host resources of the user host are sufficient based on the host resource information; if so, execute the step of submitting the script to be executed to the target computing engine for processing.

[0021] The beneficial effects of the third aspect and each embodiment of the third aspect can be referred to the beneficial effects of the first aspect and each embodiment of the first aspect, and will not be repeated here.

[0022] In a fourth aspect, the present application provides a data processing device, comprising: an acquisition module, used to acquire task data submitted by a data middleware to a target computing engine; the task data comprises: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the target computing engine is a computing engine determined by the data middleware according to a mapping relationship between the script language type and the computing engine type to conform to the script language type of the script to be executed and the computing engine type of the script to be executed; a processing module, used to execute the script to be executed according to the script language type of the script to be executed.

[0023] In an optional implementation, the acquisition module is also used to: acquire the user name submitted by the data middleware; the user name submitted by the data middleware is submitted by the user host to the data middleware; the processing module is also used to: determine the data record range of the user name, and execute the script to be executed within the data record range.

[0024] The beneficial effects of the fourth aspect and each embodiment of the fourth aspect can be referred to the beneficial effects of the second aspect and each embodiment of the second aspect, and will not be repeated here.

[0025] In a fifth aspect, an embodiment of the present application provides a computer device, including a program or instructions, which, when executed, is used to execute the method of the above-mentioned first aspect and each embodiment of the first aspect or the second aspect and each embodiment of the second aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a storage medium, including a program or an instruction, which, when executed, is used to execute the method of the above-mentioned first aspect and each embodiment of the first aspect or the second aspect and each embodiment of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of a data processing method according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of an applicable architecture of a data processing method provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in combination with the accompanying drawings and specific implementation methods of the specification. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

[0032] In the process of financial institutions (banking institutions, insurance institutions or securities institutions) conducting business (such as bank loan business, deposit business, etc.), due to the limited data processing capabilities of the client, the client script is often transferred through the data middleware, and then the data is processed through the computing engine. However, in the prior art, the data middleware does not support the processing of multiple computing engines, nor does it support the processing of multiple script languages. It can only connect specific scripts to specific computing engines for data services, such as HiveServer2 and SparkThriftserver can only connect to data in hive, which will reduce the flexibility of data processing. This situation does not meet the needs of financial institutions such as banks, and cannot guarantee the efficient operation of various businesses of financial institutions.

[0033] For this reason, Figure 1 As shown, the present application proposes a data processing method.

[0034] Step 101: The data middleware receives task data from the user host.

[0035] The task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the data middleware pre-stores a mapping relationship between the script language type and the computing engine type.

[0036] Step 102: The data middleware determines at least one computing engine mapped to the script language type of the script to be executed based on the mapping relationship; if there is a target computing engine that matches the computing engine type of the script to be executed among the at least one computing engine, the data middleware submits the script to be executed to the target computing engine for processing.

[0037] Before step 101, the data middleware may receive the host resource information of the user host; and then determine whether the host resources of the user host are sufficient based on the host resource information; if so, the data middleware submits the script to be executed to the target computing engine for processing.

[0038] In addition, before step 101, the data middleware may also send verification information such as a user name and password to the target data engine.

[0039] In step 101, it should be noted that the mapping relationship between the script language type and the computing engine type can be a many-to-many relationship. For example, a script of script language type A can be executed by multiple types of computing engines.

[0040] In an optional implementation, the data middleware has multiple entries, and the entries are used to store data received by the data middleware; the data middleware can receive task data from the client in the following manner:

[0041] The data middleware selects a target entry from the multiple entries; the probability of each of the multiple entries being selected as the target entry is negatively correlated with the amount of data in the entry; the data middleware receives the task data through the target entry.

[0042] For example, there are three inlets A, B, and C; the corresponding data volumes are 5MB, 3MB, and 2MB respectively. The inlets can be selected with probabilities of 6 / 31, 10 / 31, and 15 / 31. Obviously, C has the highest probability of being selected.

[0043] In the above optional implementation manner, the data may be processed according to the following implementation manner:

[0044] For any of the multiple entrances, the entrance has multiple producer queues and multiple consumer queues. The multiple producer queues are used to store data received by the data middleware; the multiple consumer queues are used to obtain data from the multiple producer queues; and the data middleware processes the data in the consumer queues in parallel.

[0045] After step 102, the target computing engine may process the data through the following steps:

[0046] The target computing engine obtains the task data submitted by the data middleware.

[0047] Step (1): The task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the target computing engine is a computing engine that is determined by the data middleware according to a mapping relationship between the script language type and the computing engine type, and that conforms to the script language type of the script to be executed and the computing engine type of the script to be executed.

[0048] Step (2): the target computing engine executes the script to be executed according to the script language type of the script to be executed.

[0049] In the above step (2), in an optional implementation, the task data in the data middleware comes from the user host; the target computing engine obtains the user name submitted by the data middleware; the user name submitted by the data middleware is submitted by the user host to the data middleware; the target computing engine determines the data record range of the user name, and executes the script to be executed within the data record range.

[0050] Therefore, the above method supports user permission control and fine-grained control of data permissions. User B who connects through the JDBC client can only see the data that user B has permission to.

[0051] The method of step 101 to step 102 can be applied in multiple scenarios. For example, under the Java Database Connectivity (JDBC) service. It can provide JDBC services to support multiple computing engines and multiple scripting languages ​​to connect to the underlying computing and storage components, support components such as Spark, Hive, TIDB, HBAS, and support languages ​​such as SQL, HQL, Python, and Scala. In addition, it also supports high concurrency and high availability of tasks. Moreover, this method supports refined management of resource usage, supports hot start of computing engines, and automatically stops during idle time to reduce resource waste.

[0052] Combine the following Figure 2, the scheme of the present invention is described in detail through specific embodiments.

[0053] Step 1: The user applies for a database connection from the data middleware through the client.

[0054] Specifically, the user first submits the data middleware's (uniform resource locator, URL), user name, and password to the data driver of the data middleware.

[0055] The data driver obtains the username, password and URL from the client.

[0056] The data driver locates the corresponding data middleware through the URL and submits the username and password to the data middleware.

[0057] The data middleware verifies the user name and password from the data driver, and if the verification is successful, it issues a database connection to the client. A database connection is a channel dedicated to transmitting data in a database format.

[0058] Step 2: The user submits data processing tasks to the data middleware through the client.

[0059] Since the data middleware pre-encapsulates the mapping relationship between multiple scripting languages ​​and multiple computing engine types, and provides communication interfaces for multiple computing engines, the data middleware can process multiple scripting languages ​​with multiple computing engines. For example, the SQL language can be executed through Hive and TIDB. When a SQL script is received, if the engine type corresponding to the SQL script exists in the SQL mapping relationship, such as the computing engine indicated by the engine type value is Hive, the SQL script will be submitted to Hive for execution.

[0060] The data processing task may be a query task, such as a task of querying data records for a certain keyword.

[0061] The specific process of the second step is:

[0062] The user submits task data (including: script file, script language type and engine type) to the data driver through the client.

[0063] The data driver then submits the task data to the data middleware.

[0064] Specifically, multiple identical services are running on the data middleware at the same time; each service can be used to receive and submit task data to the corresponding computing engine. This multi-service approach can ensure the high availability of the data middleware.

[0065] The data middleware has multiple entrances. When the service receives the task data, it will determine an entrance for the task data. Multiple entrances can adopt a load balancing strategy. The more data there is in the entrance, the lower the probability of receiving new task data. Each entrance corresponds to multiple producer consumer queues. The arriving task data is first placed in the producer queue. According to the preset retrieval rules, the task data in the producer queue is taken to the producer queue. The task data in each producer queue can be processed by multiple services at the same time and submitted to the corresponding computing engine. In this way, high concurrent processing of tasks is achieved.

[0066] Step 3: The data middleware handles data processing tasks through the computing engine.

[0067] Specifically, the data middleware will query whether there is a computing engine corresponding to the engine type through the engine type in the task data. If there is, a computing engine will be randomly assigned, and the script in the data processing task will be executed through the computing engine, and the corresponding data will be returned. If not, a computing engine corresponding to the engine type will be restarted to execute the script in the data processing task. Before the computing engine executes the script, the data middleware can also first determine whether the user's resources (such as CPU, memory, etc.) are sufficient, and only execute the script if the user's resources are sufficient.

[0068] Step 4: The calculation engine executes the script and returns the executed data results to the data driver.

[0069] It should be noted that in the fourth step, the computing engine can isolate permissions, and the permissions of each user are independent of each other. During the execution of a user's script by the computing engine, requests to access data records not under the user name will be prohibited.

[0070] Step 5: The data driver returns the executed data results to the client.

[0071] After the fourth step, the calculation engine will drive the execution status return value of the execution script. After the user obtains the execution status, if the execution status indicates that the script is executed successfully, the data result is obtained from the data driver.

[0072] It should be noted that the data results in the fifth step can be returned to the client in a predetermined format, such as a table format.

[0073] The entire task execution and result reading process is shown above. Through the above steps, we can know the JDBC service provided by Linkis. Through the passed-in users, we can achieve permission isolation and resource isolation. Through the high concurrency and high availability capabilities provided by Linkis, we can achieve enterprise-level capabilities such as high availability and high concurrency for users' JDBC services. And through the passed-in scripts and engine types, we can support multiple languages ​​and multiple computing engines.

[0074] like Figure 3 As shown, the present application provides a data processing device, including: a receiving module 301, used to receive task data from a user host; the task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the data middleware pre-stores a mapping relationship between the script language type and the computing engine type; a processing module 302, used to determine at least one computing engine mapped to the script language type of the script to be executed according to the mapping relationship; if there is a target computing engine that meets the computing engine type of the script to be executed in the at least one computing engine, the script to be executed is submitted to the target computing engine for processing.

[0075] In an optional implementation, the device has multiple entrances, and the entrances are used to store data received by the data middleware; the processing module 302 is specifically used to: select a target entrance from the multiple entrances; the probability of each entrance in the multiple entrances being selected as the target entrance is negatively correlated with the amount of data in the entrance; the receiving module 301 is specifically used to: receive the task data through the target entrance.

[0076] In an optional implementation, for any one of the multiple entrances, there are multiple producer queues and multiple consumer queues in the entrance; the multiple producer queues are used to store data received by the data middleware; the multiple consumer queues are used to obtain data from the multiple producer queues; the processing module 302 is also used to: process the data in the consumer queues in parallel.

[0077] In an optional implementation, the receiving module 301 is further used to: receive host resource information of the user host; the processing module 302 is further used to: determine whether the host resources of the user host are sufficient based on the host resource information; if so, execute the step of submitting the script to be executed to the target computing engine for processing.

[0078] like Figure 4 As shown, the present application provides a data processing device, including: an acquisition module 401, used to acquire task data submitted by a data middleware to a target computing engine; the task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the target computing engine is a computing engine determined by the data middleware according to a mapping relationship between the script language type and the computing engine type to conform to the script language type of the script to be executed and the computing engine type of the script to be executed; a processing module 402, used to execute the script to be executed according to the script language type of the script to be executed.

[0079] In an optional implementation, the acquisition module 401 is also used to: obtain the user name submitted by the data middleware; the user name submitted by the data middleware is submitted by the user host to the data middleware; the processing module 402 is also used to: determine the data record range of the user name, and execute the script to be executed within the data record range.

[0080] An embodiment of the present application provides a computer device, including a program or an instruction. When the program or the instruction is executed, it is used to execute a data processing method and any optional method provided in the embodiment of the present application.

[0081] An embodiment of the present application provides a storage medium, including a program or an instruction, which, when executed, is used to execute a data processing method and any optional method provided in an embodiment of the present application.

[0082] Finally, it should be noted that: those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code.

[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0085] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: include: The data middleware receives task data from the user host; The task data includes: the script to be executed, the script language type of the script to be executed and the computing engine type of the script to be executed; the data middleware pre-stores the mapping relationship between the script language type and the computing engine type; The data middleware determines at least one computing engine mapped with the script language type of the script to be executed according to the mapping relationship; if there is a target computing engine that matches the computing engine type of the script to be executed among the at least one computing engine, the data middleware submits the script to be executed to the target computing engine for processing; The data middleware has multiple entrances, and the entrances are used to store data received by the data middleware; the data middleware receives task data from the client, including: The data middleware selects a target entry from the multiple entries; the probability of each of the multiple entries being selected as the target entry is negatively correlated with the amount of data in the entry; The data middleware receives the task data through the target entry.

2. The method according to claim 1, characterized in that For any of the multiple entrances, there are multiple producer queues and multiple consumer queues in the entrance; the multiple producer queues are used to store data received by the data middleware; The multiple consumer queues are used to obtain data from the multiple producer queues; The method further comprises: The data middleware processes the data in the consumer queue in parallel.

3. The method according to any one of claims 1 or 2, characterized in that: Before the data middleware submits the script to be executed to the target computing engine for processing, the method further includes: The data middleware receives the host resource information of the user host; The data middleware determines whether the host resources of the user host are sufficient according to the host resource information; if so, the data middleware submits the script to be executed to the target computing engine for processing.

4. A data processing method, characterized in that: include: The target computing engine obtains the task data submitted by the data middleware; The task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the target computing engine is a computing engine that the data middleware determines to be consistent with the script language type of the script to be executed and the computing engine type of the script to be executed according to the mapping relationship between the script language type and the computing engine type; The target computing engine executes the script to be executed according to the script language type of the script to be executed; The data middleware has multiple entrances, which are used to store data received by the data middleware; the task data submitted by the data middleware is received by a target entrance selected by the data middleware from the multiple entrances; the probability of each entrance among the multiple entrances being selected as the target entrance is negatively correlated with the amount of data in the entrance.

5. The method according to claim 4, characterized in that The task data in the data middleware comes from the user host; the method also includes: The target computing engine obtains the user name submitted by the data middleware; the user name submitted by the data middleware is submitted by the user host to the data middleware; The target computing engine determines the data record range of the user name, and executes the script to be executed within the data record range.

6. A data processing device, characterized in that: include: A receiving module, used for receiving task data from a user host; The task data includes: the script to be executed, the script language type of the script to be executed and the computing engine type of the script to be executed; the data middleware pre-stores the mapping relationship between the script language type and the computing engine type; a processing module, configured to determine at least one computing engine to which the script language type of the script to be executed is mapped according to the mapping relationship; if a target computing engine that matches the computing engine type of the script to be executed exists in the at least one computing engine, submit the script to be executed to the target computing engine for processing; The device has multiple entrances, each of which is used to store data received by the data middleware; the processing module is specifically used to: Selecting a target entrance from the multiple entrances; the probability of each entrance in the multiple entrances being selected as the target entrance is negatively correlated with the amount of data in the entrance; The receiving module is specifically used for: The task data is received through the target entry.

7. The device according to claim 6, characterized in that For any of the multiple entrances, the entrance has multiple producer queues and multiple consumer queues; the multiple producer queues are used to store data received by the data middleware; the multiple consumer queues are used to obtain data from the multiple producer queues; the processing module is also used to: The data in the consumer queues are processed in parallel.

8. A data processing device, characterized in that: include: The acquisition module is used to obtain the task data submitted by the data middleware to the target computing engine; The task data includes: a script to be executed, a script language type of the script to be executed, and a computing engine type of the script to be executed; the target computing engine is a computing engine that the data middleware determines to be consistent with the script language type of the script to be executed and the computing engine type of the script to be executed according to the mapping relationship between the script language type and the computing engine type; A processing module, configured to execute the script to be executed according to the script language type of the script to be executed; The data middleware has multiple entrances, which are used to store data received by the data middleware; the task data submitted by the data middleware is received by a target entrance selected by the data middleware from the multiple entrances; the probability of each entrance among the multiple entrances being selected as the target entrance is negatively correlated with the amount of data in the entrance.

9. A computer device, characterized in that: The method comprises a program or an instruction. When the program or the instruction is executed, the method according to any one of claims 1 to 5 is performed.

10. A storage medium, characterized in that: The method comprises a program or an instruction. When the program or the instruction is executed, the method according to any one of claims 1 to 5 is performed.

Images