Task processing method and device

Abstract

The invention discloses a task processing method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the steps of receiving a target task, and determining a plurality of processing services and target data corresponding to the target task; splitting the plurality of processing services into at least one service group; wherein the service group comprises at least one processing service; splitting the target data into at least one data group; for each data group, controlling each service group to process the data group in parallel; obtaining a processing result of the data group according to the processing result of each service group on the data group; and obtaining a processing result corresponding to the target task according to the processing result of each data group. According to the embodiment, data processing in various business scenes can be flexibly and efficiently carried out.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a task processing method and device. Background Art

[0002] With the development of information systems, large-scale data processing is often required. In existing technologies, large data sets are usually finely divided and paired with powerful computing resources. Specifically, data and computing power are divided into multiple independent parts, and the data part corresponds to the computing power part one by one, ensuring that each piece of data can be processed by the corresponding computing resources. However, this processing method cannot flexibly and efficiently process data in various business scenarios. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a task processing method and device, which can flexibly and efficiently perform data processing in various business scenarios.

[0004] In a first aspect, an embodiment of the present invention provides a task processing method, including:

[0005] receiving a target task, and determining a plurality of processing services and target data corresponding to the target task;

[0006] Splitting the plurality of processing services into at least one service group; wherein the service group includes at least one processing service;

[0007] Splitting the target data into at least one data group;

[0008] For each data group, control each service group to process the data group in parallel; obtain the processing result of the data group according to the processing result of each service group on the data group;

[0009] According to the processing results of each data group, the processing results corresponding to the target task are obtained.

[0010] Optionally, the plurality of processing services are split into at least one service group, including:

[0011] Get the associated information of each processing service;

[0012] According to the association information, the multiple processing services are divided into at least one service group; wherein each processing service in the service group has an association relationship.

[0013] Optionally, the target data includes: a plurality of data records;

[0014] After determining the multiple processing services and target data corresponding to the target task, it also includes:

[0015] For each data record, the data record is encoded so that the encoded data record includes its corresponding record identifier.

[0016] Optionally, the target data is split into at least one data group, including:

[0017] Determine the number of grouped records;

[0018] According to the number of grouped records, the plurality of encoded data records are split into at least one data group.

[0019] Optionally, controlling each service group to process the data group in parallel includes:

[0020] Determine the current record from the data set;

[0021] Control each service group to process the current record in parallel, and obtain the processing result corresponding to each service group; wherein the processing result corresponding to the service group includes: the processing record of the current record, and the processing record includes: the record identifier;

[0022] According to the processing results of each service group on the data group, the processing results of the data group are obtained, including:

[0023] According to the record identifiers in the processing records corresponding to the service groups, the processing records corresponding to the service groups are aggregated to obtain the processing results of the current records.

[0024] Optionally, according to the processing results of each data group, the processing results corresponding to the target task are obtained, including:

[0025] Determine the processing information corresponding to the target task;

[0026] According to the processing information, the processing results of each data group are processed to obtain the processing results corresponding to the target task.

[0027] In a second aspect, an embodiment of the present invention provides a task processing device, including:

[0028] A task receiving module, used for receiving a target task and determining a plurality of processing services and target data corresponding to the target task;

[0029] A service splitting module, used to split a plurality of processing services into at least one service group; wherein the service group includes at least one processing service;

[0030] A data splitting module, used for splitting the target data into at least one data group;

[0031] A control module is used to control each service group to process the data group in parallel for each data group; and obtain the processing result of the data group according to the processing result of each service group on the data group;

[0032] The result determination module is used to obtain the processing result corresponding to the target task according to the processing results of each data group.

[0033] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0034] one or more processors;

[0035] The storage device is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the method of any of the above embodiments.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the method of any of the above embodiments is implemented.

[0037] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which implements the method of any of the above embodiments when the computer program is executed by a processor.

[0038] An embodiment of the above invention has the following advantages or beneficial effects: multiple processing services of the target task are split into at least one service group. The target data of the target task is split into at least one data group. Each service group is controlled to process each data group in parallel, and the processing result of the data group is obtained according to the processing result of each service group on the data group. Then, according to the processing result of each data group, the processing result corresponding to the target task is obtained. There is no need to ensure a one-to-one correspondence between the data part and the computing power part. By controlling each service group to process each data group in parallel, data processing in various business scenarios can be performed flexibly and efficiently.

[0039] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0041] Figure 1 is a schematic diagram of a process of a task processing method provided by an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of a process of a task processing method provided by another embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of a process of a task processing method provided by another embodiment of the present invention;

[0044] Figure 4is a schematic diagram of the architecture of a task processing system provided by an embodiment of the present invention;

[0045] Figure 5 is a schematic diagram of a process of a task processing method provided by yet another embodiment of the present invention;

[0046] Figure 6 is a structural schematic diagram of a task processing device provided by an embodiment of the present invention;

[0047] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0049] It should be noted that the acquisition, storage, use, and processing of data in the technical solutions of the embodiments of the present invention are in compliance with the relevant provisions of national laws and regulations.

[0050] Figure 1 FIG. 1 is a schematic diagram of a task processing method provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0051] Step 101: Receive a target task, and determine a plurality of processing services and target data corresponding to the target task.

[0052] The target task can be set according to specific needs. The target task can correspond to multiple processing services. For example, if the target task is a map processing task, the processing services corresponding to the target task include: labeling service and address segmentation service. The labeling service is used to mark whether it is a home address or a company address. The address segmentation service is used to split a complete address into provinces, cities, districts, counties, streets, POIs (Point of Interest), buildings, units, house numbers, etc.

[0053] For example, if the target task is a user login task, the processing services corresponding to the target task include: security verification service, page rendering service, reminder service, and portrait service. The security verification service is used to perform security verification on the user's identity and login environment. The page rendering service is used to render and display the user's login page. The reminder service is used to remind the user to complete the unprocessed process. The portrait service is used to analyze and build user portraits to facilitate subsequent business processing.

[0054] The target data corresponding to the target task can be sent to the target system. The target system can also obtain the target data from the data source according to the task information of the target task.

[0055] Step 102: Split a plurality of processing services into at least one service group; wherein the service group includes at least one processing service.

[0056] Each processing service in the same service group has an association relationship. Services in different service groups do not have an association relationship to ensure that each service group can operate without affecting each other. A first service has an association relationship with a second service, which indicates that the first service needs to directly or indirectly use the processing result of the second service to process information, or the second service needs to directly or indirectly use the processing result of the first service to process information.

[0057] For example, the target task is a map processing task, and the processing services corresponding to the target task include: labeling service and address segmentation service. The labeling service and the address segmentation service are not associated, so they can be split into two service groups, the first service group includes the labeling service, and the second service group includes the address segmentation service.

[0058] For another example, the target task is a user login task, and the processing services corresponding to the target task include: security verification service, page rendering service, reminder service, and portrait service. The page rendering service can only be executed if the user passes the security verification service. Therefore, the security verification service and the page rendering service are associated. It can be divided into three service groups. The first service group includes the security verification service and the page rendering service, the second service group includes the reminder service, and the third service group includes the portrait service.

[0059] Step 103: Split the target data into at least one data group.

[0060] The target data may be split into at least one data group according to demand or human designation. The target data may also be split into at least one data group by: determining the number of group records; and splitting the plurality of encoded data records into at least one data group according to the number of group records. The data group includes data records of the number of group records.

[0061] The number of group records can be 1, 10, 1000, etc. The number of group records can be manually specified or dynamically determined based on the remaining resources of the server, the amount of data, the probability of error, etc. Generally, the greater the probability of error, the smaller the number of group records. The smaller the probability of error, the larger the number of group records, so as to reduce the resource cost of re-execution in case of execution error.

[0062] Step 104: for each data group, control each service group to process the data group in parallel; obtain the processing result of the data group according to the processing result of each service group on the data group.

[0063] Since services in different service groups are not associated, control each service group to process data groups in parallel to speed up the execution efficiency of the target task. When each service group processes data separately, it can dynamically increase or decrease computing resources through containerization technology according to its actual operation situation and business timeliness requirements to achieve elastic processing.

[0064] The processing result of the data group is obtained according to the processing result of each service group on the data group. Specifically, the processing result corresponding to the service group includes multiple processing records, and each processing record includes: a record identifier. According to the record identifier in each processing record, the multiple processing records corresponding to each service group are aggregated to obtain the processing result of the data group.

[0065] Aggregation processing may include: merging processing, numerical calculation processing, etc. Merging processing is to merge processing records from different service groups with the same record identifier into one processing record. Numerical calculation processing is to calculate the numerical fields in the processing records using preset formulas or operation rules.

[0066] Step 105: Obtain the processing result corresponding to the target task according to the processing result of each data group.

[0067] The processing results of each data group can be directly merged to obtain the processing result corresponding to the target task. The processing result corresponding to the target task can also be obtained based on the processing information. Specifically, the processing information corresponding to the target task is determined; according to the processing information, the processing results of each data group are processed to obtain the processing result corresponding to the target task.

[0068] The processing information may be set according to business requirements. The processing information may include: execution time information, target field, processing method of the target field, etc. The execution time information is used to represent the time information or frequency information of executing step 105, etc. Performing information processing on the processing results of each data group includes: performing information processing corresponding to the processing method of the target field on the target field in the processing record in the processing results of each data group.

[0069] In the solution of the embodiment of the present invention, multiple processing services of the target task are divided into at least one service group. The target data of the target task is divided into at least one data group. Each service group is controlled to process each data group in parallel, and the processing result of the data group is obtained according to the processing result of each service group on the data group. Then, the processing result corresponding to the target task is obtained according to the processing result of each data group.

[0070] The solutions in the prior art need to ensure that the data part corresponds to the computing power part. However, the amount of data, computing power resources, and processing services will change. The solution of the embodiment of the present invention does not need to ensure that the data part corresponds to the computing power part. By controlling each service group to process each data group in parallel, data processing in various business scenarios can be performed flexibly and efficiently.

[0071] Figure 2 FIG. 1 is a schematic diagram of a task processing method according to another embodiment of the present invention. Figure 2 As shown, the method includes:

[0072] Step 201: Receive a target task, and determine a plurality of processing services and target data corresponding to the target task.

[0073] Step 202: Obtain association information of each processing service; split the multiple processing services into at least one service group according to the association information; wherein each processing service in the service group has an association relationship.

[0074] The association information represents the connection information between the services of the processing service. The association information may include: input parameters, output parameters, upstream services, downstream services, etc. According to the association information of each processing service, the upstream and downstream relationship of each processing service is determined. If two processing services have an upstream and downstream relationship, the two processing services have an association relationship.

[0075] For example, if the input parameter of the first service is the output parameter of the second service, and the input parameter of the second service is the output parameter of the third service, then the second service and the third service are upstream services of the first service, and the first service, the second service, and the third service have an associated relationship, and the first service, the second service, and the third service need to be divided into the same service group.

[0076] Step 203: Split the target data into at least one data group.

[0077] Step 204: for each data group, control each service group to process the data group in parallel; obtain the processing result of the data group according to the processing result of each service group on the data group.

[0078] Step 205: Obtain the processing result corresponding to the target task according to the processing result of each data group.

[0079] In the solution of the embodiment of the present invention, multiple processing services are split into at least one service group according to the association information of each processing service. Each service group is controlled to process each data group in parallel, and the processing result of the data group is obtained, and the processing result corresponding to the target task is obtained. The solution of the embodiment of the present invention can effectively improve the execution efficiency of the task.

[0080] Figure 3 FIG. 1 is a schematic diagram of a task processing method provided by another embodiment of the present invention. Figure 3 As shown, the method includes:

[0081] Step 301: Receive a target task, and determine a plurality of processing services and target data corresponding to the target task; wherein the target data includes: a plurality of data records.

[0082] Step 302: Split a plurality of processing services into at least one service group; wherein the service group includes at least one processing service.

[0083] Step 303: For each data record, perform encoding processing on the data record so that the encoded data record includes its corresponding record identifier.

[0084] The target data includes: a plurality of data records. The encoding process is used to determine the record identifier of each data record. The record identifier is used to uniquely identify the data record.

[0085] Step 304: Split the target data into at least one data group.

[0086] Determine the number of group records; split the plurality of encoded data records into at least one data group according to the number of group records. The number of group records can be manually specified or dynamically determined according to the remaining resources of the server, the amount of data, the probability of error, etc. Each data record of the data group stores a record identifier of the data record.

[0087] Step 305: for each data group, control each service group to process the data group in parallel; according to the record identifier in the processing record of the processing result of each service group, aggregate the processing records corresponding to each service group to obtain the processing result of the data group.

[0088] Determine the current record from the data group; control each service group to process the current record in parallel, and obtain the processing result corresponding to each service group; wherein the processing result corresponding to the service group includes: a processing record of the current record, and the processing record includes: a record identifier. The processing result corresponding to the service group includes multiple processing records. The processing record corresponds to the data record one by one. The processing record retains the processing record in the data record corresponding to the processing record.

[0089] According to the record identifiers in the processing records corresponding to each service group, the processing records corresponding to each service group are aggregated to obtain the processing result of the current record. Aggregation processing may include: merging processing, numerical calculation processing, etc. Merging processing is to merge processing records from different service groups with the same record identifier into one processing record. Numerical calculation processing is to calculate the numerical fields in the processing records using preset formulas or operation rules.

[0090] Step 306: Obtain the processing result corresponding to the target task according to the processing result of each data group.

[0091] You can directly merge the processing results of each data group to obtain the processing results corresponding to the target task. You can also perform statistical analysis and other operations on the processing results of each data group to obtain the processing results corresponding to the target task. Statistical analysis can be set according to business needs, such as statistical index values ​​of a certain user group, index values ​​of a certain business, etc.

[0092] In the scheme of the embodiment of the present invention, before processing the target data, the data record is encoded. Each service group is controlled to process the data group in parallel to obtain the processing result corresponding to the service group. According to the processing result of each service group on the data group, the processing result of the data group is obtained, and the processing result corresponding to the target task is obtained. The processing result corresponding to the service group and the processing result of the data group both contain a record identifier. Using the record identifier, the aggregation processing of the processing results is completed, thereby ensuring the smooth execution of the target task.

[0093] Figure 4 FIG. 1 is a schematic diagram of the architecture of a task processing system provided by an embodiment of the present invention. Figure 4 As shown, the solution of the embodiment of the present invention relies on the message middleware MQ (Message queue) and KV (Key-Value) storage to build a batch task center. The target data is transmitted to the batch task center in some way, such as Http file upload, and the batch task center initiates task processing.

[0094] The batch task center is used to split multiple processing services into at least one service group; split the target data into at least one data group; for each data group, control each service group to process the data group in parallel; obtain the processing result of the data group based on the processing result of each service group on the data group, etc.

[0095] The batch task center is built with uncoupled service groups and middleware. Different businesses or different needs can rely on the batch task center to execute large-scale time-consuming tasks.

[0096] In addition to implementing the core services for initiating batch task processing, the batch task center also provides performance monitoring, visual dashboards, and exception alarms. For example, if it is found that the processing speed of a certain business is too slow or the task backlog is too heavy, the computing resources for processing the business can be dynamically increased. The batch task center also issues exception alarms, which are used to notify operation and maintenance or R&D personnel when an exception occurs, and solve related problems through manual intervention.

[0097] The core of the solution of the embodiment of the present invention is to build a batch task center. Each business relies on the batch task center to realize the processing of each task. The following takes the map processing task as the target task and the number of group records as 1 as an example to illustrate the solution of the embodiment of the present invention. Figure 5 FIG. 1 is a schematic diagram of a task processing method according to another embodiment of the present invention. Figure 5 As shown, assuming that there are 1 billion pieces of address data, it is necessary to perform labeling services and address segmentation services on the 1 billion pieces of address data respectively. The 1 billion pieces of address data are the target data in the embodiment of the present invention.

[0098] The tagging service and the address segmentation service are not related. Therefore, the tagging service and the address segmentation service are divided into two service groups: business one service and business two service. Business one service includes: tagging service. Business two service includes: address segmentation service.

[0099] The address data can be imported into the Elasticsearch bound to the batch task center through offline scripts or other methods. There are many attributes of address data. For the convenience of description, only the address and longitude and latitude are retained in the data structure of the address data. The data records in the address data can be "X Province Y City Z Street A Community 1-101, 10.1, 101.1", "B City Economic Development Zone X Community East Gate", "A Province B City C Village Residents Committee, 10.3, 101.3", etc.

[0100] The batch task center will traverse and read the 1 billion address data in Elasticsearch, and perform data pre-processing on the address data. Pre-processing may include: data verification, encoding processing, etc. Encoding processing is to configure a unique record identifier for each data record and update the record identifier to each data record. The data records after encoding processing are shown in Table 1.

[0101]

[0102] Since the number of group records is 1, each data record is regarded as a data group. Each data corresponds to a message, and the message is sent to the message middleware. For the fields that are not involved in the subsequent calculation, they are filtered to generate the message body. The data structure of the message body is shown in Table 2.

[0103]

[0104] Each service group listens to the message middleware and executes its own tasks after receiving the message. Through containerization technology, computing resources can be dynamically increased or decreased according to actual performance, time consumption and business needs, achieving elastic scalability to handle time-consuming tasks.

[0105] After each service group completes processing, it returns the processing results to the batch task center through the message middleware. The processing results include processing records of multiple data records, and each processing record includes its corresponding record identifier. The batch task center consumes messages and aggregates the processing results and data records returned by each service group through the record identifier to obtain the processing results of each data record, and stores the processing results of each data record in Elasticsearch.

[0106]

[0107] Table 3 shows the processing results of Business 1 service and Business 2 service. The first three processing records in Table 3 are the processing results of Business 2 service. Business 2 service includes: address segmentation service. The first three processing records in Table 3 all contain address segmentation results. The last three processing records in Table 3 are the processing results of Business 1 service. Business 1 service includes: tagging service. The last three processing records in Table 3 all contain tagging results. Among them, c represents the home address and b represents the company address.

[0108] After the service group completes processing of each data record, it uploads the processing results to the batch task center through the message middleware for update and storage. The batch task center aggregates the processing results and data records returned by each service group through the record identifier to obtain the processing results of each data record. When reading data, all processing results of a data record can be read in full according to the record identifier.

[0109] The above description is that after each service group completes the processing, the processing results are sent to the message middleware. The batch task center consumes the messages in the message middleware and saves the processing results. Other middleware can also monitor the processing messages of the batch task center and save the processing results, reducing the number of data transfers and reducing the coupling of the system.

[0110] The number of group records in the solution of the embodiment of the present invention is set to 1, that is, each data group includes only one data record, and the minimum processing unit is one data record. If a data record fails to be processed, the problem of processing failure can be solved through the retry consumption of the message middleware, ensuring the stability of the overall processing flow. In addition, when each service group performs data processing separately, it can dynamically increase or decrease computing resources through containerization technology according to its actual operating conditions and in combination with the timeliness requirements of the business to achieve elastic processing.

[0111] Figure 6 FIG. 1 is a schematic diagram of a task processing device provided by an embodiment of the present invention. Figure 6 As shown, the device comprises:

[0112] The task receiving module 601 is used to receive a target task and determine a plurality of processing services and target data corresponding to the target task;

[0113] The service splitting module 602 is used to split the multiple processing services into at least one service group; wherein the service group includes at least one processing service;

[0114] A data splitting module 603 is used to split the target data into at least one data group;

[0115] The control module 604 is used to control each service group to process the data group in parallel for each data group; and obtain the processing result of the data group according to the processing result of each service group on the data group;

[0116] The result determination module 605 is used to obtain the processing result corresponding to the target task according to the processing result of each data group.

[0117] Optionally, the service splitting module 602 is specifically used for:

[0118] Get the associated information of each processing service;

[0119] According to the association information, the multiple processing services are divided into at least one service group; wherein each processing service in the service group has an association relationship.

[0120] Optionally, the target data includes: a plurality of data records;

[0121] The device also includes:

[0122] The encoding module is used to encode each data record so that the encoded data record includes its corresponding record identifier.

[0123] Optionally, the data splitting module 603 is specifically used for:

[0124] Determine the number of group records;

[0125] According to the number of grouped records, the plurality of encoded data records are split into at least one data group.

[0126] Optionally, the control module 604 is specifically configured to:

[0127] Determine the current record from the data set;

[0128] Control each service group to process the current record in parallel, and obtain the processing result corresponding to each service group; wherein the processing result includes: the processing record of the current record, and the processing record includes: the record identifier;

[0129] According to the record identifiers in the processing records corresponding to the service groups, the processing records corresponding to the service groups are aggregated to obtain the processing results of the current records.

[0130] Optionally, the result determination module 605 is specifically used to:

[0131] Determine the processing information corresponding to the target task;

[0132] According to the processing information, the processing results of each data group are processed to obtain the processing results corresponding to the target task.

[0133] An embodiment of the present invention provides an electronic device, including:

[0134] one or more processors;

[0135] The storage device is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the method of any of the above embodiments.

[0136] An embodiment of the present invention provides a computer program product, including a computer program. When the computer program is executed by a processor, the method of any of the above embodiments is implemented.

[0137] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0138] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0139] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0140] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0141] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0142] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be arranged in a processor, for example, they may be described as: a task receiving module, a service splitting module, a data splitting module, a control module and a result determination module. The names of these modules do not constitute limitations on the modules themselves in certain circumstances, for example, the task receiving module may also be described as "a module for receiving a target task and determining a plurality of processing services and target data corresponding to the target task".

[0144] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0145] receiving a target task, and determining a plurality of processing services and target data corresponding to the target task;

[0146] Splitting the plurality of processing services into at least one service group; wherein the service group includes at least one of the processing services;

[0147] Splitting the target data into at least one data group;

[0148] For each of the data groups, controlling each of the service groups to process the data group in parallel; obtaining a processing result of the data group according to a processing result of each of the service groups on the data group;

[0149] According to the processing results of each of the data groups, the processing results corresponding to the target task are obtained.

[0150] According to the technical solution of the embodiment of the present invention, multiple processing services of the target task are split into at least one service group. The target data of the target task is split into at least one data group. Each service group is controlled to process each data group in parallel, and the processing result of the data group is obtained according to the processing result of each service group on the data group. Then, according to the processing result of each data group, the processing result corresponding to the target task is obtained. There is no need to ensure a one-to-one correspondence between the data part and the computing power part. By controlling each service group to process each data group in parallel, data processing in various business scenarios can be performed flexibly and efficiently.

[0151] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A task processing method, characterized in that: include: receiving a target task, and determining a plurality of processing services and target data corresponding to the target task; Splitting the plurality of processing services into at least one service group; wherein the service group includes at least one of the processing services; Splitting the target data into at least one data group; For each of the data groups, controlling each of the service groups to process the data group in parallel; obtaining a processing result of the data group according to a processing result of each of the service groups on the data group; According to the processing results of each of the data groups, the processing results corresponding to the target task are obtained.

2. The method according to claim 1, characterized in that The step of splitting the plurality of processing services into at least one service group comprises: Obtaining the associated information of each of the processing services; According to the association information, the multiple processing services are divided into at least one service group; wherein each processing service in the service group has an association relationship.

3. The method according to claim 1, characterized in that The target data includes: a plurality of data records; After determining the multiple processing services and target data corresponding to the target task, the method further includes: For each of the data records, encoding processing is performed on the data record so that the encoded data record includes its corresponding record identifier.

4. The method according to claim 3, characterized in that The step of splitting the target data into at least one data group comprises: Determine the number of group records; According to the number of grouped records, the plurality of encoded data records are divided into at least one data group.

5. The method according to claim 3, characterized in that: The controlling each of the service groups to process the data group in parallel includes: determining a current record from the data set; Controlling each of the service groups to process the current record in parallel, and obtaining processing results corresponding to each of the service groups; wherein the processing results corresponding to the service groups include: a processing record of the current record, and the processing record includes: a record identifier; The obtaining the processing result of the data group according to the processing result of each service group on the data group comprises: According to the record identifiers in the processing records corresponding to the service groups, the processing records corresponding to the service groups are aggregated to obtain the processing results of the current records.

6. The method according to claim 1, characterized in that: The step of obtaining the processing result corresponding to the target task according to the processing result of each of the data groups includes: Determining processing information corresponding to the target task; According to the processing information, information processing is performed on the processing results of each of the data groups to obtain the processing results corresponding to the target task.

7. A task processing device, characterized in that: include: A task receiving module, used for receiving a target task and determining a plurality of processing services and target data corresponding to the target task; A service splitting module, used for splitting the plurality of processing services into at least one service group; wherein the service group includes at least one of the processing services; A data splitting module, used for splitting the target data into at least one data group; A control module, for controlling each of the service groups to process the data groups in parallel; and obtaining a processing result of the data group according to the processing result of each of the service groups on the data group; The result determination module is used to obtain the processing result corresponding to the target task according to the processing results of each of the data groups.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images