A data processing method and device

Abstract

The embodiment of the present invention provides a data processing method and device, which is applicable to a server with a central processing unit with streaming processing capability and a graphics processor with parallel processing capability. The method includes: for each processing logic in the business system, the central processing unit converts the processing logic that can be processed in parallel into a computing structure applicable to the graphics processor, and sends each computing structure to the graphics processor; for each computing structure, the central processing unit sends the business data of the computing structure received from the business system to the graphics processor; for each computing structure, the graphics processor performs parallel processing on the business data corresponding to the computing structure according to the computing structure, and sends the processing result to the central processing unit. The above method can reduce monitoring delay, reduce monitoring server resource loss, and save computing costs.

Description

Technical Field

[0001] The present application relates to the field of network technology, and in particular to a data processing method and device. Background Art

[0002] With the business development and technological iteration of the enterprise, the current Information Center has realized real-time monitoring functions based on various data sources such as database flow, network traffic, system indicators, application logs, etc., covering about 200 core key systems, key application components, and common operation systems, with a total of 37,000 monitoring objects.

[0003] In the prior art, business data of business systems are generally monitored and processed through various general stream computing frameworks, such as Apache Storm, Apache Samza, etc., based on the distributed architecture of the host's CPU (central processing unit). This method can monitor business data in the business systems of various enterprises, institutions or organizations to ensure the healthy operation and development of business systems. However, with the development of technology and the rapid increase in business data, this monitoring and computing method has a large monitoring delay, large server resource loss, and high computing cost.

[0004] Therefore, there is an urgent need for a data processing method and device that can reduce monitoring delay, reduce monitoring server resource loss, and save computing costs. Summary of the invention

[0005] The embodiments of the present invention provide a data processing method and device, which can reduce monitoring delay, reduce monitoring server resource loss, and save computing costs.

[0006] In a first aspect, an embodiment of the present invention provides a data processing method, which is applicable to a server having a central processing unit with streaming processing capability and a graphics processing unit with parallel processing capability, and the method comprises:

[0007] For each processing logic in the business system, the central processing unit converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, and sends each computing structure to the graphics processor;

[0008] For each computing structure, the central processor sends the business data of the computing structure received from the business system to the graphics processor;

[0009] For each computing structure, the image processor processes the business data corresponding to the computing structure in parallel according to the computing structure, and sends the processing result to the central processing unit.

[0010] In the above method, for each processing logic in the business system, the central processor converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, and sends each computing structure to the graphics processor. In this way, the central processor converts each processing logic in the business system that is only applicable to itself into a computing structure suitable for the graphics processor, so that the graphics processor receives a recognizable computing structure and can execute the computing structure to implement the corresponding processing function. Further, for each computing structure, the central processor sends the business data of the computing structure received from the business system to the graphics processor. In this way, the graphics processor receives the business data and processes the business data according to the computing structure corresponding to the business data that has been received to obtain the processing result. Compared with the streaming processing capability of the central processor in the prior art (a single central processor can only execute processing logic in a single process at the same time), the present application can realize parallel "execution of processing logic" by executing the computing result through the graphics processor, and there is no need to rewrite the code executed on the central processor into code adapted to the graphics processor, thereby reducing labor costs, improving processing efficiency, and reducing monitoring delay.

[0011] Optionally, the central processing unit converts the processing logic that can be processed in parallel into a computing structure suitable for a graphics processor, including:

[0012] For any processing logic, the central processing unit constructs a semantic tree of the processing logic;

[0013] The central processing unit traverses the semantic tree, extracts data objects, operators and coefficients corresponding to the data objects in the semantic tree, thereby constructing a calculation structure;

[0014] If the calculation structure has coefficients for each data object, the calculation structure is suitable for parallel processing by a graphics processor.

[0015] In the above method, the central processing unit obtains the processing logic from the memory, extracts the data objects, the coefficients of the data objects and the operators between the data objects from the processing logic, and generates a semantic tree with the data objects, the coefficients of the data objects and the data objects as nodes. Further, the coefficients corresponding to the data objects, operators and data objects in the calculation structure are obtained through semantic tree analysis, and then a complete calculation structure recognizable by the graphics processor is further constructed according to the semantic tree to improve the processing efficiency of the graphics processor and the accuracy of the calculation results. If the calculation structure has the coefficients of each data object, it is determined that the calculation structure is suitable for parallel processing of the graphics processor, otherwise the business data of the calculation structure can be processed by the central processing unit. In this way, it is ensured that the business data of all processing logics can obtain the processing results, further ensuring the reliability of the processing function.

[0016] Optionally, the central processing unit traverses the semantic tree, extracts data objects, operators and coefficients corresponding to the data objects in the semantic tree, and thereby constructs a calculation structure, including:

[0017] The central processing unit traverses the semantic tree, and for each layer in the semantic tree, extracts the operator, data object and operation result of the layer by using a quadruple method;

[0018] The calculation structure is obtained by determining the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator through the quaternary formula constructed in each layer.

[0019] In the above method, the central processing unit traverses the semantic tree, and for each layer in the semantic tree, extracts the operator, data object and operation result of the layer by means of a quadruple, so as to realize the quadruple constructed according to the operator, data object and operation result corresponding to each layer node in the semantic tree, determine the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator, and obtain the calculation structure. In this way, the central processing unit transforms the processing logic suitable for the central processing unit stored in itself into a calculation structure in the form of a matrix that can be recognized by the graphics processor. After the subsequent graphics processor receives the calculation structure and the corresponding business data, the corresponding data in the business data is filled in the corresponding data object in the calculation structure to obtain the data calculation structure, so that the graphics processor directly calculates according to the data calculation structure matrix. Compared with the manpower cost of rewriting the code executed on the CPU (central processing unit) into the code adapted to the GPU (graphics processing unit) in the prior art, the present application can simply and quickly rewrite the code executed on the CPU into the GPU calculation structure matrix.

[0020] Optionally, the calculation structure is obtained by determining the coefficient matrix of the data object, the data object matrix and the matrix operation method of the operator through the quaternary formula constructed at each layer, including:

[0021] For the quadruple constructed in each layer, if the quadruple has data objects with the same attributes or data objects with the same attributes and calculation results, the data objects with the same attributes or the data objects with the same attributes and the calculation results are calculated according to the operator in the quadruple to obtain a simplified result and the quadruple is deleted, and the simplified result is set to the corresponding position in the lower layer quadruple to obtain simplified quadruple; through the simplified quadruple, the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator are determined to obtain the calculation structure.

[0022] In the above method, according to the operator in the quadruple, the data objects with the same attributes or the data objects with the same attributes and the operation result are operated to obtain a simplified result and the quadruple is deleted, and the simplified result is set to the corresponding position in the lower quadruple, thereby obtaining the simplified quadruple. In this way, the final quadruple is the most simplified quadruple without repeated attribute objects, which is convenient for obtaining the coefficient matrix of the data object, the data object matrix and the matrix operation method that meets the operator, and further obtaining the calculation structure. In this way, the accuracy and simplicity of the calculation structure are improved.

[0023] Optionally, the central processor sends the business data of the computing structure received from the business system to the graphics processor, including:

[0024] The central processor sends the business data of the computing structure received from the business system to the graphics processor after the data volume threshold is met.

[0025] In the above method, a data volume threshold is set, and when the data volume of the received business data of the computing structure is not less than the data volume threshold, the central processing unit sends the received business data of the computing structure to the graphics processing unit. In this way, a large amount of time loss in the process of CPU to GPU transmission caused by the small data volume and high data transmission frequency in the streaming computing is eliminated.

[0026] Optionally, it also includes: if the business data of the computing structure received by the central processor from the business system does not meet the data volume threshold and the time length for the central processor to receive the business data of the computing structure is greater than a preset time threshold, the business data of the computing structure is sent to the graphics processor.

[0027] In the above method, in order to ensure the validity of the processing results, a preset time threshold is set, that is, if the time interval between the earliest collection time of the business data in the business data of the computing structure and the current time exceeds the preset time threshold, in order to ensure the validity of the processing results of the earliest collected business data, the business data of the computing structure should be sent to the graphics processor immediately to obtain the calculation results as soon as possible.

[0028] Optionally, the data volume threshold is obtained in the following manner, including: the central processing unit determines the data volume threshold based on a limit time, a data calculation time, a first time and a second time, the limit time being the effective time of the calculation result of the business data, the data calculation time being the time for the graphics processor to process the business data to obtain the calculation result, the first time being the time for the graphics processor to receive and store unit data, and the second time being the time for the central processing unit to receive and store unit data.

[0029] In the above method, the limit time is obtained according to the validity of the processing result, and the data volume threshold is obtained according to the limit time and the time length of the graphics processor processing the business data to obtain the calculation result, the time length of the graphics processor receiving and storing the unit data, and the time length of the central processor receiving and storing the unit data. The determination of the limit time takes into account the data collection speed and data transmission speed of the central processor. In this way, the validity of the processing result is guaranteed.

[0030] Optionally, it also includes: the central processor performs streaming processing on the business data corresponding to the non-computing structure received from the business system to obtain a processing result.

[0031] In the above method, if the central processor cannot convert the processing logic into a computing structure, the central processor performs streaming processing on the business data of the processing logic to obtain the processing result, thereby ensuring the reliability of the host processing function.

[0032] In a second aspect, an embodiment of the present invention provides a data processing device, which is applicable to a server having a central processing unit with streaming processing capability and a graphics processing unit with parallel processing capability, and the device includes:

[0033] A processing module, for each processing logic in the business system, the central processing unit converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, and sends each computing structure to the graphics processor through the transceiver module;

[0034] The transceiver module is also used for, for each computing structure, the central processor to send the business data of the computing structure received from the business system to the graphics processor;

[0035] The processing module is also used for, for each computing structure, the image processor to perform parallel processing on the business data corresponding to the computing structure according to the computing structure, and to send the processing result to the central processing unit through the transceiver module.

[0036] In a third aspect, an embodiment of the present application further provides a computing device, comprising: a memory for storing programs; a processor for calling the programs stored in the memory, and executing the methods described in the various possible designs of the first aspect according to the obtained programs.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable non-volatile storage medium, comprising a computer-readable program. When a computer reads and executes the computer-readable program, the computer executes the method described in the various possible designs of the first aspect.

[0038] These implementations or other implementations of the present application will be more concise and understandable in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 A schematic diagram of the architecture of a data processing system provided by an embodiment of the present invention;

[0041] Figure 2 A flowchart of a data processing method provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of a semantic tree provided by an embodiment of the present invention;

[0043] Figure 4 A flowchart of a data processing method provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a data processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Figure 1A system architecture of a data processing system provided for an embodiment of the present invention, wherein the business server 101 can be any business server in the business system, and the business system can be run based on a business server cluster. The data processing server 102 can be any data processing server in the data processing system, and the data processing system can be run based on a data processing server cluster. The physical hardware architecture on which the business system or the data processing system is based is not specifically limited. In addition, the business server 101 can also be a client in a possible design, and the data processing server 102 collects the business data of the client. The business server 101 here indicates an example for clearly describing this solution, but does not limit the specific implementation of this solution. The data processing server 102 includes a central processing unit and a graphics processing unit, and the memory of the data processing server 102 includes processing logic corresponding to the business data. The data processing server 102 obtains each processing logic from the memory, constructs each business processing logic into a semantic tree, and obtains the corresponding quadruple and the attribute type of each element in the quadruple (the element here refers to the data object, the coefficient corresponding to the data object, and the operator) according to each layer of the semantic tree. Further, according to the quadruple corresponding to the nodes at each layer of the semantic tree and the same attribute elements in the quadruple, the quadruple is simplified to obtain a simplified quadruple. According to the simplified quadruple, the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator are determined to obtain a calculation structure. The data processing server 102 sends the calculation structure to the graphics processor. When the business data corresponding to the calculation structure is not less than the data volume threshold, the data processing server 102 also sends the business data corresponding to the calculation structure to the graphics processor. The graphics processor processes the business data according to the calculation structure to obtain a processing result, and returns the processing result to the central processing unit. When the business data corresponding to the calculation structure is less than the data volume threshold, the data processing server 102 also determines the time interval between the collection time of the earliest collected business data in the business data of the current calculation result and the current time. If the time interval exceeds the preset time threshold, in order to ensure the validity of the processing result of the business data of the calculation structure, the business data corresponding to the calculation structure is still sent to the graphics processor to obtain the processing result. In addition, if the central processing unit cannot obtain the calculation structure according to the processing logic, the central processing unit processes the business data corresponding to the processing logic through the streaming processing capability to obtain the processing result.

[0047] Based on this, the embodiment of the present application provides a data processing method flow, which is applicable to a server with a central processing unit with streaming processing capability and a graphics processing unit with parallel processing capability, such as Figure 2 As shown, including:

[0048] Step 201: For each processing logic in the business system, the central processing unit converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, and sends each computing structure to the graphics processor;

[0049] Here, the business data may be data in a business processing request. For example, if the business processing request is a transfer request, the account balances of the transferor and the transferee may be calculated based on the transfer amount. It may also be business operation and maintenance data. When the business data is business operation and maintenance data, the data processing server in the system architecture may be used for monitoring and alarming business operation and maintenance. For example, if the business operation and maintenance data includes daily transaction volume, transaction success rate, etc., the data processing server in the system architecture may generate an alarm when the daily transaction volume exceeds a preset threshold or the transaction success rate is lower than a preset threshold. That is, there is no specific restriction on the type of business data here, and the corresponding processing function of the corresponding data processing server may be set accordingly according to the business data.

[0050] Step 202: For each computing structure, the central processor sends the business data of the computing structure received from the business system to the graphics processor;

[0051] Step 203: For each computing structure, the image processor processes the business data corresponding to the computing structure in parallel according to the computing structure, and sends the processing result to the central processor.

[0052] In the above method, for each processing logic in the business system, the central processor converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, and sends each computing structure to the graphics processor. In this way, the central processor converts each processing logic in the business system that is only applicable to itself into a computing structure suitable for the graphics processor, so that the graphics processor receives a recognizable computing structure and can execute the computing structure to implement the corresponding processing function. Further, for each computing structure, the central processor sends the business data of the computing structure received from the business system to the graphics processor. In this way, the graphics processor receives the business data and processes the business data according to the computing structure corresponding to the business data that has been received to obtain the processing result. Compared with the streaming processing capability of the central processor in the prior art (a single central processor can only execute processing logic in a single process at the same time), the present application can realize parallel "execution of processing logic" by executing the computing result through the graphics processor, and there is no need to rewrite the code executed on the central processor into code adapted to the graphics processor, thereby reducing labor costs, improving processing efficiency, and reducing monitoring delay.

[0053] The embodiment of the present application provides a data processing method, wherein the central processing unit converts a processing logic that can be processed in parallel into a computing structure suitable for a graphics processor, including:

[0054] For any processing logic, the central processing unit constructs a semantic tree of the processing logic;

[0055] The central processing unit traverses the semantic tree, extracts data objects, operators and coefficients corresponding to the data objects in the semantic tree, thereby constructing a calculation structure;

[0056] If the calculation structure has coefficients for each data object, the calculation structure is suitable for parallel processing of a graphics processor. That is to say, because the central processing unit is a streaming processing function, the processing efficiency is lower than that of the parallel processing function of the graphics processor. Therefore, in order to improve processing efficiency and reduce monitoring latency (at this time, the business data is monitoring-related data such as business operation and maintenance data), consider transferring the processing tasks in the central processing unit to the graphics processor and utilizing the parallel processing function of the graphics processor. However, if the processing tasks in the central processing unit are transferred to the graphics processor, the graphics processor needs to include the corresponding processing logic, and the processing logic form of the central processing unit is different from the processing logic form of the graphics processor. If the processing logic in the central processing unit is directly set in the graphics processor, the graphics processor cannot recognize and run it. If the processing logic (such as the calculation program code) that can be run by the central processing unit is rewritten as the processing logic that can be run by the graphics processor, it is necessary to rewrite each processing logic in the central processing unit, which requires a lot of manpower and time costs. The above method can be used to construct a semantic tree of the processing logic in the central processing unit, extract data objects, operators and coefficients corresponding to the data objects in the semantic tree, thereby constructing a computing structure that can be recognized and run by the graphics processor. This can realize the use of the parallel processing function of the graphics processor to process business data without consuming manpower and time costs.

[0057] The embodiment of the present application provides a method for obtaining a computing structure, wherein the central processing unit traverses the semantic tree, extracts data objects, operators and coefficients corresponding to the data objects in the semantic tree, and thereby constructs a computing structure, including:

[0058] The central processing unit traverses the semantic tree, and for each layer in the semantic tree, extracts the operator, data object and operation result of the layer by using a quadruple method;

[0059] Through the quaternion constructed at each layer, the coefficient matrix of the data object, the data object matrix and the matrix operation method of the operator are determined to obtain the calculation structure. In one example, the processing logic in the central processing unit is as follows:

[0060]

[0061] The CPU then obtains the corresponding semantic tree based on the above processing logic analysis. Figure 3 As shown, id1 is the identifier of Position, id2 is the identifier of initial, and id3 is the identifier of rate. That is, the CPU obtains id1=id2+id3*60 by performing lexical analysis on the above processing logic, further obtains a tree representing the grammar through grammatical analysis, and further obtains a semantic tree through semantic analysis.

[0062] The CPU also obtains the attributes of each data object, data object coefficient and operator according to the above processing logic to obtain an attribute table. Then, for each layer in the semantic tree, the CPU extracts the operator, data object and operation result of the layer through the quadruple method to obtain the following quadruple (operator, data object 1, data object 2, operation result):

[0063]

[0064] Among them, the quaternion in (1) comes from Figure 3 The first layer node in (2) comes from Figure 3 The second-layer nodes in (3) come from Figure 3 The third-layer node in (4) comes from Figure 3 The fourth layer node in . According to the quaternary formula, the coefficient matrix of the data object, the data object matrix and the matrix operation method that meets the operator are determined to obtain the calculation structure.

[0065] The embodiment of the present application provides a method for obtaining a calculation structure, which determines the coefficient matrix of the data object, the data object matrix and the matrix operation method of the operator through the quaternary formula constructed at each layer, and obtains the calculation structure, including:

[0066] For the quadruple constructed at each layer, if the quadruple has data objects with the same attributes or data objects with the same attributes and operation results, then according to the operator in the quadruple, the data objects with the same attributes or the data objects with the same attributes and the operation results are operated to obtain a simplified result and the quadruple is deleted, and the simplified result is set to the corresponding position in the lower layer quadruple, thereby obtaining each simplified quadruple; through each simplified quadruple, the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator are determined to obtain the calculation structure. Based on the above example, according to the attribute table inttoreal and 60 have the same attributes, then t1=60, and t1=60 is substituted into the quadruple (2) to obtain the quadruple (*id3 60 t2). id1 and t3 have the same attributes, then id1=t3, and id1=t3 is substituted into the quadruple (3) to obtain (+id2 t2 id1). According to the quaternary form (*id3 60 t2) and the quaternary form (+id2 t2 id1), the coefficient matrix (1 60) and the data object matrix of the data object are obtained, and the matrix operation method that conforms to the operator is obtained: If initial=10, rate=1, the result is 10+60=70.

[0067] The embodiment of the present application further provides a method for sending business data, wherein the central processor sends the business data of the computing structure received from the business system to the graphics processor, including:

[0068] The central processor sends the business data of the computing structure received from the business system to the graphics processor after the data volume threshold is met.

[0069] Here, during the data processing process, the amount of business data sent at a single time may be small and the sending frequency may be high. In this case, the time for the central processor to receive and store data and the time for the graphics processor to receive and store data become longer relative to the data processing time. In one example, the time for the central processor to receive and store unit data is t1, and the time for the graphics processor to receive and store data is t2. The time for the graphics processor to process unit data is t3. If each data transmission transmits unit data, the time consumed to process 10 units of data is T=10(t1+t2+t3). If each data transmission transmits 10 units of data, the time consumed to process 10 units of data is T=10t1+t2+t3. This is only used to simply describe the impact of the amount of data transmitted each time on the data processing efficiency, assuming that there are no other influencing factors in the data transmission. Therefore, a data volume threshold is set in the central processor, and when the amount of business data is not less than the data volume threshold, the business data is sent to the graphics processor. In this way, the data processing efficiency is effectively improved and data transmission and reading resources are saved.

[0070] The embodiment of the present application also provides a method for sending service data, further comprising:

[0071] The central processor determines that the business data of the computing structure received from the business system does not meet the data volume threshold and the time length for the central processor to receive the business data of the computing structure is greater than the preset time threshold, then the business data of the computing structure is sent to the graphics processor. In other words, there may be a situation where the data volume of the business data has not reached the data volume threshold, then the business data is in an unprocessed state and there is no corresponding processing result. However, the processing result of the business data is time-limited. If the time limit is exceeded, the corresponding service will be affected. Therefore, in order to prevent this situation from happening, a timeout mechanism is set, that is, even if the data volume of the business data does not exceed the data volume threshold, it is confirmed whether the time interval between the collection time of the earliest collected business data in the current data volume window and the current time exceeds the preset time threshold. If it exceeds, the business data of the data volume window should be sent immediately to ensure that the processing result returned by the graphics processor is valid.

[0072] The embodiment of the present application also provides a method for determining a data volume threshold, wherein the data volume threshold is obtained in the following manner, including:

[0073] The central processor determines the data volume threshold value according to the limit time, the data calculation time, the first time and the second time. The limit time is the effective time of the calculation result of the business data, the data calculation time is the time when the graphics processor processes the business data to obtain the calculation result, the first time is the time when the graphics processor receives and stores the unit data, and the second time is the time when the central processor receives and stores the unit data. In other words, the limit time is obtained by comprehensively considering the effective time of the processing result of the business data and the data flow size of the central processor's collected business data. The time when the graphics processor receives and stores the unit data and the time when the central processor receives and stores the unit data are obtained according to the data transmission rate of the central processor / graphics processor, and the time when the graphics processor processes the business data to obtain the calculation result are obtained. In this way, the size of the data volume threshold value can be accurately determined, so that the processing efficiency can be improved and the validity of the processing results can be guaranteed.

[0074] The embodiment of the present application also provides a data processing method, further comprising: the central processor performs streaming processing on the business data corresponding to the non-computing structure received from the business system to obtain a processing result. In other words, if there is a processing logic that cannot be converted into a computing structure, the central processor processes the corresponding business data according to the processing logic to ensure the reliability of business data processing.

[0075] Based on the above method flow, the present application embodiment provides a data processing method flow, such as Figure 4 As shown, including:

[0076] Step 401: The central processing unit obtains processing logic from the memory.

[0077] Step 402: The central processing unit converts the processing logic into a semantic tree and obtains an attribute table.

[0078] Here, if the central processor cannot convert the processing logic into a semantic tree, the central processor collects the business data corresponding to the processing logic and processes it to obtain the processing result.

[0079] Step 403: The central processing unit obtains the corresponding initial quadruple expressions according to the nodes at each layer of the semantic tree.

[0080] Step 404: The central processing unit simplifies each initial quadruple according to the attribute table to obtain simplified quadruple.

[0081] Step 405: The central processing unit obtains the data object matrix, the coefficient matrix of the data object and the matrix operation method corresponding to the operator according to the simplified quaternary expressions.

[0082] Step 406: The central processing unit constructs a calculation structure using the data object matrix, the coefficient matrix of the data object, and the matrix operation method that conforms to the operator.

[0083] Step 407: The CPU sends the calculation structure to the GPU.

[0084] Step 408: The central processing unit collects business data.

[0085] Step 409: The CPU determines whether the data volume of the service data in the current data volume window exceeds the data volume threshold. If it exceeds, execute step 411; otherwise, execute step 410.

[0086] Here, the data volume window is the business data in the CPU that needs to be sent to the GPU to obtain the processing result, which may include multiple groups of business data, and each group of business data may correspond to at least one processing result.

[0087] Step 410 , the central processing unit determines whether the time interval between the collection time of the earliest collected business data in the current data volume window and the current time exceeds a preset time threshold, if so, execute step 411 , otherwise, execute step 408 .

[0088] Step 411: The central processing unit sends the business data in the data volume window to the graphics processing unit.

[0089] Step 412: After receiving the service data, the graphics processor obtains a processing result according to a calculation structure corresponding to the service data, and returns the processing result to the central processor.

[0090] Step 413: The central processing unit receives the processing result.

[0091] It should be noted that the above process steps are not unique, for example, step 408 can be executed before or after any step in the calculation structure acquisition process from step 401 to step 407. The calculation structure acquisition process from step 401 to step 407 can be executed only once, and the processing logic of the calculation structure remains unchanged, then steps 408 to step 413 can be directly executed.

[0092] Based on the same concept, an embodiment of the present invention provides a data processing device, which is applicable to a server having a central processing unit with streaming processing capability and a graphics processing unit with parallel processing capability. Figure 5 A schematic diagram of a data processing device provided in an embodiment of the present application is shown in FIG. Figure 5 Including:

[0093] The processing module 501 is used for converting the processing logics that can be processed in parallel into a computing structure suitable for the graphics processor for each processing logic in the business system, and sending each computing structure to the graphics processor through the transceiver module 502;

[0094] The transceiver module 502 is also used for, for each computing structure, the central processor to send the business data of the computing structure received from the business system to the graphics processor;

[0095] The processing module 501 is also used for, for each computing structure, the image processor to perform parallel processing on the business data corresponding to the computing structure according to the computing structure, and to send the processing result to the central processor through the transceiver module 502 .

[0096] Optionally, the processing module 501 is specifically used to construct a semantic tree of the processing logic for any processing logic; traverse the semantic tree, extract data objects, operators and coefficients corresponding to the data objects in the semantic tree, so as to construct a calculation structure; if the calculation structure has coefficients of each data object, then the calculation structure is suitable for parallel processing of a graphics processor.

[0097] Optionally, the processing module 501 is specifically used to traverse the semantic tree, and for each layer in the semantic tree, extract the operators, data objects and calculation results of the layer through quadruple method; through the quadruple constructed for each layer, determine the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator to obtain the calculation structure.

[0098] Optionally, the processing module 501 is specifically used to, for each layer of quadruple constructed, if the quadruple has data objects with the same attributes or data objects with the same attributes and calculation results, then according to the operator in the quadruple, the data objects with the same attributes or the data objects with the same attributes and the calculation results are operated to obtain a simplified result and the quadruple is deleted, and the simplified result is set to a corresponding position in the lower layer quadruple, thereby obtaining simplified quadruple formulas; through the simplified quadruple formulas, the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator are determined to obtain the calculation structure.

[0099] Optionally, the transceiver module 502 is specifically configured to send the business data of the computing structure received from the business system to the graphics processor after the business data meets a data volume threshold.

[0100] Optionally, the transceiver module 502 is also used to determine that the business data of the computing structure received from the business system does not meet the data volume threshold and the duration of receiving the business data of the computing structure is greater than a preset time threshold, and then send the business data of the computing structure to the graphics processor.

[0101] Optionally, the data volume threshold is obtained in the following manner: the processing module 501 is specifically used to determine the data volume threshold based on a limit time, a data calculation time, a first time and a second time, the limit time being the effective time of the calculation result of the business data, the data calculation time being the time for the graphics processor to process the business data to obtain the calculation result, the first time being the time for the graphics processor to receive and store unit data, and the second time being the time for the central processing unit to receive and store unit data.

[0102] Optionally, the processing module 501 is further configured to perform stream processing on the business data corresponding to the non-computing structure received from the business system to obtain a processing result.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0104] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: A server applicable to a central processing unit with streaming processing capability and a graphics processing unit with parallel processing capability; the method comprising: For each processing logic in the business system, the central processing unit converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, and sends each computing structure to the graphics processor; For each computing structure, the central processor sends the business data of the computing structure received from the business system to the graphics processor; For each computing structure, the graphics processor processes the business data corresponding to the computing structure in parallel according to the computing structure, and sends the processing result to the central processing unit; the central processing unit converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, including: For any processing logic, the central processing unit constructs a semantic tree of the processing logic; The central processing unit traverses the semantic tree, extracts data objects, operators and coefficients corresponding to the data objects in the semantic tree, thereby constructing a calculation structure; If the calculation structure has coefficients for each data object, the calculation structure is suitable for parallel processing by a graphics processor.

2. The method according to claim 1, characterized in that The central processing unit traverses the semantic tree, extracts data objects, operators and coefficients corresponding to the data objects in the semantic tree, thereby constructing a calculation structure, including: The central processing unit traverses the semantic tree, and for each layer in the semantic tree, extracts the operator, data object and operation result of the layer by using a quadruple method; The calculation structure is obtained by determining the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator through the quaternary formula constructed in each layer.

3. The method as claimed in claim 2, characterized in that The calculation structure is obtained by determining the coefficient matrix of the data object, the data object matrix and the matrix operation method of the operator through the quaternary formula constructed at each layer, including: For the quadruple constructed in each layer, if the quadruple has data objects with the same attributes or data objects with the same attributes and calculation results, the data objects with the same attributes or the data objects with the same attributes and the calculation results are calculated according to the operator in the quadruple to obtain a simplified result and the quadruple is deleted, and the simplified result is set to the corresponding position in the lower layer quadruple to obtain simplified quadruple; through the simplified quadruple, the coefficient matrix of the data object, the data object matrix and the matrix operation method that conforms to the operator are determined to obtain the calculation structure.

4. The method according to any one of claims 1 to 3, characterized in that The central processor sends the business data of the computing structure received from the business system to the graphics processor, including: The central processor sends the business data of the computing structure received from the business system to the graphics processor after the data volume threshold is met.

5. The method as claimed in claim 4, characterized in that Also includes: The central processor determines that the business data of the computing structure received from the business system does not meet the data volume threshold and the time length for the central processor to receive the business data of the computing structure is greater than a preset time threshold, and then sends the business data of the computing structure to the graphics processor.

6. The method as claimed in claim 4, characterized in that The data volume threshold is obtained by: The central processing unit determines the data volume threshold based on a limit time, a data calculation time, a first time, and a second time. The limit time is the effective time of the calculation result of the business data. The data calculation time is the time for the graphics processor to process the business data to obtain the calculation result. The first time is the time for the graphics processor to receive and store unit data. The second time is the time for the central processing unit to receive and store unit data.

7. The method according to any one of claims 1 to 3, characterized in that Also includes: The central processor performs stream processing on the business data corresponding to the non-computing structure received from the business system to obtain a processing result.

8. A data processing device, characterized in that: A server suitable for a central processing unit with streaming processing capability and a graphics processing unit with parallel processing capability; the device comprises: A processing module, for each processing logic in the business system, wherein the central processing unit converts the processing logic that can be processed in parallel into a computing structure suitable for the graphics processor, and sends each computing structure to the graphics processor through the transceiver module; The transceiver module is also used for, for each computing structure, the central processor to send the business data of the computing structure received from the business system to the graphics processor; The processing module is also used for, for each computing structure, the graphics processor to perform parallel processing on the business data corresponding to the computing structure according to the computing structure, and to send the processing result to the central processor through the transceiver module; The processing module is also used to construct a semantic tree of the processing logic for any processing logic; traverse the semantic tree, extract data objects, operators and coefficients corresponding to the data objects in the semantic tree, so as to construct a calculation structure; if the calculation structure has coefficients of each data object, then the calculation structure is suitable for parallel processing of a graphics processor.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is run on a computer, the computer is enabled to implement the method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: Memory for storing computer programs; A processor is used to call the computer program stored in the memory, and execute the method according to any one of claims 1 to 7 according to the obtained program.

Images