Distributed data processing method and system of computer equipment

Abstract

The invention discloses a distributed data processing method and system for computer equipment, relates to the technical field of computers, and mainly aims at solving the problem that distributed data processing of existing computer equipment is poor in effectiveness. Comprising the following steps: acquiring data processing task information by a sovereignty control device, extracting at least one key task point from the data processing task information, and determining a distributed sub-authority device matched with the key task point; after the main authority control equipment and the distributed sub-authority equipment complete authority verification, the split task data are sent to the distributed sub-authority equipment; when the main authority control equipment receives a data processing result processed by the distributed sub-authority equipment, acquiring distributed processing times counted by the data processing result; and when the number of distributed processing times is greater than a preset number threshold, completing processing of the task data, and integrating the data processing results.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a distributed data processing method and system for computer devices. Background Technology

[0002] With the gradual development of big data processing technology, massive amounts of data require clustered distributed processing of computer equipment to reduce the processing pressure on a single computer and improve data processing efficiency.

[0003] Currently, when performing distributed processing, existing computer equipment typically relies on a master server for data distribution, with each computer processing the distributed data. However, since each computer typically processes data independently relative to the others, if the processing result of one computer is abnormal, it will affect the overall accuracy of data processing during data integration, thus significantly reducing the effectiveness of distributed data processing. Summary of the Invention

[0004] In view of this, the present invention provides a distributed data processing method and system for computer devices, the main purpose of which is to solve the problem of poor effectiveness of distributed data processing in existing computer devices.

[0005] According to one aspect of the present invention, a distributed data processing method for a computer device is provided, comprising:

[0006] Data processing task information is obtained through the main access control device, and at least one key task point is extracted from the data processing task information to determine the distributed sub-access device that matches the key task point.

[0007] After the main permission control device and the distributed sub-permission device complete permission verification, the main permission control device sends the split task data to the distributed sub-permission device. The task data is obtained by splitting the data processing task information according to the key task points and the processing efficiency coefficient of the distributed sub-permission device. The processing efficiency coefficient is used to characterize the distributed processing capability of the distributed sub-permission device so that the distributed sub-permission device can process the task data.

[0008] When the main access control device receives the data processing result processed by the distributed sub-access device, it obtains the number of distributed processing times counted by the data processing result;

[0009] When the number of distributed processing operations exceeds a preset threshold, the processing of the task data is completed, and the data processing results are integrated.

[0010] According to another aspect of the present invention, a distributed data processing system for a computer device is provided, comprising:

[0011] The main access control device is used to acquire data processing task information, extract at least one key task point from the data processing task information, and determine the distributed sub-access devices that match the key task point.

[0012] The main permission control device is further configured to send the split task data to the distributed sub-permission device after the main permission control device and the distributed sub-permission device have completed permission verification. The task data is obtained by splitting the data processing task information according to the key task points and the processing efficiency coefficient of the distributed sub-permission device. The processing efficiency coefficient is used to characterize the distributed sub-permission device's ability to perform distributed processing.

[0013] Distributed sub-permission settings are used to process the task data;

[0014] The main access control device is also used to obtain the number of distributed processing times counted by the data processing result when it receives the data processing result processed by the distributed sub-access device.

[0015] The main access control device is also used to complete the processing of the task data and integrate the data processing results when the number of distributed processing operations exceeds a preset threshold.

[0016] By employing the above-described technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages:

[0017] This invention provides a distributed data processing method and system for computer devices. Compared with existing technologies, this invention obtains data processing task information through a primary access control device, extracts at least one key task point from the data processing task information, and determines a distributed sub-access device matching the key task point. After the primary access control device and the distributed sub-access device complete access verification, the primary access control device sends split task data to the distributed sub-access device. The task data is obtained by splitting the data processing task information according to the key task point and the processing efficiency coefficient of the distributed sub-access device. The processing efficiency coefficient is used to... The distributed sub-authorization device is characterized by its ability to perform distributed processing, enabling it to process the task data. When the main authorization control device receives the data processing result from the distributed sub-authorization device, it obtains the number of distributed processing operations counted by the data processing result. If the number of distributed processing operations exceeds a preset threshold, the processing of the task data is completed, and the data processing results are integrated to achieve multi-party collaborative processing of multiple distributed computer devices. This improves the accuracy of task data processing, avoids errors from single-computer processing, and thus improves the overall data processing accuracy of the computer device cluster, thereby greatly enhancing the effectiveness of distributed data processing.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0020] Figure 1 A flowchart of a distributed data processing method for a computer device provided by an embodiment of the present invention is shown;

[0021] Figure 2 This diagram illustrates a polling method between a master controller and a distributed system, as provided in an embodiment of the present invention.

[0022] Figure 3 A schematic diagram of a distributed data processing system structure for a computer device provided by an embodiment of the present invention is shown. Detailed Implementation

[0023] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0024] This invention provides a distributed data processing method for computer devices, such as... Figure 1 As shown, the method includes:

[0025] 101. Obtain data processing task information through the main access control device, extract at least one key task point from the data processing task information, and determine the distributed sub-access device that matches the key task point.

[0026] In this embodiment of the invention, the distributed data processing system includes multiple computer devices. A computer device acting as the primary access control device distributes task data, and each computer device acting as a distributed sub-access control device receives and processes the distributed task data. The applicable scenarios for the task data include, but are not limited to, financial data, medical data, and industrial data; this embodiment of the invention does not impose specific limitations. After one computer in the computer cluster is determined to be the primary access control device, the computer acting as the primary access control device obtains data processing task information. At this time, the data processing task information is used to characterize the specific task content to be processed on the task data. The task content includes, but is not limited to, processing time, processing quantity, processed data objects, and processing rules; this embodiment of the invention does not impose specific limitations. Key task points are used to characterize the tasks that each distributed sub-access control device prioritizes or must process, so that matching distributed sub-access control devices are determined according to these key task points.

[0027] It should be noted that when determining the distributed sub-permission device that matches the key task point, the main permission control device can load or request the pre-configured sub-permission task points in each distributed sub-permission device to match each sub-permission task point with the key task point and determine the distributed sub-permission device that can match the data processing task information. At this time, the determined distributed sub-permission device can be one or more, and the embodiments of the present invention do not make specific limitations.

[0028] 102. After the main permission control device and the distributed sub-permission device complete permission verification, the main permission control device sends the split task data to the distributed sub-permission device.

[0029] In this embodiment of the invention, to ensure that the primary access control device can publish task data to each distributed sub-access device, the primary access control device, after determining the distributed sub-access devices, performs access verification. Following access verification, the primary access control device sends the split task data to the distributed sub-access devices. The task data is obtained by splitting the data processing task information according to the key task points and the processing efficiency coefficients of the distributed sub-access devices. Here, the processing efficiency coefficients characterize the distributed processing capability of the distributed sub-access devices, enabling them to process the task data.

[0030] It should be noted that after processing the task data, each distributed sub-authorization device can send the task data to other distributed sub-authorization devices of the same type or with the same processing function to avoid the uniformity of the processing results and the possibility of errors, thereby obtaining multiple processing results for the same task data.

[0031] 103. When the main access control device receives the data processing result processed by the distributed sub-access device, it obtains the number of distributed processing times counted by the data processing result.

[0032] In this embodiment of the invention, since multiple distributed sub-authorization devices can process the same task data, resulting in multiple data processing results, the master authorization control device obtains the number of distributed processing operations of each distributed sub-authorization device when determining whether the data processing result is usable. Specifically, after processing the task data, each distributed sub-authorization device sends it to the master authorization control device so that the master authorization control device can count the number of distributed processing operations.

[0033] 104. When the number of distributed processing operations exceeds a preset threshold, the processing of the task data is completed, and the data processing results are integrated.

[0034] In this embodiment of the invention, when the number of distributed processing operations obtained is greater than a preset threshold, it indicates that each distributed sub-permission device has reached the point of avoiding processing tasks. Therefore, the main permission control device determines that the task data has been processed and integrates the multiple data processing results to obtain the final task data processing result. Furthermore, in this embodiment of the invention, the preset threshold is configured based on the type and function of different distributed sub-permission devices. For example, if five distributed sub-permission devices of the same type send task data to each other for processing, the preset threshold can be 2.5, meaning that as long as half of the multiple distributed sub-permission devices of the same type have completed processing the same task data and obtained the data processing result, the task data processing is complete.

[0035] It should be noted that when sending task data between multiple distributed sub-authority devices, in one specific implementation scenario, each distributed sub-authority device stores device identifiers of other distributed sub-authority devices with the same system type and processing function as itself. This allows the first distributed sub-authority device to process task data and then send it to other distributed sub-authority devices for multiple task data processing based on the device identifier. In another specific implementation scenario, the first distributed sub-authority device can synchronously send task data to other second distributed sub-authority devices. If a second distributed sub-authority device receives the task data, it determines the current system resource ratio. If the system resource ratio is greater than a preset resource ratio, the second distributed sub-authority device processes the task data. In this case, the preset resource ratio is the resource ratio that the second distributed sub-authority device has reserved for task data sent by other distributed sub-authority devices. This can be configured based on different processing needs, and this embodiment of the invention does not impose specific limitations.

[0036] In another embodiment of the invention, for further definition and explanation, the step of obtaining data processing task information through a master access control device and extracting at least one key task point from the data processing task information includes:

[0037] The device with master access control downloads data processing task information from the data task publisher;

[0038] At least one key task point is extracted from the data task volume, data task type, data task scenario, and data task processing strategy according to the preset key identifier.

[0039] To achieve the goal of extracting key task points from data processing task information and identify tasks with important processing nodes, thereby improving the effectiveness of data processing based on defined distributed sub-permission devices, the data task publisher can be a server executing applications for different businesses to generate different data and corresponding processing tasks. It can be a cloud server or a terminal server; this embodiment of the invention does not impose specific limitations. The downloaded data processing task information includes the data task volume, data task type, data task scenario, and data task processing strategy for different task points. The data task volume is the maximum amount of data to be processed when executing the data processing task. The data task type is the specific task type to be processed when executing the data processing task, such as a specified calculation task, statistical task, or classification task. The data task scenario is the specific scenario corresponding to the execution of the data processing task, such as a Christmas scenario or a marketing scenario. The data task processing strategy is the rule or method to be executed when executing the data processing task, including but not limited to regular expression rules and artificial intelligence algorithms; this embodiment of the invention does not impose specific limitations. Furthermore, the data processing task information is generated in the data task publisher. It can be matched for different data sources, requirements, or system paths, or it can be written in real time by developers to meet different data processing needs. After downloading the data processing task information, at least one key task point is extracted from the data task volume, data task type, data task scenario, and data task processing strategy based on preset key identifiers. At this time, the preset key identifiers include preset key task volume identifiers, preset key task type identifiers, preset key task scenario identifiers, and preset key task processing strategy identifiers, which are pre-configured when publishing the data task processing information. The current execution end divides the tasks from the data task processing information according to each identifier in order to determine the tasks that need to be focused on. This embodiment of the invention does not make specific limitations.

[0040] In another embodiment of the invention, for further definition and explanation, the steps further include:

[0041] When the master control device is in the polling period and the distributed sub-permission device has not generated a permission change request, the master control device retrieves the processing energy efficiency coefficient of the distributed sub-permission device, and when the processing energy efficiency coefficient is less than a preset energy efficiency threshold, permission verification for the distributed sub-permission device is determined.

[0042] The master control device splits the data processing task information according to the preset energy efficiency task mapping relationship to obtain the split task data.

[0043] To ensure the accuracy and effectiveness of critical task processing, the master control device first determines whether its master control permissions are in the polling phase, i.e., whether it has the authority to split tasks, and whether there is a need for permission changes among the distributed sub-control devices. Therefore, the master control device can retrieve the processing energy efficiency coefficients of each distributed sub-control device and compare them with a preset energy efficiency threshold to determine whether the distributed sub-control devices have the authority to process tasks, thus completing the permission verification. Simultaneously, to optimize resources for each distributed sub-control device and ensure that key task data is processed preferentially, thereby improving processing efficiency, the master control device can also split the data processing task information according to a preset energy efficiency task mapping relationship, obtaining the split task data. The preset energy efficiency task mapping relationship includes different processing efficiency coefficients for splitting different critical task points and the content of the split, to meet the precise allocation needs of different distributed sub-control devices for processing capacity. In this case, the split content may include data volume splitting, task level splitting, etc., which are not specifically limited in this embodiment of the invention. At this point, since the processing efficiency coefficient is used to characterize the distributed sub-permission device's ability to perform distributed processing, the processing efficiency coefficient is obtained by weighted calculation after weighting based on factors such as system CPU computing power, system thread ratio, and device energy consumption, in order to decompose key task points.

[0044] In another embodiment of the invention, for further definition and explanation, the steps further include:

[0045] The distributed sub-permission device determines the real-time processing volume, expected processing volume, and extreme processing volume corresponding to different task points, and calculates the processing efficiency coefficient based on the real-time processing volume, the expected processing volume, and the extreme processing volume.

[0046] Since the processing efficiency coefficient is used to characterize the distributed processing capability of the distributed sub-authority devices, in order to achieve the purpose of reasonably distributing data to the distributed sub-authority devices based on the processing efficiency coefficient, thereby improving the accuracy of distributed processing, each distributed sub-authority device can also calculate the processing efficiency coefficient and send it to the master authorization control device. Specifically, the distributed sub-authority devices first determine the real-time processing volume, expected processing volume, and processing volume extreme value corresponding to different real-time execution task points. At this time, the real-time processing volume, expected processing volume, and processing volume extreme value can be the real-time processing volume, expected processing volume, and processing volume extreme value of the system CPU computing power, system thread ratio, and device energy consumption. After statistics, the processing efficiency coefficient is calculated based on the obtained real-time processing volume, expected processing volume, and processing volume extreme value. The calculation can be based on a formula, and this embodiment of the invention does not impose specific limitations.

[0047] Specifically, the formula for calculating the processing efficiency coefficient is as follows: Where M represents the total number of attributes selected as the processing power of a computer device, such as system CPU computing power, system thread ratio, and device energy consumption, i = 1, 2, ..., M, x i For the real-time processing volume of the aforementioned i-th attribute, y i z represents the expected processing volume of the i-th attribute mentioned above. i The processing capacity extreme value of the aforementioned i-th attribute is used to calculate the processing efficiency coefficient of the distributed sub-permission devices. This calculation can be performed in the main control device at a preset time interval (e.g., 1 day or 2 days), or it can be performed separately in each of the distributed sub-permission devices at a preset time interval (e.g., 1 day or 2 days) and then sent to the main control device. This embodiment of the invention does not impose specific limitations.

[0048] In another embodiment of the invention, for further definition and explanation, the steps further include:

[0049] When the first distributed sub-authorization device receives the task data, it processes the task data and sends the first data processing result and the task data to the second distributed sub-authorization device.

[0050] After the second distributed sub-authorization device processes the received task data, it compares the second data processing result with the first data processing result. When the comparison result matches a preset comparison threshold, it sends the first data processing result and the second data processing result to the master authorization control device.

[0051] To enable multiple computer devices to process a single task data multiple times, ensuring the accuracy of the task data processing, a first distributed sub-authorization device, as the first to receive the task data, processes it according to the task content. This task content includes, but is not limited to, transaction processing, distribution processing, and computation processing; this embodiment of the invention does not impose specific limitations. After processing the task data, the first distributed sub-authorization device obtains a first data processing result. At this point, the first distributed sub-authorization device can simultaneously send the task data to a second distributed sub-authorization device with which it has a communication connection. The second distributed sub-authorization device can be a computer device with the same model and system resources as the first distributed sub-authorization device. Before receiving the task data, a connection can be established between the first and second distributed sub-authorization devices through communication protocols such as a handshake protocol; this embodiment of the invention does not impose specific limitations.

[0052] It should be noted that after the second distributed sub-authority device processes the received task data, it processes the data based on the task content carried in the task data to obtain a second data processing result. At this point, the second distributed sub-authority device can compare the obtained second data processing result with the first data processing result, for example, by performing a difference calculation. When the comparison result matches a preset comparison threshold, the second distributed sub-authority device sends both the first and second data processing results to the main control authority device. The preset comparison threshold, used as a constraint on the comparison result, can be 0.5, 0.8, 1, etc., and can be configured based on different error requirements; this embodiment of the invention does not impose specific limitations. Furthermore, after the main control authority device receives the first and second data processing results, it counts the number of distributed processing operations based on the device identifiers carried in the first and second data processing results. Since there can be multiple second distributed sub-authority devices communicating with the first distributed sub-authority device, and multiple corresponding first data processing results, the number of distributed processing operations can be counted based on the received device identifiers.

[0053] In another embodiment of the invention, for further definition and explanation, the step of integrating the data processing results includes:

[0054] If a second data processing result exists, then the first data processing result and the second data processing result are averaged and integrated to obtain the integrated data processing result;

[0055] If there are at least two second data processing results, then the average coefficient is determined based on the extreme values ​​in the second data processing results, and the average value is integrated based on the average coefficient and the first data processing results to obtain the integrated data processing result.

[0056] To ensure the effectiveness of processing the same task data across multiple computer devices, when the primary access control device receives a second data processing result, it indicates that only one second distributed sub-access device is processing the same task data. Therefore, when obtaining the final data processing result, the evaluation value is directly calculated using both the first and second data processing results. In another embodiment, when at least two second data processing results exist, it indicates that multiple second distributed sub-access devices are processing the same task data. Therefore, the maximum and minimum values ​​are first extracted from all the second data processing results (i.e., the results obtained by the second distributed sub-access devices), and the average value is calculated based on the maximum and minimum values ​​as the average coefficient. Furthermore, the final data processing result is obtained by averaging the average coefficient and the first data processing result again, thereby optimizing the final data processing result and improving the accuracy of task data processing.

[0057] In another embodiment of the invention, for further definition and explanation, the steps further include:

[0058] When the master control authority of the master control device exceeds the polling time, a polling instruction is sent to the distributed sub-authority device to re-determine the master control authority. The polling instruction carries the polling time.

[0059] To enable various computer devices to perform data task processing, thereby reducing the permission pressure on a single master control device and meeting the permission control needs of different computer devices, the computer device acting as the master permission control device can change permissions as needed. In this embodiment of the invention, permission changes can be based on a polling time. That is, when the master permission control device actively detects the time it takes to acquire master control permissions, and if the time elapsed since acquiring master control permissions exceeds the polling time, it sends a polling instruction to all distributed sub-permission devices. The polling time can be configured based on the system cache time, device model, computing power, etc., of each computer, and can also be configured based on the application scenario corresponding to the task data (e.g., transaction scenario, information push scenario, etc.), such as 1 week, 2 weeks, 1 month, etc., without specific limitations in this embodiment. When each distributed sub-permission device receives the polling instruction, since the polling instruction carries the polling time, each distributed sub-permission device can choose whether to request master control permissions based on this polling time, thus becoming the master permission control device for the next round.

[0060] It should be noted that after receiving the polling time, the distributed sub-authority devices can either manually choose whether to request master control permissions, or they can request them based on the triggering conditions for sovereign permissions configured in the computer device. For example, the triggering condition could be sending a request as soon as a polling instruction is received, or sending a request when a polling instruction is received and the polling time is less than the system cache time, etc. This embodiment of the invention does not impose specific limitations. Furthermore, when multiple distributed sub-authority devices send master control permission change requests, the master control permission device can select according to the order of receipt, or randomly, or based on the model, system percentage, etc., set in the master control permission device. This embodiment of the invention does not impose specific limitations. Figure 2 In the system architecture diagram shown, one master control device communicates with multiple distributed sub-control devices. After sending a polling command, the distributed sub-control device c responds with a master control permission request based on an automatically set trigger command, so as to re-determine the original distributed sub-control device c as the new master control device.

[0061] This invention provides a distributed data processing method for computer devices. Compared with existing technologies, this invention obtains data processing task information through a primary access control device, extracts at least one key task point from the data processing task information, and determines a distributed sub-access device matching the key task point. After the primary access control device and the distributed sub-access device complete access verification, the primary access control device sends split task data to the distributed sub-access devices. The task data is obtained by splitting the data processing task information according to the key task point and the processing efficiency coefficient of the distributed sub-access devices. The processing efficiency coefficient is used to characterize the distributed processing capability of the distributed sub-access devices, enabling the distributed sub-access devices to process the task data. When the primary access control device receives the data processing result processed by the distributed sub-access devices, it obtains the number of distributed processing operations counted by the data processing result. When the number of distributed processing operations exceeds a preset threshold, the processing of the task data is completed, and the data processing results are integrated to achieve multi-party collaborative processing of multiple distributed computer devices. This improves the accuracy of task data processing, avoids errors from single-computer processing, and thus improves the overall data processing accuracy of the computer device cluster, thereby greatly improving the effectiveness of distributed data processing.

[0062] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a distributed data processing system for computer devices, such as... Figure 3 As shown, the system includes:

[0063] The main access control device 21 is used to acquire data processing task information, extract at least one key task point from the data processing task information, and determine the distributed sub-access device that matches the key task point.

[0064] The main permission control device 21 is further configured to send the split task data to the distributed sub-permission device after the main permission control device and the distributed sub-permission device have completed permission verification. The task data is obtained by splitting the data processing task information according to the key task points and the processing efficiency coefficient of the distributed sub-permission device. The processing efficiency coefficient is used to characterize the distributed sub-permission device's ability to perform distributed processing.

[0065] Distributed sub-authorization device 22 is used to process the task data;

[0066] The main permission control device 21 is also used to obtain the number of distributed processing times counted by the data processing result when it receives the data processing result processed by the distributed sub-permission device;

[0067] The main access control device 21 is also used to complete the processing of the task data and integrate the data processing results when the number of distributed processing times exceeds a preset threshold.

[0068] This invention provides a distributed data processing system for computer devices. Compared with existing technologies, this invention obtains data processing task information through a primary access control device, extracts at least one key task point from the data processing task information, and determines a distributed sub-access device matching the key task point. After the primary access control device and the distributed sub-access device complete access verification, the primary access control device sends split task data to the distributed sub-access devices. The task data is obtained by splitting the data processing task information according to the key task point and the processing efficiency coefficient of the distributed sub-access device. The processing efficiency coefficient is used to characterize the distributed processing capability of the distributed sub-access devices, enabling the distributed sub-access devices to process the task data. When the primary access control device receives the data processing result processed by the distributed sub-access devices, it obtains the number of distributed processing operations counted by the data processing result. When the number of distributed processing operations exceeds a preset threshold, the processing of the task data is completed, and the data processing results are integrated to achieve multi-party collaborative processing of multiple distributed computer devices. This improves the accuracy of task data processing, avoids errors from single-computer processing, and thus improves the overall data processing accuracy of the computer device cluster, thereby greatly improving the effectiveness of distributed data processing.

[0069] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing systems. They can be centralized on a single computing system or distributed across a network of multiple computing systems. Optionally, they can be implemented using program code executable by a computing system, thereby storing them in a storage system for execution by the computing system. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A distributed data processing method of a computer device, characterized by, The method comprises the following steps: obtaining data processing task information by a main authority control device, and extracting at least one key task point from the data processing task information, and determining a distributed sub-authority device matched with the key task point; after the main authority control device and the distributed sub-authority device complete authority verification, sending the split task data to the distributed sub-authority device through the main authority control device, wherein the task data is obtained by splitting the data processing task information according to the key task point and the processing efficiency coefficient of the distributed sub-authority device, and the processing efficiency coefficient is used to represent the ability of the distributed sub-authority device for distributed processing, so that the distributed sub-authority device processes the task data; when the main authority control device receives the data processing result processed by the distributed sub-authority device, obtaining the distributed processing times counted by the data processing result; when the distributed processing times are greater than a preset number threshold, completing the processing of the task data, and integrating the data processing result.

2. The method of claim 1, wherein, The method further comprises the following steps: downloading the data processing task information from a data task publisher by the main authority control device, wherein the data processing task information comprises data task quantity, data task type, data task scene and data task processing strategy of different task points; extracting at least one key task point from the data task quantity, data task type, data task scene and data task processing strategy according to a preset key identifier, wherein the preset key identifier comprises a preset key task quantity identifier, a preset key task type identifier, a preset key task scene identifier and a preset key task processing strategy identifier.

3. The method of claim 2, wherein, The method further comprises the following steps: when the main control authority control device is in the polling time, and the distributed sub-authority device does not generate an authority change request, the processing efficiency coefficient of the distributed sub-authority device is called by the main control authority control device, and when the processing efficiency coefficient is less than a preset efficiency threshold, the authority verification of the distributed sub-authority device is determined; the data processing task information is split by the main control authority control device according to a preset efficiency task mapping relationship, and the split task data is obtained, wherein the preset efficiency task mapping relationship comprises different processing efficiency coefficients for splitting different key task points and the split content.

4. The method of claim 3, wherein, The method further comprises the following steps: determining the real-time processing quantity, expected processing quantity and processing quantity extreme value corresponding to different task points by the distributed sub-authority device, and calculating the processing efficiency coefficient based on the real-time processing quantity, expected processing quantity and processing quantity extreme value.

5. The method of claim 4, wherein, The method further comprises the following steps: after the first distributed sub-authority device receives the task data, processing the task data, and sending the obtained first data processing result and the task data to the second distributed sub-authority device; When the second distributed sub-privilege device processes the received task data, the obtained second data processing result is compared with the first data processing result, and when the comparison result matches a preset comparison threshold, the first data processing result and the second data processing result are sent to the master privilege control device, and the first data processing result and the second data processing result carry device identifiers for distributed processing times statistics.

6. The method of claim 5, wherein, The integrating the data processing result comprises: If there is one second data processing result, the first data processing result and the second data processing result are averaged to obtain the integrated data processing result; If there are at least two second data processing results, an average coefficient is determined based on the extreme value in the second data processing result, and the first data processing result is averaged based on the average coefficient to obtain the integrated data processing result.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: When the master privilege of the master privilege control device exceeds the polling time, a polling instruction is sent to the distributed sub-privilege device to re-determine the master privilege control device, and the polling instruction carries a polling time.

8. A distributed data processing system of a computer device, characterized by, Comprise: The master privilege control device is used for obtaining data processing task information, and extracting at least one key task point from the data processing task information, and determining a distributed sub-privilege device matched with the key task point; The master privilege control device is also used for sending the split task data to the distributed sub-privilege device after the master privilege control device and the distributed sub-privilege device complete privilege verification, wherein the task data is obtained by splitting the data processing task information according to the key task point and the processing efficiency coefficient of the distributed sub-privilege device, and the processing efficiency coefficient is used to represent the ability of the distributed sub-privilege device for distributed processing; The distributed sub-privilege device is used for processing the task data; The master privilege control device is also used for receiving the data processing result processed by the distributed sub-privilege device, and obtaining the distributed processing times counted by the data processing result; The master privilege control device is also used for completing the processing of the task data when the distributed processing times are greater than a preset number threshold, and integrating the data processing result.

Images