Data Processing Method, Apparatus, Electronic Device and Computer-Readable Storage Medium

Abstract

The embodiments of the present application provide a data processing method, apparatus, electronic device, and computer-readable storage medium, which relate to the technical field of big data operation and maintenance. The method includes: determining the data required to complete the task to be processed; wherein the data includes a plurality of sub-data, and the plurality of sub-data are distributed and stored in a plurality of processing nodes; screening out target processing nodes from the plurality of processing nodes according to the estimated value of resource consumption for processing the task to be processed; wherein the surplus degree of the target processing node is greater than a preset first threshold; the surplus degree is determined according to the estimated value of resource consumption and the resource available information of each processing node; processing the task to be processed according to the target processing node to obtain a processing result. The embodiments of the present application screen out target processing nodes for task processing through the estimated value of resource consumption of the task and the resource available information of each processing node, effectively improving the analysis and calculation efficiency of big data operation and maintenance.

Description

Technical Field

[0001] This application relates to the technical field of big data operation and maintenance. Specifically, this application relates to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the continuous development of the information age, IT (Information Technology) operation and maintenance has become an important part of the IT service connotation. Facing increasingly complex services and increasingly diverse user requirements, the continuously expanding IT applications require a more reasonable mode to ensure that IT services can be flexibly, conveniently, securely, and stably continuously guaranteed. The guarantee factor in this mode is IT operation and maintenance; operation and maintenance essentially involve the operation and maintenance of each stage of the life cycle of networks, servers, and services, achieving a state that is uniformly acceptable in terms of cost, stability, and efficiency.

[0003] In the prior art, an intelligent operation and maintenance analysis solution is usually built based on a distributed framework to carry out the analysis and operation of a large amount of data sets. Taking the PySpark (an application programming interface of Spark, a distributed open-source processing system) distributed framework as an example, its deployment depends on a large number of associated components. When obtaining data from a data source, a large amount of resources are required for data network transmission, resulting in the problem of low analysis and calculation efficiency in big data operation and maintenance. Summary of the Invention

[0004] Embodiments of this application provide a data processing method, apparatus, electronic device, and computer-readable storage medium, which can solve the problem of low analysis and calculation efficiency in big data operation and maintenance in the prior art. The technical solutions are as follows:

[0005] According to one aspect of the embodiments of this application, a data processing method is provided. The method includes:

[0006] Determine the data required to complete the task to be processed; where the data includes multiple sub-data, and the multiple sub-data are distributed and stored in multiple processing nodes;

[0007] According to the predicted value of resource consumption for processing the task to be processed, select target processing nodes from multiple processing nodes; where the surplus degree of the target processing nodes is greater than a preset first threshold; the surplus degree is determined according to the predicted value of resource consumption and the resource availability information of each processing node;

[0008] Process the task to be processed according to the target processing nodes to obtain a processing result.

[0009] Optionally, before selecting the target processing nodes from multiple processing nodes according to the predicted value of resource consumption for processing the task to be processed, it includes:

[0010] For each sub-data, determine the unique coding value of the sub-data according to the attribute information of the sub-data;

[0011] Calculate the index value of the sub-data according to the unique coding value, and use the processing node corresponding to the index value as the processing node for storing the sub-data.

[0012] Optionally, the above screening of the target processing node from multiple processing nodes according to the estimated resource consumption for processing the to-be-processed task includes:

[0013] Determine the resource availability information of each processing node;

[0014] For each processing node, determine the surplus degree of the resource availability information of the processing node relative to the estimated resource consumption;

[0015] When there is at least one reference processing node among the processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes; wherein, the reference processing nodes are the processing nodes with a surplus degree greater than a second threshold; the second threshold is not greater than the first threshold.

[0016] Optionally, the types of the above sub-data include primary and backup; the processing nodes for storing each primary sub-data are first-level processing nodes, the processing nodes for storing each backup sub-data are second-level processing nodes, and the processing nodes other than the first-level processing nodes and the second-level processing nodes are third-level processing nodes;

[0017] Screening out the target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes includes:

[0018] When the reference processing nodes include first-level processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the first-level processing nodes.

[0019] Optionally, the above method further includes:

[0020] When the reference processing nodes include second-level processing nodes and do not include first-level processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the second-level processing nodes;

[0021] When the reference processing nodes only include third-level processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the third-level processing nodes.

[0022] Optionally, the above resource availability information includes: the available number of CPU cores and the available memory size;

[0023] Determining the surplus degree of the resource availability information of the processing node relative to the estimated resource consumption includes:

[0024] Weightedly sum up the available core numbers of the CPU and the available memory size to obtain the resource surplus value of the processing node;

[0025] Weight the resource surplus value according to the difference between the available resource information and the estimated resource consumption to obtain the surplus degree.

[0026] Optionally, the above-mentioned weighted summation of the available core numbers of the CPU and the available memory size to obtain the resource surplus value of the processing node includes:

[0027] Determine the resource surplus base number of the processing node according to the type of the processing node; among them, the resource surplus base number of the first-level processing node is greater than that of the second-level processing node, and the resource surplus base number of the second-level processing node is greater than that of the third-level processing node;

[0028] Weightedly sum up the resource surplus base number, the available core numbers of the CPU and the available memory size to obtain the resource surplus value of the processing node.

[0029] According to another aspect of the embodiments of the present application, a data processing device is provided, and the device includes:

[0030] A determination module, configured to determine the data required to complete the to-be-processed task; wherein the data includes a plurality of sub-data, and the plurality of sub-data are distributed and stored in a plurality of processing nodes;

[0031] A screening module, configured to screen out a target processing node from a plurality of processing nodes according to the estimated resource consumption for processing the to-be-processed task; wherein the surplus degree of the target processing node is greater than a first preset threshold; the surplus degree is determined according to the estimated resource consumption and the resource available information of each processing node;

[0032] A processing module, configured to process the to-be-processed task according to the target processing node to obtain a processing result.

[0033] Optionally, before the above-mentioned screening module screens out a target processing node from a plurality of processing nodes according to the estimated resource consumption for processing the to-be-processed task, it is used for:

[0034] For each sub-data, determine the unique coding value of the sub-data according to the attribute information of the sub-data;

[0035] Calculate the index value of the sub-data according to the unique coding value, and use the processing node corresponding to the index value as the processing node for storing the sub-data.

[0036] Optionally, when the above-mentioned screening module screens out a target processing node from a plurality of processing nodes according to the estimated resource consumption for processing the to-be-processed task, it is used for:

[0037] Determine the resource available information of each processing node;

[0038] For each processing node, determine the surplus degree of the available resource information of the processing node relative to the estimated resource consumption;

[0039] When there is at least one reference processing node among the processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes; wherein, the reference processing nodes are the processing nodes with a surplus degree greater than a second threshold; the second threshold is not greater than the first threshold.

[0040] Optionally, the types of the above sub-data include primary and backup; the processing nodes storing the primary sub-data are first-level processing nodes, the processing nodes storing the backup sub-data are second-level processing nodes, and the processing nodes other than the first-level and second-level processing nodes are third-level processing nodes;

[0041] When the above screening module screens out the target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes, it is used for:

[0042] When the reference processing nodes include first-level processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the first-level processing nodes.

[0043] Optionally, the above screening module is further used for:

[0044] When the reference processing nodes include second-level processing nodes and do not include first-level processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the second-level processing nodes;

[0045] When the reference processing nodes only include third-level processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the third-level processing nodes.

[0046] Optionally, the above available resource information includes: the available number of CPU cores and the available memory size;

[0047] When the above screening module determines the surplus degree of the available resource information of the processing node relative to the estimated resource consumption, it is used for:

[0048] Perform a weighted sum of the available number of CPU cores and the available memory size to obtain the resource surplus value of the processing node;

[0049] Weight the resource surplus value according to the difference between the available resource information and the estimated resource consumption to obtain the surplus degree.

[0050] Optionally, when the above screening module performs a weighted sum of the available number of CPU cores and the available memory size to obtain the resource surplus value of the processing node, it is used for:

[0051] Determine the resource surplus base of the processing node according to the type of the processing node; among them, the resource surplus base of the first-level processing node is greater than that of the second-level processing node, and the resource surplus base of the second-level processing node is greater than that of the third-level processing node;

[0052] Perform a weighted sum of the resource surplus base, the available CPU cores, and the available memory size to obtain the resource surplus value of the processing node.

[0053] According to another aspect of the embodiments of the present application, there is provided an electronic device, which includes: a memory, a processor, and a computer program stored on the memory, and the above processor executes the computer program to implement the steps of the method shown in the first aspect of the embodiments of the present application.

[0054] According to still another aspect of the embodiments of the present application, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method shown in the first aspect of the embodiments of the present application are implemented.

[0055] According to an aspect of the embodiments of the present application, there is provided a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the method shown in the first aspect of the embodiments of the present application are implemented.

[0056] The beneficial effects brought by the technical solutions provided in the embodiments of the present application are:

[0057] In the embodiments of the present application, by distributing and storing multiple sub-data included in the data required to complete the to-be-processed task in multiple processing nodes, the surplus degree of each processing node can then be determined according to the estimated resource consumption and the available resource information of each processing node, and then target processing nodes with a surplus degree greater than the first threshold are screened out from multiple processing nodes to process the to-be-processed task based on the target processing nodes to obtain a processing result; in the embodiments of the present application, since the estimated resource consumption of the task and the available resource information of the processing node are considered simultaneously during the screening process of the target processing node, the resource consumption of the task can be effectively reduced when processing the to-be-processed task based on the target processing node. Different from the prior art, when performing data processing based on a distributed framework, a large amount of data network transmission is required. In the present application, each sub-data is distributed and stored in multiple processing nodes, and the target processing node can directly analyze and calculate based on the local data, further reducing the network loss of data transmission and effectively improving the analysis and calculation efficiency of big data operation and maintenance. Description of the Drawings

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description in the embodiments of the present application.

[0059] Figure 1 Schematic diagram of an application scenario of a data processing method provided by an embodiment of the present application;

[0060] Figure 2 Flow chart of a data processing method provided by an embodiment of the present application;

[0061] Figure 3 Flow chart of screening target processing nodes in a data processing method provided by an embodiment of the present application;

[0062] Figure 4 Flow chart of screening reference processing nodes in a data processing method provided by an embodiment of the present application;

[0063] Figure 5 Flow chart of a data processing method of an example provided by an embodiment of the present application;

[0064] Figure 6 Schematic diagram of the structure of a data processing device provided by an embodiment of the present application;

[0065] Figure 7 Schematic diagram of the structure of a data processing electronic device provided by an embodiment of the present application. Detailed implementation manners

[0066] The embodiments of the present application will be described below with reference to the accompanying drawings in the present application. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute limitations on the technical solutions of the embodiments of the present application.

[0067] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or their combinations supported by the art of the present technology. It should be understood that when we say an element is "connected" or "coupled" to another element, this element can be directly connected or coupled to the other element, or it can mean that this element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or implemented as "B", or implemented as "A and B".

[0068] To make the objectives, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0069] With the wide application of cloud platforms, microservices, and containerization, the scale of IT system operation and maintenance data has been continuously growing, reaching a scale of hundreds of billions. Storing and performing algorithm analysis on massive data increasingly consume computing resources, and it is necessary to continuously improve computing efficiency.

[0070] In the prior art, a distributed framework is usually used for IT intelligent operation and maintenance analysis of big data. Taking PySpark as an example, this distributed framework can implement data operations on a distributed platform according to the algorithm logic implemented in Python (a computer programming language), and perform iterative analysis on the data connected to data sources such as Hive (a data warehouse tool) or HBase (a distributed database oriented to columnar storage). The inventor found that the prior art has the following problems:

[0071] (1) There are many associated components in the distributed platform, and the deployment and use are complex;

[0072] (2) The efficiency of filtering and screening sample data from data sources such as Hive or HBase is not high;

[0073] (3) Extracting data from data sources requires a large amount of data network transmission.

[0074] The data processing method, device, electronic device, and computer-readable storage medium provided by the present application aim to solve the above technical problems of the prior art.

[0075] The technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described below through the description of several exemplary embodiments. It should be noted that the following embodiments can be referred to, learned from, or combined with each other. For the same terms, similar features, and similar implementation steps in different embodiments, they will not be described repeatedly.

[0076] As Figure 1 shown, the data processing method of the present application can be applied to Figure 1In the scenario shown, specifically, the resource scheduling server 102 receives the task to be processed sent by the terminal 101, determines the data required to complete the task to be processed, where the data includes multiple sub-data, and each sub-data is distributed and stored in multiple resource processing servers; the resource scheduling server 102 can calculate the surplus degree of each resource processing server according to the estimated resource consumption of the task to be processed and the resource availability information of each resource processing server; and filter out the target resource processing servers with a surplus degree greater than the first threshold, so as to process the task to be processed based on the target resource processing servers. The resource scheduling server 102 receives the processing results of the target resource processing servers and returns the processing results to the terminal 101.

[0077] Figure 1 In the scenario shown, the above data processing method can be performed in the resource scheduling server. In other scenarios, it can also be performed in the terminal.

[0078] Those skilled in the art of this technology can understand that the "terminal" used here can be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a MID (Mobile Internet Device), etc.; the "server" can be implemented by an independent server or a server cluster composed of multiple servers.

[0079] In an embodiment of the present application, a data processing method is provided, as Figure 2 shown, which can be applied to a server or a terminal for data processing. The method includes:

[0080] S201, determine the data required to complete the task to be processed.

[0081] Among them, the data includes multiple sub-data, and the multiple sub-data are distributed and stored in multiple processing nodes.

[0082] Specifically, the above data can be data in a data table or a Schema (a collection of database objects). A Schema is a namespace in a database that can represent a collection of the logical structures of data or schema objects, and it can include named objects such as tables, views, indexes, primary keys, foreign keys, and stored procedures.

[0083] In some embodiments, a server or a terminal for data processing can determine the data source corresponding to the data required to complete the task to be processed, such as a Schema set or a data table, and partition the data source to obtain multiple blocks without overlapping data, where each block corresponds to a sub-data; and distribute and store the above multiple sub-data in multiple processing nodes.

[0084] In some other embodiments, a server or a terminal for data processing may determine a data source corresponding to the data required to complete a task to be processed, such as a Schema set, and partition the data source to obtain multiple main blocks without overlapping data; then, at least one backup block is constructed for each main block, and each main block or backup block corresponds to a sub-data; then, the above-mentioned multiple sub-datas are distributed and stored in multiple processing nodes.

[0085] Among them, the above-mentioned processing nodes may be each processing node in a task processing server cluster or a database cluster, and no specific limitation is made in the embodiments of the present application.

[0086] S202, according to the estimated value of resource consumption for processing the task to be processed, screen out target processing nodes from multiple processing nodes.

[0087] Among them, the surplus degree of the target processing node is greater than a preset first threshold; the surplus degree is determined according to the estimated value of resource consumption and the resource available information of each processing node;

[0088] Specifically, a server or a terminal for data processing may determine the surplus degree of the resources of each processing node according to the estimated value of resource consumption required for processing the task to be processed and the resource available information of each processing node, and use the processing node with a surplus degree greater than the preset first threshold as the target processing node.

[0089] In the embodiments of the present application, the first threshold may be the surplus degree of the processing node ranked second, where the above-mentioned processing nodes may be arranged in descending order according to the surplus degree.

[0090] S203, process the task to be processed according to the target processing node to obtain a processing result.

[0091] Among them, a corresponding algorithm module and a storage module are configured in each processing node. The algorithm module is used to execute different algorithms according to different tasks to be processed for data inference and analysis to obtain a calculation result, that is, the processing result of the task to be processed; the storage module is used to store the sub-data corresponding to each processing node.

[0092] Specifically, a server or a terminal for data processing may query and load each sub-data according to the algorithm module in the target processing node; among them, the above-mentioned sub-data may be stored in the storage module of the target processing node or in the storage module of other processing nodes except the target processing node; then, preprocessing operations such as cleaning, duplicate removal, and alignment are performed on each sub-data based on the algorithm module of the target processing node to obtain preprocessed data; then, corresponding algorithms are used to calculate and infer the preprocessed data to obtain the processing result of the task to be processed.

[0093] In an embodiment of the present application, when the processing node is a system operation and maintenance server, the task to be processed may be an abnormal detection task for system operation data, and the data required to complete the task to be processed may be system operation data; the server or terminal for data processing may obtain system operation data based on a preset time interval, and distribute and store the sub-data included in the system operation data in each system operation and maintenance server; when the server or terminal for data processing receives an abnormal detection task, it may determine the required data source according to the abnormal detection task, query and load each sub-data based on the system operation and maintenance server corresponding to the data source, and then perform detection processing on each sub-data based on the corresponding abnormal detection algorithm to obtain the processing result of whether the system is abnormal.

[0094] In the embodiment of the present application, by distributing and storing multiple sub-data included in the data required to complete the task to be processed in multiple processing nodes, the surplus degree of each processing node can be determined according to the estimated resource consumption value and the resource available information of each processing node, and then the target processing node with a surplus degree greater than the first threshold can be selected from multiple processing nodes to process the task to be processed based on the target processing node to obtain the processing result; in the embodiment of the present application, since the estimated resource consumption value of the task and the resource available information of the processing node are considered simultaneously in the selection process of the target processing node, the resource consumption of the task can be effectively reduced when processing the task to be processed based on the target processing node. Different from the prior art that a large amount of data network transmission is required when processing data based on a distributed framework, in the present application, each sub-data is distributed and stored in multiple processing nodes, and when the target processing node can directly analyze and calculate based on the local data, the network loss of data transmission is further reduced, and the analysis and calculation efficiency of big data operation and maintenance is effectively improved.

[0095] A possible implementation manner is provided in the embodiment of the present application. Before selecting the target processing node from multiple processing nodes according to the estimated resource consumption value for processing the task to be processed, it includes:

[0096] S301. For each sub-data, determine the unique encoding value of the sub-data according to the attribute information of the sub-data.

[0097] Among them, the attribute information of each sub-data includes data type, the organization corresponding to the data, the object type of data storage, object identifier, timestamp, etc. The unique encoding value of the sub-data may be a hash value.

[0098] A hash value refers to a method of creating a small digital "fingerprint" from arbitrary data. A message or data can be compressed into a digest through a hash function, reducing the amount of data and fixing the data format. The function shuffles and mixes the data to recreate a fingerprint called a hash value (hash values, hash codes, hash sums, or hashes). Hash values are usually represented by a short string of random letters and numbers. Since hash values are numerical values obtained through logical operations based on the attribute information of sub-data, different sub-data will have different hash values. Therefore, the hash value can be used as the ID card of the sub-data in the data source.

[0099] S302. Calculate the index value of the sub-data according to the unique coding value, and use the processing node corresponding to the index value as the processing node for storing the sub-data.

[0100] Among them, each processing node corresponds to an index value of a sub-data, and the corresponding relationship between the processing node and the index value can be pre-constructed.

[0101] In some embodiments, a server or terminal for data processing can determine the index value of the sub-data according to the unique coding value of the sub-data and the number of processing nodes, and then determine the processing node for storing the sub-data according to the above corresponding relationship.

[0102] In other embodiments, a server or terminal for data processing can first determine the total number of partitions of the data source corresponding to the data required to complete the task to be processed, and then use the remainder obtained by dividing the unique coding value of the sub-data by the total number of partitions as the index value of the sub-data, and further determine the processing node for storing the sub-data based on the above corresponding relationship.

[0103] In the embodiments of the present application, each sub-data included in the data required to process the task to be processed can be distributed and stored in each processing node based on a dynamic load balancing strategy. The dynamic load balancing strategy can monitor the status of the processing node cluster, quantify the performance differences of different processing nodes, and periodically adjust the proportion of processing nodes to achieve efficient scheduling of data storage, improve the speed of data reading and writing, and provide strong support for data scheduling during subsequent processing tasks.

[0104] In the embodiments of the present application, a possible implementation manner is provided, as Figure 3 shown. Screening out target processing nodes from multiple processing nodes according to the estimated resource consumption for processing the task to be processed includes:

[0105] S401. Determine the available resource information of each processing node.

[0106] Among them, the resource availability information includes CPU availability information and memory availability information; correspondingly, the above resource consumption estimations include CPU consumption estimations and memory consumption estimations.

[0107] S402. For each processing node, determine the surplus degree of the resource availability information of the processing node relative to the resource consumption estimation.

[0108] Specifically, for a server or terminal used for data processing, it can estimate the resource consumption estimations required to process the to-be-processed tasks, and respectively determine the CPU surplus degree of the CPU availability information relative to the CPU consumption estimation, and the memory surplus degree of the memory availability information relative to the memory consumption estimation, and then determine the surplus degree of the processing node according to the CPU surplus degree and the memory surplus degree.

[0109] S403. When there is at least one reference processing node among the processing nodes, screen out the target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes; among them, the reference processing node is a processing node with a surplus degree greater than a second threshold; the second threshold is not greater than the first threshold.

[0110] Specifically, it is possible to first screen out the reference processing nodes with a surplus degree greater than the second threshold from the processing nodes, and then determine the target processing nodes from the reference processing nodes according to the first threshold.

[0111] Among them, the first threshold and the second threshold can be determined according to the actual engineering application situation of the processing node. In the embodiments of the present application, the second threshold can be set to 0. When the surplus degree of the processing node is greater than 0, it indicates that the processing node has sufficient resources to process the to-be-processed task; when the surplus degree of the processing node is less than 0, it indicates that the resource availability information of the processing node cannot support the processing of the to-be-processed task.

[0112] At the same time, the first threshold can be set to the surplus degree of the reference processing node ranked second, where each reference processing node can be arranged in descending order according to the surplus degree. That is to say, the reference processing node with the largest surplus degree can be selected as the target processing node.

[0113] In the embodiments of the present application, by the surplus degree of the resource availability information of the processing node relative to the resource consumption estimation, each processing node is screened to obtain the target processing nodes whose surplus degree meets the preset threshold; according to the calculation of the surplus degree in the embodiments of the present application, the screening process of the processing node cluster is optimized, ensuring the reliability of the target processing node when processing tasks.

[0114] In an embodiment of the present application, a possible implementation manner is provided. The types of the above-mentioned sub-data include primary and backup. The processing nodes storing the primary sub-data of each item are primary-level processing nodes, the processing nodes storing the backup sub-data of each item are secondary-level processing nodes, and the processing nodes other than the primary-level and secondary-level processing nodes are tertiary-level processing nodes.

[0115] Screening out target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes includes:

[0116] When the reference processing nodes include primary-level processing nodes, screening out target processing nodes with a surplus degree greater than a preset first threshold from the primary-level processing nodes.

[0117] Specifically, when the reference processing nodes include primary-level processing nodes, it means that among the processing nodes with sufficient resources to process tasks, there are primary-level processing nodes. Regardless of whether the reference processing nodes include secondary-level or tertiary-level processing nodes, directly screen out target processing nodes from the primary-level processing nodes.

[0118] In an embodiment of the present application, a possible implementation manner is provided. The above method further includes:

[0119] When the reference processing nodes include secondary-level processing nodes and do not include primary-level processing nodes, screening out target processing nodes with a surplus degree greater than a preset first threshold from the secondary-level processing nodes;

[0120] When the reference processing nodes only include tertiary-level processing nodes, screening out target processing nodes with a surplus degree greater than a preset first threshold from the tertiary-level processing nodes.

[0121] In an embodiment of the present application, since the primary sub-data is stored in the primary-level processing nodes, when the primary-level processing nodes process the tasks to be processed, they can directly call the primary sub-data to reduce the network transmission loss during data scheduling. Therefore, the priorities of the primary-level, secondary-level, and tertiary-level processing nodes can be set to decrease gradually to increase the efficiency of data scheduling. As Figure 4 shown, when there are primary-level, secondary-level, and tertiary-level processing nodes in the reference processing nodes, first screen out target processing nodes from the primary-level processing nodes; when no target processing nodes are screened out from the primary-level processing nodes, then screen out target processing nodes from the secondary-level processing nodes; when no target processing nodes are screened out from the secondary-level processing nodes, then screen out target processing nodes from the tertiary-level processing nodes; if no target processing nodes are screened out from the tertiary-level processing nodes, wait for a preset duration and then screen again.

[0122] In an embodiment of the present application, a possible implementation manner is provided. The above resource availability information includes: the number of available CPU cores and the size of available memory;

[0123] Determine the surplus degree of the available resource information of the processing node relative to the estimated resource consumption, including:

[0124] S501, perform a weighted sum of the available CPU cores and the available memory size to obtain the resource surplus value of the processing node.

[0125] Specifically, for a server or terminal used for data processing, the total resources of the processing node can be determined, where the total resources include the total number of CPU cores and the total memory. The available CPU cores are determined based on the total number of CPU cores and the occupied CPU cores. A weighted sum of the available CPU cores, the occupied CPU cores, the available memory size, and the occupied memory size can be performed to obtain the resource surplus value of the processing node. The specific calculation process will be introduced in detail below.

[0126] S502, weight the resource surplus value according to the difference between the available resource information and the estimated resource consumption to obtain the surplus degree.

[0127] Among them, the estimated resource consumption can include the estimated CUP consumption and the estimated memory consumption.

[0128] Specifically, the weight can be determined based on the difference between the available CPU cores and the estimated CPU consumption, as well as the difference between the available memory size and the estimated memory consumption, and the resource surplus value is weighted based on this weight to obtain the surplus degree.

[0129] In an embodiment of the present application, a possible implementation manner is provided. The above-mentioned weighted sum of the available CPU cores and the available memory size to obtain the resource surplus value of the processing node includes:

[0130] S601, determine the resource surplus base number of the processing node according to the type of the processing node; among them, the resource surplus base number of the first-level processing node is greater than that of the second-level processing node, and the resource surplus base number of the second-level processing node is greater than that of the third-level processing node.

[0131] In an embodiment of the present application, the resource surplus base number of the first-level processing node can be set to 50, the surplus base number of the second-level processing node is 35, and the surplus base number of the third-level processing node is 0.

[0132] S602, perform a weighted sum of the resource surplus base number, the available CPU cores, and the available memory size to obtain the resource surplus value of the processing node.

[0133] In an embodiment of the present application, the resource surplus value N of the processing node can be calculated according to the following formula:

[0134] N = DM * 50 + DN * 35 + ((CC * 2) - REC) * 25 - REC * 5 + (AR - RER) * 5 (1)

[0135] Wherein, when the processing node is a primary processing node, DM = 1, otherwise 0;

[0136] When the processing node is a secondary processing node, DN = 1, otherwise 0;

[0137] CC is the total number of CPU cores of the processing node, REC is the number of occupied cores of the CPU for other tasks processed by the processing node, AR is the available memory size of the processing node, and RER is the occupied memory size for other tasks processed by the processing node.

[0138] Meanwhile, the weight of the resource surplus value can be determined according to the following formula:

[0139] T = AR > ER || (CC * 2 - REC) > EC (2)

[0140] Wherein, EC is the predicted value of the CUP consumption of the task to be executed; ER is the predicted value of the memory consumption of the task to be executed. The logical OR operation is performed on the two propositions AR > ER and (CC * 2 - REC) > EC. When at least one proposition is true, T is a true proposition, and at this time T = 1, otherwise T = 0.

[0141] In the embodiment of the present application, after obtaining the resource surplus value N and the weight T of the resource surplus value of the processing node, when the first threshold is set to the surplus degree of the processing node ranked second, and each reference processing node can be sorted in descending order according to the surplus degree, the target processing node can be determined based on the following formula:

[0142] E = max(N1 * T1, N2 * T2, N3 * T3,...., Ni * Ti) (3)

[0143] Wherein, E is the target processing node, Ni and Ti are the resource surplus value and weight of each processing node, and i is the total number of processing nodes in the node cluster.

[0144] When T1 to Ti are all 0, the target processing node cannot be obtained, and then after waiting for a preset time period, each processing node can be screened based on the above method.

[0145] For a better understanding of the above data processing method, the following combines Figure 5 A detailed example of the data processing method of the present application is elaborated. This method can be applied to a resource scheduling server, and the resource scheduling server can include a storage module, an algorithm module, and a resource scheduling module; the method includes the following steps:

[0146] S701, the resource scheduling module partitions a preset data source to obtain multiple main blocks without overlapping data; wherein, the data source includes data required for multiple processing tasks.

[0147] Wherein, the data source can be a data table or a Schema set.

[0148] S702, the resource scheduling module constructs at least one backup block for each main block, and each main block or backup block corresponds to a sub-data; all sub-data are distributed and stored in multiple processing nodes of the processing cluster.

[0149] Wherein, the above-mentioned processing nodes can be each processing node in a task processing server cluster or a database cluster.

[0150] The types of the above-mentioned sub-data include primary and backup; the processing nodes storing each primary sub-data are primary-level processing nodes, the processing nodes storing each backup sub-data are secondary-level processing nodes, and the processing nodes other than the primary-level processing nodes and the secondary-level processing nodes are tertiary-level processing nodes.

[0151] S703, the resource scheduling module determines the data required to complete the to-be-processed task, and the multiple sub-data included in the data; for each sub-data, according to the attribute information of the sub-data, determines the unique encoding value of the sub-data.

[0152] Wherein, the attribute information of each sub-data includes data type, the organization corresponding to the data, the object type stored by the data, object identifier, timestamp, etc. The unique encoding value of the sub-data can be a hash value.

[0153] S704, the resource scheduling module calculates the index value of the sub-data according to the unique encoding value, and uses the processing node corresponding to the index value as the processing node for storing the sub-data.

[0154] Wherein, each processing node is configured with a storage module for storing data;

[0155] Each processing node corresponds to an index value of a sub-data, and the corresponding relationship between the processing node and the index value can be pre-constructed, and the processing node corresponding to the index value is determined according to this corresponding relationship.

[0156] S705, for each primary-level processing node, determines the resource availability information of each primary-level processing node, and determines the surplus degree of the resource availability information of each primary-level processing node relative to the estimated resource consumption.

[0157] S706, when there is at least one reference processing node among the primary-level processing nodes, filters out the target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes; wherein, the reference processing nodes are the processing nodes with a surplus degree greater than a second threshold; the second threshold is not greater than the first threshold.

[0158] When there is no reference processing node in the primary processing node and there is at least one reference processing node in the secondary processing node, screen out the target processing node with the largest surplus degree from the secondary processing nodes.

[0159] When there is no reference processing node in the secondary processing node and there is at least one reference processing node in the tertiary processing node, screen out the target processing node with the largest surplus degree from the tertiary processing nodes.

[0160] When there is no reference processing node in the primary, secondary, and tertiary processing nodes, after waiting for a preset duration, return to step S705 to continue screening until the target processing node is determined.

[0161] S707, process the task to be processed according to the algorithm module in the target processing node to obtain a processing result.

[0162] Wherein, each processing node is configured with an algorithm module, and the algorithm module is used to call the data in the storage module and perform different algorithms according to different tasks to be processed for data reasoning and analysis to obtain a calculation result, that is, the processing result of the task to be processed.

[0163] In the embodiment of the present application, by distributing and storing multiple sub-data included in the data required to complete the task to be processed in multiple processing nodes, the surplus degree of each processing node can then be determined according to the estimated resource consumption value and the resource available information of each processing node, and then the target processing node with a surplus degree greater than the first threshold is screened out from the multiple processing nodes to process the task to be processed based on the target processing node to obtain a processing result; in the embodiment of the present application, since the estimated resource consumption value of the task and the resource available information of the processing node are considered simultaneously during the screening process of the target processing node, when processing the task to be processed based on the target processing node, the resource consumption of the task can be effectively reduced. Different from the prior art, when data processing is performed based on a distributed framework, a large amount of data network transmission is required. In the present application, each sub-data is distributed and stored in multiple processing nodes, and when the target processing node can directly analyze and calculate based on the local data, the network loss of data transmission is further reduced, effectively improving the analysis and calculation efficiency of big data operation and maintenance.

[0164] The embodiment of the present application provides a data processing device, as Figure 6 shown, the data processing device 60 may include: a determination module 601, a screening module 602, and a processing module 603;

[0165] Wherein, the determination module 601 is used to determine the data required to complete the task to be processed; wherein, the data includes multiple sub-data, and the multiple sub-data are distributed and stored in multiple processing nodes;

[0166] A screening module 602, configured to screen out a target processing node from multiple processing nodes according to an estimated value of resource consumption for processing a to-be-processed task; wherein, the surplus degree of the target processing node is greater than a first preset threshold; the surplus degree is determined according to the estimated value of resource consumption and the resource availability information of each processing node;

[0167] A processing module 603, configured to process the to-be-processed task according to the target processing node to obtain a processing result.

[0168] In a possible implementation manner provided in an embodiment of the present application, before the screening module 602 screens out a target processing node from multiple processing nodes according to the estimated value of resource consumption for processing the to-be-processed task, it is configured to:

[0169] For each sub-data, determine a unique coding value of the sub-data according to the attribute information of the sub-data;

[0170] Calculate an index value of the sub-data according to the unique coding value, and use the processing node corresponding to the index value as the processing node for storing the sub-data.

[0171] In a possible implementation manner provided in an embodiment of the present application, when the screening module 602 screens out a target processing node from multiple processing nodes according to the estimated value of resource consumption for processing the to-be-processed task, it is configured to:

[0172] Determine the resource availability information of each processing node;

[0173] For each processing node, determine the surplus degree of the resource availability information of the processing node relative to the estimated value of resource consumption;

[0174] When there is at least one reference processing node among the processing nodes, screen out a target processing node with a surplus degree greater than a preset first threshold from the reference processing nodes; wherein, the reference processing node is a processing node with a surplus degree greater than a second threshold; the second threshold is not greater than the first threshold.

[0175] In a possible implementation manner provided in an embodiment of the present application, the types of the above sub-data include primary copies and backups; the processing nodes storing each primary copy sub-data are first-level processing nodes, the processing nodes storing each backup sub-data are second-level processing nodes, and the processing nodes other than the first-level processing nodes and the second-level processing nodes are third-level processing nodes;

[0176] When the screening module 602 screens out a target processing node with a surplus degree greater than a preset first threshold from the reference processing nodes, it is configured to:

[0177] When the reference processing nodes include first-level processing nodes, screen out a target processing node with a surplus degree greater than a preset first threshold from the first-level processing nodes.

[0178] In an embodiment of the present application, a possible implementation is provided. The above-mentioned screening module 602 is further configured to:

[0179] When the reference processing node includes a secondary processing node and does not include a primary processing node, screen out target processing nodes from the secondary processing nodes whose surplus degree is greater than a preset first threshold;

[0180] When the reference processing node only includes a tertiary processing node, screen out target processing nodes from the tertiary processing nodes whose surplus degree is greater than a preset first threshold.

[0181] In an embodiment of the present application, a possible implementation is provided. The above-mentioned available resource information includes: the available number of CPU cores and the available memory size;

[0182] When the above-mentioned screening module 602 determines the surplus degree of the available resource information of the processing node relative to the estimated value of resource consumption, it is configured to:

[0183] Perform a weighted sum of the available number of CPU cores and the available memory size to obtain the resource surplus value of the processing node;

[0184] According to the difference between the available resource information and the estimated value of resource consumption, weight the resource surplus value to obtain the surplus degree.

[0185] In an embodiment of the present application, a possible implementation is provided. When the above-mentioned screening module 602 performs a weighted sum of the available number of CPU cores and the available memory size to obtain the resource surplus value of the processing node, it is configured to:

[0186] Determine the resource surplus base number of the processing node according to the type of the processing node; wherein, the resource surplus base number of the primary processing node is greater than that of the secondary processing node, and the resource surplus base number of the secondary processing node is greater than that of the tertiary processing node;

[0187] Perform a weighted sum of the resource surplus base number, the available number of CPU cores and the available memory size to obtain the resource surplus value of the processing node.

[0188] The device of the embodiment of the present application can execute the method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed function description of each module of the device, reference can be specifically made to the description in the corresponding method shown above, and details are not described herein again.

[0189] In the embodiments of the present application, by distributing and storing multiple sub - data included in the data required to complete a task to be processed in multiple processing nodes, the surplus degree of each processing node can then be determined according to the estimated value of resource consumption and the available resource information of each processing node. Furthermore, target processing nodes with a surplus degree greater than a first threshold are selected from the multiple processing nodes to process the task to be processed based on the target processing nodes, and a processing result is obtained. In the embodiments of the present application, since the estimated value of resource consumption of the task and the available resource information of the processing nodes are considered simultaneously during the selection process of the target processing nodes, when processing the task to be processed based on the target processing nodes, the resource consumption of this task can be effectively reduced. Different from the prior art where a large amount of data network transmission is required when performing data processing based on a distributed framework, in the present application, each sub - data is distributed and stored in multiple processing nodes. When the target processing nodes can directly analyze and calculate based on the local data, the network loss of data transmission is further reduced, and the analysis and calculation efficiency of big data operation and maintenance is effectively improved.

[0190] In the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory. The processor executes the above - mentioned computer program to implement the steps of the data processing method. Compared with the related art, it can be achieved that: in the embodiments of the present application, by distributing and storing multiple sub - data included in the data required to complete a task to be processed in multiple processing nodes, the surplus degree of each processing node can then be determined according to the estimated value of resource consumption and the available resource information of each processing node. Furthermore, target processing nodes with a surplus degree greater than a first threshold are selected from the multiple processing nodes to process the task to be processed based on the target processing nodes, and a processing result is obtained. In the embodiments of the present application, since the estimated value of resource consumption of the task and the available resource information of the processing nodes are considered simultaneously during the selection process of the target processing nodes, when processing the task to be processed based on the target processing nodes, the resource consumption of this task can be effectively reduced. Different from the prior art where a large amount of data network transmission is required when performing data processing based on a distributed framework, in the present application, each sub - data is distributed and stored in multiple processing nodes. When the target processing nodes can directly analyze and calculate based on the local data, the network loss of data transmission is further reduced, and the analysis and calculation efficiency of big data operation and maintenance is effectively improved.

[0191] In an alternative embodiment, an electronic device is provided, as Figure 7 shown Figure 7The electronic device 700 shown includes: a processor 701 and a memory 703. Among them, the processor 701 and the memory 703 are connected, such as through a bus 702. Optionally, the electronic device 700 may further include a transceiver 704, and the transceiver 704 can be used for data interaction between this electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in practical applications, the transceiver 704 is not limited to one, and the structure of the electronic device 700 does not constitute a limitation to the embodiments of the present application.

[0192] The processor 701 may be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in combination with the disclosure of the present application. The processor 701 may also be a combination that implements a computing function, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0193] The bus 702 may include a path for transmitting information between the above components. The bus 702 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The bus 702 may be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 7 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0194] The memory 703 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited herein.

[0195] The memory 703 is used to store the computer program for implementing the embodiments of the present application and is controlled by the processor 701 for execution. The processor 701 is used to execute the computer program stored in the memory 703 to implement the steps shown in the foregoing method embodiments.

[0196] Among them, the electronic device includes but is not limited to: mobile terminals such as mobile phones, laptop computers, PADs, etc., and fixed terminals such as digital TVs, desktop computers, etc.

[0197] The embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

[0198] The embodiments of the present application provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that when the computer device executes, the following situations are realized:

[0199] Determine the data required to complete the task to be processed; wherein, the data includes multiple sub-data, and the multiple sub-data are distributed and stored in multiple processing nodes;

[0200] According to the estimated value of resource consumption for processing the task to be processed, screen out the target processing nodes from the multiple processing nodes; wherein, the surplus degree of the target processing node is greater than a preset first threshold; the surplus degree is determined according to the estimated value of resource consumption and the resource available information of each processing node;

[0201] Process the task to be processed according to the target processing node to obtain a processing result.

[0202] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the description, claims and the above-mentioned drawings of this application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than the one illustrated or described in words.

[0203] It should be understood that although the flowcharts of the embodiments of this application indicate each operation step by an arrow, the execution order of these steps is not limited to the order indicated by the arrow. Unless there is a clear description in this article, in some implementation scenarios of the embodiments of this application, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage of these sub-steps or stages can also be executed at different times. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of this application do not limit this.

[0204] The above are only optional implementation manners of some implementation scenarios of this application. It should be noted that for those of ordinary skill in the art, without departing from the technical concept of the solution of this application, adopting other similar implementation means based on the technical idea of this application also belongs to the protection scope of the embodiments of this application.

Claims

1. A data processing method, characterized in that, including: determining data required to complete a task to be processed; wherein the data includes a plurality of sub-data, and the plurality of sub-data are distributed and stored in a plurality of processing nodes; screening target processing nodes from the plurality of processing nodes according to a predicted value of resource consumption for processing the task to be processed; wherein the surplus degree of the target processing node is greater than a preset first threshold; the surplus degree is determined according to the predicted value of resource consumption and the resource availability information of each processing node; processing the task to be processed according to the target processing node to obtain a processing result; the screening target processing nodes from the plurality of processing nodes according to the predicted value of resource consumption for processing the task to be processed includes: determining the resource availability information of each processing node; for each processing node, determining the surplus degree of the resource availability information of the processing node relative to the predicted value of resource consumption; when there is at least one reference processing node among the processing nodes, screening target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes; wherein the reference processing node is a processing node with a surplus degree greater than a second threshold; the second threshold is not greater than the first threshold; the types of the sub-data include primary copies and backups; the processing nodes storing the primary copy sub-data are primary-level processing nodes, the processing nodes storing the backup sub-data are secondary-level processing nodes, and the processing nodes other than the primary-level processing nodes and the secondary-level processing nodes are tertiary-level processing nodes; the screening target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes includes: when the reference processing nodes include primary-level processing nodes, screening target processing nodes with a surplus degree greater than a preset first threshold from the primary-level processing nodes.

2. The method according to claim 1, wherein before the screening target processing nodes from the plurality of processing nodes according to the predicted value of resource consumption for processing the task to be processed, it includes: for each sub-data, determining a unique coding value of the sub-data according to the attribute information of the sub-data; calculating an index value of the sub-data according to the unique coding value, and using the processing node corresponding to the index value as the processing node storing the sub-data.

3. The method according to claim 1, characterized in that, also including: when the reference processing nodes include secondary-level processing nodes and do not include primary-level processing nodes, screening target processing nodes with a surplus degree greater than a preset first threshold from the secondary-level processing nodes; when the reference processing nodes only include tertiary-level processing nodes, screening target processing nodes with a surplus degree greater than a preset first threshold from the tertiary-level processing nodes.

4. The method according to claim 1, characterized in that, the resource availability information includes: the number of available CPU cores and the size of available memory; the determining the surplus degree of the resource availability information of the processing node relative to the predicted value of resource consumption includes: performing a weighted sum of the number of available CPU cores and the size of available memory to obtain a resource surplus value of the processing node; weighting the resource surplus value according to the difference between the resource availability information and the predicted value of resource consumption to obtain the surplus degree.

5. The method according to claim 4, characterized in that, Performing a weighted sum of the available CPU cores and the available memory size of the CPU to obtain the resource surplus value of the processing node, including: Determining the resource surplus base of the processing node according to the type of the processing node; wherein, the resource surplus base of the first-level processing node is greater than that of the second-level processing node, and the resource surplus base of the second-level processing node is greater than that of the third-level processing node; Performing a weighted sum of the resource surplus base, the available CPU cores, and the available memory size to obtain the resource surplus value of the processing node.

6. A data processing device, characterized in that, Including: A determination module, configured to determine the data required to complete the task to be processed; wherein, the data includes a plurality of sub-data, and the plurality of sub-data are distributed and stored in a plurality of processing nodes; A screening module, configured to screen out target processing nodes from the plurality of processing nodes according to the estimated resource consumption for processing the task to be processed; wherein, the surplus degree of the target processing node is greater than a preset first threshold; the surplus degree is determined according to the estimated resource consumption and the resource available information of each processing node; A processing module, configured to process the task to be processed according to the target processing node to obtain a processing result; The screening out target processing nodes from the plurality of processing nodes according to the estimated resource consumption for processing the task to be processed includes: Determining the resource available information of each processing node; For each processing node, determining the surplus degree of the resource available information of the processing node relative to the estimated resource consumption; When there is at least one reference processing node among the processing nodes, screening out target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes; wherein, the reference processing node is a processing node with a surplus degree greater than a second threshold; the second threshold is not greater than the first threshold; The types of the sub-data include primary copies and backups; the processing nodes storing the primary sub-data are first-level processing nodes, the processing nodes storing the backup sub-data are second-level processing nodes, and the processing nodes other than the first-level processing nodes and the second-level processing nodes are third-level processing nodes; The screening out target processing nodes with a surplus degree greater than a preset first threshold from the reference processing nodes includes: When the reference processing nodes include first-level processing nodes, screening out target processing nodes with a surplus degree greater than a preset first threshold from the first-level processing nodes.

7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images