Computing Scheduling System, Method and Medium Based on Industrial Internet of Things Data Center

Abstract

The present invention provides a computing scheduling system, method and medium based on an industrial Internet of Things data center, which relates to the field of computing scheduling. The system includes a management platform, a data center and multiple data sub-platforms that are communicatively connected. The management platform is configured to: obtain operation characteristics; determine a first sub-platform and a second sub-platform based on the operation characteristics; determine at least one service to be split based on the service characteristics and operation characteristics corresponding to multiple services of the first sub-platform, split the at least one service to be split into multiple sub-services, and determine the processing resource requirements corresponding to the multiple sub-services; determine service adjustment parameters based on the processing resource requirements and the operation characteristics. The present invention can split some services of a data sub-platform with a large computing load and allocate the split services to other data sub-platforms with a lighter load, so as to more reasonably utilize the computing resources of multiple data sub-platforms and improve the processing efficiency of services.

Description

Technical Field

[0001] The present invention relates to the field of computing scheduling, and particularly to a computing scheduling system, method and medium based on an industrial Internet of Things data center. Background Art

[0002] Each data sub-platform of the industrial Internet of Things usually relies on its own computing power to process services. However, different services have different data volumes, and services with different data volumes occupy different computing resources of the data sub-platforms. Thus, in the operation of the industrial Internet of Things, there may be a situation where the computing resources of some data sub-platforms cannot meet the requirements for processing services, while the computing resources of some data sub-platforms are idle.

[0003] In view of this, it is necessary to provide a computing scheduling system, method and medium based on an industrial Internet of Things data center, which can split and schedule services of different data sub-platforms, make full use of the computing resources of each data sub-platform, and balance the computing pressure of each data sub-platform. Summary of the Invention

[0004] The summary of the invention includes a computing scheduling system based on an industrial Internet of Things data center. The system includes a management platform, a data center and a plurality of data sub-platforms connected by communication. The management platform is configured to: obtain the operation characteristics of the plurality of data sub-platforms; based on the operation characteristics, determine a first sub-platform and a second sub-platform among the plurality of data sub-platforms; based on the service characteristics corresponding to a plurality of services of the first sub-platform and the operation characteristics, determine at least one service to be split; based on the service characteristics of the at least one service to be split, split the at least one service to be split into a plurality of sub-services; based on the service characteristics of the at least one service to be split, the operation characteristics and the sub-service characteristics corresponding to the plurality of sub-services, determine the processing resource requirements corresponding to the plurality of sub-services; based on the processing resource requirements and the operation characteristics, determine service adjustment parameters; based on the service adjustment parameters, generate a service adjustment instruction and send it to the first sub-platform and the second sub-platform; the first sub-platform is configured to: based on the service adjustment instruction, perform data processing on the plurality of sub-services to obtain a plurality of data packets; send the plurality of data packets to the second sub-platform; the second sub-platform is configured to: receive the plurality of data packets; analyze the plurality of data packets to determine the data volume corresponding to the plurality of data packets; based on the data volume, allocate corresponding parsing memory for the plurality of data packets and analyze the plurality of data packets to obtain parsed data; based on the parsed data, establish processes corresponding to the plurality of sub-services; and based on the service adjustment instruction, determine the processing resources for processing the processes.

[0005] The invention content includes a computing scheduling method based on an industrial Internet of Things data center. The method is executed by a management platform and includes: obtaining the operating characteristics of multiple data sub-platforms; determining a first sub-platform and a second sub-platform among the multiple data sub-platforms based on the operating characteristics; determining at least one service to be split based on the service characteristics corresponding to multiple services of the first sub-platform and the operating characteristics; splitting the at least one service to be split into multiple sub-services based on the service characteristics of the at least one service to be split; determining the processing resource requirements corresponding to the multiple sub-services based on the service characteristics of the at least one service to be split, the operating characteristics, and the sub-service characteristics corresponding to the multiple sub-services; determining service adjustment parameters based on the processing resource requirements and the operating characteristics; generating a service adjustment instruction based on the service adjustment parameters and sending it to the first sub-platform and the second sub-platform.

[0006] The invention content includes a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the above-mentioned computing scheduling method based on an industrial Internet of Things data center.

[0007] The beneficial effects of the present invention include but are not limited to: (1) Through the computing scheduling system based on an industrial Internet of Things data center, the services of different data sub-platforms can be split and service adjustment between different data sub-platforms can be achieved. (2) It is possible to split some services of a data sub-platform with a large computing load and allocate the split services to other data sub-platforms with a lighter load, which can more reasonably utilize the computing resources of multiple data sub-platforms and improve the processing efficiency of services. (3) By using a machine learning model to process the operating characteristics and the processing resource requirements corresponding to multiple candidate sub-services, find the rules from a large amount of data, and obtain the correlation between the operating characteristics, the processing resource requirements corresponding to multiple candidate sub-services, and the service operation efficiency, thereby improving the accuracy and efficiency of determining the service operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present invention will be further described by way of exemplary embodiments, which will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0009] Figure 1 is a schematic diagram of the platform structure of a computing scheduling system based on an industrial Internet of Things data center shown in some embodiments of this specification;

[0010] Figure 2 is an exemplary flowchart of a computing scheduling method based on an industrial Internet of Things data center shown in some embodiments of this specification;

[0011] Figure 3 is an exemplary flowchart for splitting a service to be split according to some embodiments of this specification;

[0012] Figure 4 is an exemplary schematic diagram of an efficiency prediction model according to some embodiments of this specification;

[0013] Figure 5 is an exemplary flowchart for determining service adjustment parameters according to some embodiments of this specification. Detailed implementation manners

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. The accompanying drawings do not represent all implementation manners.

[0015] In the embodiments of the present invention, when the operations performed are described step by step, unless otherwise specified, the order of the steps can be adjusted, the steps can be omitted, and other steps can also be included during the operation process.

[0016] Figure 1 is a schematic diagram of the platform structure of a computing scheduling system based on an industrial Internet of Things data center according to some embodiments of this specification.

[0017] As Figure 1 shown, the computing scheduling system 100 based on the industrial Internet of Things data center includes a management platform 110, a data center 120, and a plurality of data sub-platforms (such as data sub-platform 131, data sub-platform 132, data sub-platform 133, data sub-platform 134, etc.) that are communicatively connected.

[0018] The management platform refers to a platform that processes information and / or instructions related to the computing scheduling system 100 based on the industrial Internet of Things data center. In some embodiments, the management platform is configured on a server or a processor, etc.

[0019] In some embodiments, the processor includes a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction processor (ASIP), etc., or any combination thereof.

[0020] In some embodiments, the management platform includes the data center 120.

[0021] The data center is configured to store and manage information and / or instructions related to the computing scheduling system 100 based on the industrial Internet of Things data center. The data center is configured to store devices, etc.

[0022] A data sub - platform refers to a platform used to complete operations. Different data sub - platforms are used to complete multiple different operations. An operation refers to processing data such as computing or analyzing it. For example, pre - processing data or analyzing the product quality of each product in product data. Pre - processing includes cleaning, normalization, etc.

[0023] In some embodiments, the data sub - platform obtains various data required to execute operations from the data center. For example, data items to be processed, etc.

[0024] Multiple data sub - platforms can be communicatively connected. In some embodiments, the data sub - platform is configured on a server, a processor, etc.

[0025] For the foregoing detailed description, reference can be made to Figures 2 to 5 the relevant description.

[0026] Through the computing scheduling system 100 based on the industrial Internet of Things data center, the operations of different data sub - platforms can be split, and the operation adjustment between different data sub - platforms can be realized.

[0027] Figure 2 is an exemplary flowchart of the computing scheduling method based on the industrial Internet of Things data center shown in some embodiments of this specification. In some embodiments, the process 200 of the computing scheduling method based on the industrial Internet of Things data center is executed by the management platform. As Figure 2 shown, the process 200 of the computing scheduling method based on the industrial Internet of Things data center includes the following steps.

[0028] For the description of the computing scheduling system based on the industrial Internet of Things data center and each platform, reference can be made to Figure 1 the corresponding description.

[0029] Step 210, obtain the operation characteristics of multiple data sub - platforms.

[0030] The operation characteristics refer to data related to the operation of the data sub - platform. Each data sub - platform corresponds to an operation characteristic.

[0031] In some embodiments, the operation characteristics include the total CPU usage rate of the data sub - platform, the total memory occupancy rate, the CPU usage rate and memory occupancy rate respectively corresponding to each operation in the data sub - platform, etc.

[0032] In some embodiments, the management platform obtains the process operation table of the data sub - platform through the data sub - platform, and then obtains the operation characteristics of the data sub - platform.

[0033] The process operation table refers to a data table automatically generated when the data sub - platform processes operations, which shows the processing resources used by each operation. The processing resources include CPU resources, memory resources, etc.

[0034] Step 220: Based on the running characteristics, determine the first sub-platform and the second sub-platform among multiple data sub-platforms.

[0035] The first sub-platform refers to the data sub-platform that needs to schedule services to other data sub-platforms.

[0036] The second sub-platform refers to other data sub-platforms except the first sub-platform.

[0037] In some embodiments, the management platform may use the data sub-platforms that meet the preset screening conditions among multiple data sub-platforms as the first sub-platform, and use the other data sub-platforms except the first sub-platform as the second sub-platform. The preset screening conditions include that the total CPU usage rate and / or the total memory occupancy rate of the data sub-platform are greater than the preset screening threshold. The preset screening threshold is set in advance based on historical experience. The preset screening conditions may also include that the total CPU usage rate of the data sub-platform is the highest.

[0038] Step 230: Based on the service characteristics and running characteristics of multiple services corresponding to the first sub-platform, determine at least one service to be split.

[0039] The service characteristic refers to the data used to reflect service-related information. One service corresponds to one service characteristic. In some embodiments, the service characteristic includes the function items corresponding to the service and the number of data items corresponding to the function items.

[0040] The function item is used to characterize the function of the service. One service includes one or more function items. The function items include preprocessing each data or analyzing the product quality of each product in the product data. The preprocessing includes cleaning, normalization, etc.

[0041] The data item refers to the data corresponding to the execution of the function item. For example, if the function item is to preprocess each data, the data item corresponding to the function item is each data that needs to be preprocessed. One function item corresponds to multiple data items.

[0042] The service to be split refers to the service that needs to be split in the first sub-platform.

[0043] In some embodiments, the management platform calculates the splitting degree of each service based on the service characteristics and running characteristics of multiple services corresponding to the first sub-platform, sorts the multiple splitting degrees, and determines the services that meet the splitting degree conditions as the services to be split. Among them, the splitting degree conditions include that the ranking in the splitting degree sorting is among the top N. N is set in advance based on historical experience.

[0044] The splitting degree refers to the data used to characterize the degree to which a service needs to be split. In some embodiments, the management platform calculates the service characteristics and operation characteristics corresponding to a single service to obtain the splitting degree of the single service. Exemplarily, the management platform calculates the splitting degree of the service through a preset formula. Exemplarily, the preset formula is as shown in formula (1):

[0045] (1)

[0046] Wherein, is the splitting degree, is the number of function items, is the number of data items, is the CPU usage rate of the service, is the memory occupancy rate of the service. ~ are preset coefficients. ~ Are set according to experience and are all greater than 0.

[0047] Step 240, based on the service characteristics of at least one service to be split, split at least one service to be split into multiple sub-services.

[0048] A sub-service refers to the service obtained after splitting the service to be split.

[0049] In some embodiments, the number of sub-services obtained after splitting is determined in various ways. For example, the number of sub-services is equal to or less than the number of data sub-platforms. Also for example, the number of sub-services is equal to the number of data sub-platforms that meet the preset conditions. The preset conditions include that the total CPU usage rate and the total memory occupancy rate are respectively less than the corresponding occupancy rate thresholds. The occupancy rate thresholds are set according to experience.

[0050] In some embodiments, the management platform, based on the service characteristics of at least one service to be split, splits at least one service to be split into multiple sub-services in various ways. For example, the management platform splits each service to be split according to a preset rule to obtain multiple sub-services. The management platform identifies each sub-service to distinguish different sub-services.

[0051] In some embodiments, the preset rule includes preferentially splitting the service to be split based on different function items, and then splitting the sub-services obtained by splitting based on different function items based on data items to obtain multiple sub-services that meet the number of sub-services. Among them, splitting based on data items includes splitting a sub-service with a data item number greater than the preset splitting number into multiple sub-services that are all not greater than the preset splitting number. The preset splitting number is preset based on the number of sub-services. For example, the more the number of sub-services, the less the preset splitting number, etc.

[0052] Exemplarily, the number of sub-services is 3, and the preset splitting quantity is 10,000. The service to be split includes two functional items, and the number of data items corresponding to one of the functional items is 20,000. Then, the service to be split is split into two sub-services corresponding to the two functional items, and the sub-service with 20,000 data items is split into two sub-services with 10,000 data items each, resulting in 3 sub-services.

[0053] In some embodiments, the management platform may also split the service to be split based on the target splitting scheme to obtain the sub-services corresponding to the service to be split. For more details, see Figure 3 its related descriptions.

[0054] Step 250: Determine the processing resource requirements corresponding to multiple sub-services based on the service characteristics, operation characteristics of at least one service to be split, and the sub-service characteristics corresponding to the multiple sub-services.

[0055] The sub-service characteristics refer to the service characteristics corresponding to the sub-service. In some embodiments, the sub-service characteristics include the functional items corresponding to the sub-service and the number of data items, etc.

[0056] The processing resource requirements refer to the computing resources required to process the sub-service. In some embodiments, the processing resource requirements include the CPU usage rate and memory occupancy rate corresponding to the sub-service, etc.

[0057] In some embodiments, the management platform determines the processing resource requirements corresponding to the sub-service based on the service characteristics, operation characteristics of at least one service to be split, and the sub-service characteristics corresponding to the sub-service. For example, the management platform calculates the ratio of the number of data items of the sub-service to the number of data items of the functional item corresponding to the sub-service in the service to be split, and multiplies the obtained ratio by the CPU usage rate and memory occupancy rate of the service to be split in the operation characteristics to obtain the CPU usage rate and memory occupancy rate corresponding to the sub-service.

[0058] Step 260: Determine the service adjustment parameters based on the processing resource requirements and the operation characteristics.

[0059] The service adjustment parameters refer to the parameters used to allocate multiple sub-services of at least one service to be split to multiple second sub-platforms. In some embodiments, the service adjustment parameters include the sub-service characteristics corresponding to each sub-service and the allocation result, etc.

[0060] The allocation result refers to the second sub-platform to which the sub-service needs to be allocated.

[0061] In some embodiments, the management platform determines the service adjustment parameters based on the processing resource requirements and the operation characteristics. For example, the management platform determines the allocation result of each sub-service in descending order based on the position of the sub-service in the first sorting result.

[0062] The first sorting result refers to the sorting result obtained by sorting sub-services in descending order according to the processing resource requirements. In some embodiments, for each sub-service, the management platform performs a weighted sum of the CPU usage rate and memory occupancy rate of the sub-service to obtain a first sum value. The management platform sorts the first sum values corresponding to each sub-service from largest to smallest to obtain the first sorting result.

[0063] In some embodiments, based on the first sorting result, the management platform sequentially determines the allocation results of each sub-service from highest to lowest in the second sorting results of multiple data sub-platforms from highest to lowest. For example, the management platform allocates the sub-service ranked first in the first sorting result to the data sub-platform ranked first in the second sorting result. The sub-service ranked second in the first sorting result is allocated to the data sub-platform ranked first in the second sorting result. If the allocated data sub-platform ranked first meets the preset allocation conditions, the allocation result of the sub-service ranked second is determined as the data sub-platform ranked first. If the allocated data sub-platform ranked first does not meet the preset allocation conditions, the sub-service ranked second is allocated to the data sub-platform ranked second in the second sorting result, and it is continuously determined whether the allocated data sub-platform ranked second meets the preset allocation conditions. If it meets the preset allocation conditions, the allocation result of the sub-service ranked second is determined as the data sub-platform ranked second.

[0064] After determining the allocation result of the sub-service ranked second, the management platform allocates the sub-service ranked third in the first sorting result to the data sub-platform ranked first in the second sorting result, and determines the allocation result of the sub-service ranked third through the above steps. The above process is repeated until the allocation results of all ranked sub-services are determined.

[0065] In some embodiments, the preset allocation conditions include that the sum of the CPU usage rate of the allocated sub-service and the total CPU usage rate of the data sub-platform does not exceed the CPU allocation threshold of the data sub-platform, and the sum of the memory occupancy rate of the sub-service and the total memory occupancy rate of the data sub-platform does not exceed the memory allocation threshold of the data sub-platform. Exemplarily, the preset allocation conditions include that the sum of the CPU usage rate of the sub-service ranked second and the total CPU usage rate of the data sub-platform allocated the sub-service ranked first does not exceed the CPU allocation threshold of the data sub-platform, and the sum of the memory occupancy rate of the sub-service ranked second and the total memory occupancy rate of the data sub-platform ranked first does not exceed the memory allocation threshold of the data sub-platform, etc. At this time, the total CPU usage rate of the data sub-platform ranked first refers to the total CPU usage rate of the sub-service ranked first that has been allocated. The same applies to the total memory occupancy rate. The CPU allocation threshold and memory allocation threshold are set according to experience.

[0066] The second sorting result refers to the sorting result of sorting data by platform in ascending order according to the running characteristics. In some embodiments, for each data sub-platform, the management platform performs a weighted sum based on the total CPU usage rate and the total memory occupancy rate of the data sub-platform to obtain a second sum value. The management platform sorts the second sum values corresponding to each data sub-platform in ascending order to obtain the second sorting result.

[0067] The running characteristics of the first sub-platform participating in the sorting are the total CPU usage rate and the total memory occupancy rate remaining after subtracting the CPU usage rate and the memory occupancy rate of the service to be split.

[0068] The weights of the various data in the above-mentioned weighting process are preset according to experience and are all greater than 0.

[0069] In some embodiments, the management platform can also determine service adjustment parameters based on the target allocation relationship. For more details, see Figure 5 and its related descriptions.

[0070] Step 270: Generate a service adjustment instruction based on the service adjustment parameter and send it to the first sub-platform and the second sub-platform.

[0071] The service adjustment instruction refers to an instruction for instructing the first sub-platform to send a sub-service to the second sub-platform service.

[0072] In some embodiments, the management platform converts the service adjustment parameter into a machine instruction and uses the machine instruction as the service adjustment instruction to send it to the first sub-platform and the second sub-platform respectively.

[0073] In some embodiments, after receiving the service adjustment instruction, the first sub-platform processes the data of multiple sub-services based on the service adjustment instruction to obtain multiple data packets corresponding to the sub-services, and sends the multiple data packets to the second sub-platform corresponding to the allocation result of the sub-services respectively. The data processing includes data packaging, etc.

[0074] In some embodiments, the data packet includes the running program code corresponding to the function item of the sub-service, the configuration file, the data item to be processed, etc.

[0075] In some embodiments, the management platform represents the data volume in the data packet by adding a length prefix to the data packet. The second sub-platform receiving the data packet can obtain the data volume of the data packet by parsing the length prefix, and then allocate memory for the parsed data packet.

[0076] In some embodiments, after receiving the service adjustment instruction sent by the management platform and multiple data packets sent by the first sub-platform, the second sub-platform parses the length prefixes of the multiple data packets to determine the data volume corresponding to each data packet. Based on the data volume corresponding to each data packet, the second sub-platform allocates corresponding parsing memory for each data packet and parses each data packet to obtain the parsed data.

[0077] In some embodiments, based on the parsed data, the second sub-platform establishes processes corresponding to multiple sub-services, and based on the service adjustment instruction, determines the processing resources for processing the processes.

[0078] The parsed data includes the running program codes, configuration files, and data items to be processed corresponding to the function items of multiple sub-services, etc.

[0079] In some embodiments, for each sub-service, the second sub-platform can establish a process corresponding to the sub-service, randomly select the processing cores with the required number of cores corresponding to the sub-service from the idle processing cores on its own CPU, and bind the process to the selected one or more processing cores.

[0080] By splitting some services of the data platform with a large computing load and allocating the split services to other data platforms with a lighter load, the computing resources of multiple data platforms can be utilized more reasonably, and the processing efficiency of services can be improved.

[0081] Figure 3 is an exemplary flowchart of splitting the service to be split according to some embodiments of this specification. As Figure 3 shown, the process 300 of splitting the service to be split includes the following steps.

[0082] In some embodiments, for a single service to be split in at least one service to be split, the management platform can execute step 310-step 330 to obtain multiple sub-services corresponding to the service to be split.

[0083] Step 310, obtain multiple candidate splitting schemes.

[0084] Step 310 further includes steps 311-step 314. After obtaining multiple candidate splitting schemes, the management platform can execute steps 311-step 314 for a single candidate splitting scheme among the multiple candidate splitting schemes to determine the service running efficiency and data transmission time corresponding to the candidate splitting scheme.

[0085] A candidate splitting scheme refers to a scheme for splitting the service to be split into multiple sub-services.

[0086] In some embodiments, the management platform obtains candidate splitting schemes in various ways. For example, the management platform sets multiple numbers of sub - services, and each number of sub - services corresponds to a candidate splitting scheme. Based on each number of sub - services, the service to be split is split in the way of splitting the service to be split in step 240, and multiple candidate splitting schemes are obtained.

[0087] In some embodiments, the management platform splits the service to be split multiple times through function item splitting and data item splitting to obtain multiple candidate splitting schemes. Among them, data item splitting includes splitting the data items of the service to be split based on historical call information and the data item characteristics of the service to be split.

[0088] For the description of function items, data items, and splitting the service to be split based on function items, see Figure 2 and its related descriptions.

[0089] Historical call information refers to information related to historical data items called at historical times. In some embodiments, historical call information includes multiple historical data items called at multiple historical time points and the historical data item characteristics of different historical data items. Historical data items refer to the data items corresponding to historical services.

[0090] Data item characteristics refer to data related to the data items themselves. In some embodiments, data item characteristics include one or more of the data item source, data item time, and data item type of the data item. The data item source includes workshops or production lines, etc. The data item time refers to the time when the data item is uploaded. The data item type refers to the type of the function item corresponding to the data item.

[0091] In some embodiments, the management platform can perform processing such as normalization on each piece of data in the data item characteristics to represent each piece of data in the data item characteristics in numerical form, facilitating subsequent calculations.

[0092] In some embodiments, the management platform determines the relevance between any two historical data items based on historical call information, and splits the data items of the service to be split based on multiple relevances and historical call information.

[0093] Relevance refers to data that measures the degree of association between two historical data items. In some embodiments, the management platform calculates the number of times two historical data items are called simultaneously based on historical call information, and uses the number of times called simultaneously as the relevance between the two historical data items.

[0094] In some embodiments, the steps for the management platform to split the data items of the service to be split based on multiple relevances and historical call information include:

[0095] S1: Divide multiple historical data items into multiple first groups based on multiple relevance degrees and historical data item features.

[0096] In some embodiments, the management platform traverses each historical data item, filters out the historical data items whose relevance degree to the currently traversed historical data item is greater than the relevance degree threshold, and divides such historical data items and the currently traversed historical data item into a first group. Continue the traversal process. If the currently traversed historical data item has been divided into a first group, then divide the historical data items whose relevance degree to the currently traversed historical data item is greater than the relevance degree threshold into the same first group. Continue the traversal process until all historical data items have been traversed. Among them, the relevance degree threshold is set according to experience.

[0097] Exemplarily, the management platform traverses historical data items a, b, c, d, and e. First, filter out historical data item b whose relevance degree to a is greater than the relevance degree threshold, and divide a and b into a first group. Then filter out historical data item d whose relevance degree to b is greater than the relevance degree threshold, and add d to the first group where b is located. Then filter out historical data item e whose relevance degree to c is greater than the relevance degree threshold, and divide c and e into a first group.

[0098] S2: Calculate the group feature of each first group. The group feature refers to the data that can characterize the relevant information of the data items in the group. In some embodiments, the management platform calculates the mean of the historical data item features of multiple historical data items in a single first group as the group feature of this first group.

[0099] S3: Based on the historical data item features and multiple group features, divide the historical data items that were not grouped in S1 into multiple first groups.

[0100] In some embodiments, for each historical data item that was not grouped in S1, the management platform constructs a data feature vector based on the historical data item features of the historical data item, constructs multiple group feature vectors based on multiple group features respectively, and calculates the similarity between the data feature vector and the multiple group feature vectors. Filter out the group feature vector with a similarity greater than the similarity threshold and the maximum similarity, and divide the historical data item into the first group corresponding to this group feature vector. The similarity threshold is set according to experience. The vector similarity is negatively correlated with the vector distance. The vector distance includes the Euclidean distance, etc.

[0101] S4: Based on the historical data item features, divide the historical data items that were not grouped in S3 into multiple second groups.

[0102] In some embodiments, the management platform constructs a clustering vector based on the historical data item features of the historical data items in the ungrouped data in S3, performs clustering based on multiple clustering vectors to form multiple clustering clusters, and divides the historical data items corresponding to the clustering vectors in each clustering cluster into a second group. The management platform calculates the grouping feature of each second group by the method of calculating the grouping feature in S2.

[0103] S5: Use the multiple first groups and second groups as target groups, and based on the data item features of the service to be split, divide the data items of the service to be split into different target groups.

[0104] In some embodiments, for each data item of the service to be split, the management platform constructs a vector to be matched based on the data item features of the data item, calculates the similarity between the vector to be matched and multiple grouping feature vectors, filters the grouping feature vector with the largest similarity, and divides the data item into the target group corresponding to the grouping feature vector. Repeat the above process until all the data items of the service to be split are divided.

[0105] After the division is completed, the management platform retains the target groups with data items of the service to be split, deletes the historical data items in the retained target groups, obtains multiple target groups that only include the data items of the service to be split, uses each target group as a sub-service, and forms a candidate split plan with multiple sub-services.

[0106] In some embodiments, when the management platform performs multiple splits on the service to be split through function item splitting and data item splitting based on the number of multiple sub-services, the data item splitting can be performed by the above method, and then multiple candidate split plans can be obtained.

[0107] By using the historical call information and the data item features of the service to be split, the data items of the service to be split are split, so that the data items in each sub-service obtained after the data item splitting are related, which is beneficial to improving the efficiency of the data sub-platform in processing the sub-services after splitting according to the candidate split plan.

[0108] Step 311: Based on the sub-service features corresponding to multiple candidate sub-services in the candidate split plan and the data features of the service to be split, determine the sub-service data volumes corresponding to the multiple candidate sub-services.

[0109] A candidate sub-service refers to a sub-service obtained by splitting the service to be split based on the candidate split plan. For more descriptions about sub-service features, see Figure 2 and its related descriptions.

[0110] The data characteristics of the service to be split refer to the data related to the data information used to characterize the service to be split. In some embodiments, the data characteristics of the service to be split include the amount of running program data, the amount of configuration file data, and the amount of data items to be processed for each function item.

[0111] The amount of sub-service data refers to the amount of data corresponding to the candidate sub-service. In some embodiments, the amount of sub-service data includes the amount of running program data, the amount of configuration file data, and the amount of data items of the function item.

[0112] In some embodiments, for each candidate sub-service, the management platform determines the same function items based on the sub-service characteristics of the candidate sub-service. The amount of running program data and the amount of configuration file data of the same function items are included in the amount of sub-service data. The management platform calculates the ratio of the number of data items of the candidate sub-service to the number of data items to be processed of the same function items, and the product of the calculated ratio and the amount of data items to be processed is included in the amount of sub-service data. The same function items refer to the function items that are the same as those corresponding to the candidate sub-service in the data characteristics of the service to be split.

[0113] Step 312, determine the first transmission amount based on the amount of sub-service data.

[0114] The first transmission amount refers to the total amount of data for transmitting multiple candidate sub-services.

[0115] In some embodiments, the management platform calculates the sum value of the amount of sub-service data based on the amount of sub-service data of multiple candidate sub-services, and takes the product of the sum value and the transmission amount coefficient as the first transmission amount. Among them, the transmission amount coefficient is related to the number of multiple candidate sub-services. For example, the transmission amount coefficient is , is the number of multiple candidate sub-services.

[0116] In some embodiments, the management platform obtains multiple candidate transmission schemes, determines the second transmission amount corresponding to the multiple candidate transmission schemes based on the amount of sub-service data, and obtains the first transmission amount based on the multiple second transmission amounts.

[0117] The candidate transmission scheme refers to the scheme for transmitting multiple candidate sub-services to the second sub-platform. In some embodiments, the candidate transmission scheme includes retaining one local service and transmitting multiple candidate sub-services to the second sub-platform. The local service refers to the service processed on the first sub-platform.

[0118] In some embodiments, for each candidate sub-service, the management platform takes the current candidate sub-service as the local service, and takes the candidate sub-services other than the current candidate sub-service as the candidate sub-services to be transmitted to the second sub-platform, thereby obtaining a candidate transmission scheme. The management platform takes each candidate sub-service as the local service respectively, and obtains multiple candidate transmission schemes through the above method.

[0119] The second transmission volume refers to the total amount of data of multiple candidate sub-services transmitted according to the candidate transmission scheme.

[0120] In some embodiments, the management platform calculates the sum of the sub-service data volumes of the candidate sub-services transmitted to the second sub-platform in the candidate transmission scheme based on the sub-service data volume, and uses the sum as the second transmission volume corresponding to the candidate transmission scheme.

[0121] In some embodiments, the management platform performs a weighted sum of multiple second transmission volumes and uses the sum as the first transmission volume. Among them, the weights of different candidate transmission schemes are negatively correlated with the first sum of the local services in the candidate transmission scheme. The larger the first sum, the more processing resources are required for the local service, and the more this local service tends to be transferred to the second sub-platform, so the weight is smaller. For the description of the first sum, see Figure 2 and its related descriptions.

[0122] Through multiple candidate transmission schemes, it is possible to consider the processing resource requirements of different candidate sub-services when determining the first transmission volume, and then obtain the first transmission volume by synthesizing multiple candidate transmission schemes, so that the determined first transmission volume is more in line with the actual data transmission volume under the candidate splitting scheme.

[0123] Step 313: Determine the data transmission time corresponding to the candidate splitting scheme based on the first transmission volume and the available communication bandwidth.

[0124] The data transmission time refers to the time required to transmit multiple candidate sub-services under the candidate splitting scheme to the second sub-platform.

[0125] The available communication bandwidth refers to the communication link bandwidth that can be used when the first sub-platform transmits multiple candidate sub-services. In some embodiments, the management platform calculates the average value of the communication link bandwidths between the first sub-platform and multiple second sub-platforms and uses the average value as the available communication bandwidth.

[0126] In some embodiments, the data transmission time is positively correlated with the first transmission volume and negatively correlated with the available communication bandwidth. Exemplarily, the management platform determines the data transmission time based on the first transmission volume and the available communication bandwidth through the following formula (2):

[0127] (2)

[0128] where F is the data transmission time, m is the first transmission volume, is the number of multiple candidate sub-services, and b is the available communication bandwidth.

[0129] Step 314: Determine the service operation efficiency corresponding to the candidate splitting scheme based on the service characteristics of the service to be split, the sub-service characteristics and operation characteristics corresponding to multiple candidate sub-services.

[0130] For the description of service characteristics and operation characteristics, see Figure 2 and its related descriptions.

[0131] Service operation efficiency refers to the efficiency of processing the service to be split after splitting the service to be split based on the candidate splitting scheme. In some embodiments, the service operation efficiency is represented by the sum of the operation efficiencies of multiple candidate sub-services. The operation efficiency of a candidate sub-service is represented by the number of executions of the candidate sub-service per unit time. Among them, when a data item of the candidate sub-service is processed, the execution count is incremented by 1.

[0132] In some embodiments, the management platform determines the processing resource requirements for each candidate sub-service based on the service characteristics of the service to be split and the sub-service characteristics corresponding to multiple candidate sub-services. For the description of determining the processing resource requirements, see Figure 2 and its related descriptions.

[0133] In some embodiments, the management platform determines the service operation efficiency in multiple ways based on the processing resource requirements and operation characteristics corresponding to multiple candidate sub-services. For example, for each candidate sub-service, the management platform queries the reference operation efficiency corresponding to the average operation characteristics and the processing resource requirements in the preset table based on the average operation characteristics and the processing resource requirements of the candidate sub-service, and uses the reference operation efficiency as the operation efficiency of the candidate sub-service. The management platform calculates the sum of the operation efficiencies of multiple candidate sub-services as the service operation efficiency. The average operation characteristics include the mean of the total CPU usage rates of multiple data sub-platforms and the mean of the total memory occupancy rates.

[0134] In some embodiments, the preset table is constructed based on experimental data. The experimental process includes: under a certain average operation characteristic, running the sub-service with determined processing resource requirements on multiple data sub-platforms to obtain multiple operation efficiencies, and taking the mean of the multiple operation efficiencies as the reference operation efficiency corresponding to the average operation characteristics and the processing resource requirements.

[0135] In some embodiments, the management platform can determine the service operation efficiency through an efficiency prediction model based on the processing resource requirements and operation characteristics corresponding to multiple candidate sub-services. For more descriptions, see Figure 4 and its related descriptions.

[0136] Step 320: Determine the target splitting scheme based on the multiple data transmission times and multiple service operation efficiencies corresponding to multiple candidate splitting schemes.

[0137] The target splitting scheme refers to the candidate splitting scheme determined to be used.

[0138] In some embodiments, the management platform selects the candidate splitting scheme with the highest business operation efficiency among multiple candidate splitting schemes where the data transmission time is less than the transmission time threshold as the target splitting scheme. The transmission time threshold is preset based on experience.

[0139] Step 330: Based on the target splitting scheme, split the service to be split to obtain multiple sub-services corresponding to the service to be split.

[0140] The management platform splits the service to be split according to the steps of splitting the service to be split into multiple candidate sub-services in the target splitting scheme to obtain multiple sub-services corresponding to the service to be split.

[0141] By obtaining multiple candidate splitting schemes and evaluating the data transmission time and business operation efficiency of the sub-services after splitting based on the candidate splitting schemes, a better splitting scheme can be found, which is beneficial to making a more appropriate split of the service to be split, avoiding excessive computing pressure on the second sub-platform due to overly large sub-services, and balancing the computing pressure of each data sub-platform.

[0142] Figure 4 It is an exemplary schematic diagram of an efficiency prediction model shown in some embodiments of this specification.

[0143] In some embodiments, the management platform can also determine the business operation efficiency 440 through the efficiency prediction model 430 based on the processing resource requirements 410 and operation characteristics 420 corresponding to multiple candidate sub-services.

[0144] For more descriptions of candidate sub-services, processing resource requirements, operation characteristics, and business operation efficiency, see Figure 2 and Figure 3 its related descriptions.

[0145] The efficiency prediction model refers to a model used to determine the business operation efficiency. In some embodiments, the efficiency prediction model is a machine learning model. For example, any one or combination of a Recurrent Neural Network (RNN) model, a Deep Neural Networks (DNN) model, or other custom model structures, etc.

[0146] In some embodiments, the management platform trains the efficiency prediction model based on a training data set through methods such as gradient descent. The training data set includes multiple groups of training samples and labels corresponding to different training samples. Each group of training samples includes sample operation characteristics and sample processing resource requirements corresponding to multiple sample sub-services of the sample service to be split, and the label of each group of training samples includes the actual business operation efficiency of the service to be split for that group of samples.

[0147] In some embodiments, the training samples and labels are obtained based on historical data. For example, the management platform may use historical business to be split as the sample business to be split, randomly allocate the multiple sample sub - businesses after splitting the sample business to be split to multiple historical second - level platforms multiple times, and calculate the actual business operation efficiency of the sample business to be split after each random allocation. The management platform takes the mean of the business operation efficiencies corresponding to the multiple random allocations as the label. For the determination of business operation efficiency, refer to Figure 3 and its related descriptions.

[0148] In some embodiments, the efficiency prediction model is trained as follows: input multiple labeled training samples into the initial efficiency prediction model, construct a loss function based on the labels and the prediction results of the initial efficiency prediction model, iteratively update the initial efficiency prediction model based on the loss function, and when the loss function meets the preset training conditions, the efficiency prediction model training is completed. Among them, the preset training conditions include the convergence of the loss function, the number of iterations reaching the set value, etc.

[0149] By processing the operation characteristics and the processing resource requirements corresponding to multiple candidate sub - businesses through the efficiency prediction model, the self - learning ability of the machine learning model can be utilized to find patterns from a large amount of data, obtain the correlation between the operation characteristics and the processing resource requirements corresponding to multiple candidate sub - businesses and the business operation efficiency, and improve the accuracy and efficiency of determining the business operation efficiency.

[0150] Figure 5 is an exemplary flowchart for determining business adjustment parameters according to some embodiments of this specification. As Figure 5 shown, the process 500 for determining business adjustment parameters includes the following steps.

[0151] Step 510, based on the processing resource requirements and operation characteristics, determine multiple candidate allocation relationships and determine the estimated operation information corresponding to the multiple candidate allocation relationships.

[0152] The candidate allocation relationship refers to the allocation relationship between multiple sub - businesses and sub - platforms. In some embodiments, the candidate allocation relationship includes the allocation results of multiple sub - businesses. For the descriptions of the allocation results, processing resource requirements, and operation characteristics, refer to Figure 2 and its related descriptions.

[0153] In some embodiments, the management platform determines multiple candidate allocation relationships based on the processing resource requirements and operation characteristics. For example, the management platform randomly allocates multiple sub - businesses to multiple second - level platforms multiple times to determine multiple candidate allocation relationships. Among them, when randomly allocating sub - businesses to the second - level platforms, the sum of the total CPU usage rates of the second - level platforms does not exceed the CPU allocation threshold. For the description of the CPU allocation threshold, refer to Figure 2 and its related descriptions.

[0154] The estimated operation information refers to the information related to the operation of multiple second sub - platforms in the future after being allocated based on the candidate allocation relationship. In some embodiments, the estimated operation information includes the estimated total CPU usage rate and the estimated total memory occupancy rate of each second sub - platform.

[0155] In some embodiments, the management platform determines the estimated operation information based on the processing resource requirements of each sub - service determined in step 250 and the operation characteristics of multiple second sub - platforms in the candidate allocation relationship. For example, the management platform calculates the sum of the CPU usage rates of multiple sub - services allocated to the second sub - platform, and calculates the sum value of the sum of the CPU usage rates and the total CPU usage rate in the operation characteristics, and takes the sum value as the estimated total CPU usage rate of the second sub - platform. The calculation method of the estimated total memory occupancy rate is similar to that of the estimated total CPU usage rate.

[0156] Step 520: Determine the target allocation relationship based on the estimated operation information.

[0157] The target allocation relationship refers to the candidate allocation relationship determined to be used.

[0158] In some embodiments, the management platform calculates the third sum value corresponding to the candidate allocation relationship based on the estimated operation information, and selects the candidate allocation relationship with the smallest third sum value as the target allocation relationship. Among them, the third sum value is the sum of the variances of multiple estimated total CPU usage rates and the variances of multiple estimated total memory occupancy rates.

[0159] Step 530: Determine the service adjustment parameters based on the target allocation relationship.

[0160] In some embodiments, the management platform determines the allocation result of each sub - service in the target allocation relationship as the allocation result of each sub - service in the service adjustment parameters.

[0161] By evaluating the service processing efficiency of multiple allocation relationships, it is beneficial to find the optimal allocation relationship and effectively improve the subsequent service processing efficiency.

[0162] It should be noted that the descriptions of the process 200 of the computing scheduling method based on the industrial Internet of Things data center, the process 300 of splitting the service to be split, and the process 500 of determining the service adjustment parameters above are only for illustration and explanation, and do not limit the scope of application of the present invention. For those skilled in the art, under the guidance of the present invention, various corrections and changes can be made to the process 200 of the computing scheduling method based on the industrial Internet of Things data center, the process 300 of splitting the service to be split, and the process 500 of determining the service adjustment parameters. However, these corrections and changes are still within the scope of the present invention.

[0163] Some embodiments of the present invention further provide a computer-readable storage medium storing computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the computing scheduling method based on the industrial Internet of Things data center described in any one of the above embodiments.

[0164] In addition, certain features, structures, or characteristics in one or more embodiments of the present invention may be appropriately combined.

[0165] In some embodiments, the numerical parameters used in the specification and claims are all approximate values, which may change according to the required characteristics of individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining general digits. Although the numerical ranges and parameters used to confirm the breadth of the scope in some embodiments of the present invention are approximate values, in specific embodiments, such numerical settings are as precise as possible within the feasible range.

[0166] If there are inconsistencies or conflicts between the descriptions, definitions, and / or uses of terms in the materials cited in the present invention and the content described in the present invention, the descriptions, definitions, and / or uses of terms in the present invention shall prevail.

Claims

1. A computing and scheduling system based on an industrial Internet of Things data center, characterized in that: The system includes a management platform, a data center and a plurality of data sub-platforms connected in communication, wherein the management platform is configured as follows: Obtaining operation characteristics of the multiple data sub-platforms; Based on the operation characteristics, determining a first sub-platform and a second sub-platform among the plurality of data sub-platforms; Determine at least one business to be split based on the business characteristics corresponding to the multiple businesses of the first sub-platform and the operation characteristics; Splitting the at least one business to be split into a plurality of sub-businesses based on business characteristics of the at least one business to be split; Determining processing resource requirements corresponding to the multiple sub-businesses based on the business characteristics of the at least one business to be split, the operation characteristics, and sub-business characteristics corresponding to the multiple sub-businesses; Determining a service adjustment parameter based on the processing resource requirement and the operation characteristics; Based on the service adjustment parameters, generate a service adjustment instruction and send it to the first sub-platform and the second sub-platform; Using the first sub-platform to process the multiple sub-services based on the service adjustment instruction to obtain multiple data packets; sending the multiple data packets to the second sub-platform; Using the second sub-platform: receiving the plurality of data packets; Parsing the multiple data packets to determine the data volumes corresponding to the multiple data packets; Based on the data volume, corresponding parsing memory is allocated to the multiple data packets, and the multiple data packets are parsed to obtain parsed data; Based on the parsed data, establishing processes corresponding to the multiple sub-businesses; Based on the service adjustment instruction, a processing resource for processing the process is determined.

2. The system according to claim 1, characterized in that The management platform is further configured to: For a single business to be split among the at least one business to be split: Obtain multiple candidate splitting solutions; Determine a target splitting scheme based on multiple data transmission times and multiple business operation efficiencies corresponding to the multiple candidate splitting schemes; Based on the target splitting plan, the business to be split is split to obtain multiple sub-businesses corresponding to the business to be split.

3. The system according to claim 2, characterized in that The management platform is further configured to: The business to be split is split multiple times by splitting functional items and data items to obtain the multiple candidate splitting solutions.

4. The system according to claim 1, characterized in that The management platform is further configured to: Based on the processing resource requirements and the operation characteristics, determining a plurality of candidate allocation relationships, and determining estimated operation information corresponding to the plurality of candidate allocation relationships; Determining a target allocation relationship based on the estimated operation information; Based on the target allocation relationship, the service adjustment parameter is determined.

5. A computing scheduling method based on an industrial Internet of Things data center, characterized in that: The method is executed by a management platform, and includes: Obtain the operating characteristics of multiple data sub-platforms; Based on the operation characteristics, determining a first sub-platform and a second sub-platform among the plurality of data sub-platforms; Determine at least one business to be split based on the business characteristics corresponding to the multiple businesses of the first sub-platform and the operation characteristics; Splitting the at least one business to be split into a plurality of sub-businesses based on business characteristics of the at least one business to be split; Determining processing resource requirements corresponding to the multiple sub-businesses based on the business characteristics of the at least one business to be split, the operation characteristics, and sub-business characteristics corresponding to the multiple sub-businesses; Determining a service adjustment parameter based on the processing resource requirement and the operation characteristics; Based on the service adjustment parameters, generate a service adjustment instruction and send it to the first sub-platform and the second sub-platform; Using the first sub-platform to process the multiple sub-services based on the service adjustment instruction to obtain multiple data packets; sending the multiple data packets to the second sub-platform; Using the second sub-platform: receiving the plurality of data packets; Parsing the multiple data packets to determine the data volumes corresponding to the multiple data packets; Based on the data volume, corresponding parsing memory is allocated to the multiple data packets, and the multiple data packets are parsed to obtain parsed data; Based on the parsed data, establishing processes corresponding to the multiple sub-businesses; Based on the service adjustment instruction, a processing resource for processing the process is determined.

6. The method according to claim 5, characterized in that The step of splitting the at least one to-be-split service into a plurality of sub-services based on the service characteristics of the at least one to-be-split service comprises: For a single business to be split among the at least one business to be split: Obtain multiple candidate splitting solutions; Determine a target splitting scheme based on multiple data transmission times and multiple business operation efficiencies corresponding to the multiple candidate splitting schemes; Based on the target splitting plan, the business to be split is split to obtain multiple sub-businesses corresponding to the business to be split.

7. The method according to claim 6, characterized in that The obtaining of multiple candidate splitting solutions comprises: The business to be split is split multiple times by splitting functional items and data items to obtain the multiple candidate splitting solutions.

8. The method according to claim 5, characterized in that The determining of the service adjustment parameter based on the processing resource requirement and the operation characteristic includes: Based on the processing resource requirements and the operation characteristics, determining a plurality of candidate allocation relationships, and determining estimated operation information corresponding to the plurality of candidate allocation relationships; Determining a target allocation relationship based on the estimated operation information; Based on the target allocation relationship, the service adjustment parameter is determined.

9. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the method as claimed in claim 5.

Images