Calculation scheduling system and method based on industrial Internet of Things data center and medium

Abstract

The invention provides a computing scheduling system and method based on an industrial Internet of Things data center and a medium, and relates to the field of computing scheduling, and the system comprises a management platform, a data center and a plurality of data sub-platforms which are in communication connection. The management platform is configured to obtain operation characteristics; determining a first sub-platform and a second sub-platform based on the operation characteristics; determining at least one to-be-split service based on the service features and the operation features corresponding to the plurality of services of the first sub-platform, splitting the at least one to-be-split service into a plurality of sub-services, and determining processing resource requirements corresponding to the plurality of sub-services; based on the processing resource requirements and the operation characteristics, service adjustment parameters are determined. According to the method and the device, part of services of the data sub-platform with relatively large calculation load can be split, and the split services are distributed to other data sub-platforms with relatively light loads, so that calculation resources of a plurality of data sub-platforms can be utilized more reasonably, and the service processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computing scheduling, and in particular to a computing scheduling system, method and medium based on an industrial Internet of Things data center. Background Art

[0002] Each data sub-platform of the Industrial Internet of Things usually relies on its own computing power to process business. However, different businesses have different data volumes, and businesses with different data volumes occupy different computing resources of data sub-platforms. Therefore, during the operation of the Industrial Internet of Things, the computing resources of some data sub-platforms may not meet the needs of processing business, while the computing resources of some data sub-platforms are idle.

[0003] In view of this, it is necessary to provide a computing scheduling system, method and medium based on the industrial Internet of Things data center, by splitting and scheduling the business of different data sub-platforms, making full use of the computing resources of each data sub-platform and balancing the computing pressure of each data sub-platform. Summary of the invention

[0004] The invention content includes a computing scheduling system based on an industrial Internet of Things data center, the system includes a management platform, a data center and multiple data sub-platforms connected in communication, the management platform is configured to: obtain the operating characteristics of the multiple data sub-platforms; based on the operating characteristics, determine the first sub-platform and the second sub-platform among the multiple data sub-platforms; based on the business characteristics corresponding to multiple businesses of the first sub-platform and the operating characteristics, determine at least one business to be split; based on the business characteristics of the at least one business to be split, split the at least one business to be split into multiple sub-businesses; based on the business characteristics of the at least one business to be split, the operating characteristics and the sub-business characteristics corresponding to the multiple sub-businesses, determine the processing resource requirements corresponding to the multiple sub-businesses; based on the processing resource requirements and the The operation characteristics are determined to determine business adjustment parameters; based on the business adjustment parameters, a business adjustment instruction is generated and sent to the first sub-platform and the second sub-platform; the first sub-platform is configured to: based on the business adjustment instruction, perform data processing on the multiple sub-businesses to obtain multiple data packets; send the multiple data packets to the second sub-platform; the second sub-platform is configured to: receive the multiple data packets; parse the multiple data packets to determine the data volume corresponding to the multiple data packets; based on the data volume, allocate corresponding parsing memory to the multiple data packets, and parse the multiple data packets to obtain parsed data; based on the parsed data, establish processes corresponding to the multiple sub-businesses; and based on the business adjustment instruction, determine the processing resources used to process the processes.

[0005] The invention content includes a computing scheduling method based on an industrial Internet of Things data center, which is executed by a management platform, and the method includes: obtaining the operating characteristics of multiple data sub-platforms; based on the operating characteristics, determining the first sub-platform and the second sub-platform among the multiple data sub-platforms; based on the business characteristics corresponding to multiple businesses of the first sub-platform and the operating characteristics, determining at least one business to be split; based on the business characteristics of the at least one business to be split, splitting the at least one business to be split into multiple sub-businesses; based on the business characteristics of the at least one business to be split, the operating characteristics and the sub-business characteristics corresponding to the multiple sub-businesses, determining the processing resource requirements corresponding to the multiple sub-businesses; based on the processing resource requirements and the operating characteristics, determining business adjustment parameters; based on the business adjustment parameters, generating business adjustment instructions and sending them to the first sub-platform and the second sub-platform.

[0006] The invention content includes a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the above-mentioned computing and scheduling method based on the industrial Internet of Things data center.

[0007] The beneficial effects of the present invention include but are not limited to: (1) Through the computing scheduling system based on the industrial Internet of Things data center, the business of different data sub-platforms can be split and the business adjustment between different data sub-platforms can be realized. (2) It is possible to split part of the business of the data sub-platform with a large computing load and allocate the split business to other data sub-platforms with a lighter load, so that the computing resources of multiple data sub-platforms can be used more reasonably to improve the processing efficiency of the business. (3) Through the machine learning model, the operating characteristics and the processing resource requirements corresponding to multiple candidate sub-businesses are processed, and the rules are found from a large amount of data. The correlation between the operating characteristics and the processing resource requirements corresponding to multiple candidate sub-businesses and the business operation efficiency is obtained, thereby improving the accuracy and efficiency of determining the business operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present invention will be further described in the form of exemplary embodiments, which will be described in detail by way of the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein: Figure 1 It is a schematic diagram of the platform structure of a computing and scheduling system based on an industrial Internet of Things data center according to some embodiments of this specification; Figure 2 is an exemplary flow chart of a computing scheduling method based on an industrial Internet of Things data center according to some embodiments of this specification; Figure 3is an exemplary flow chart of splitting a business to be split according to some embodiments of this specification; Figure 4 is an exemplary schematic diagram of an efficiency estimation model according to some embodiments of this specification; Figure 5 This is an exemplary flow chart of determining service adjustment parameters according to some embodiments of this specification. DETAILED DESCRIPTION

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. The drawings do not represent all implementation methods.

[0010] When the operations performed in the embodiments of the present invention are described in steps, unless otherwise specified, the order of the steps is interchangeable, steps may be omitted, and other steps may be included in the operation process.

[0011] Figure 1 It is a schematic diagram of the platform structure of a computing and scheduling system based on an industrial Internet of Things data center as shown in some embodiments of this specification.

[0012] like Figure 1 As shown, the computing and scheduling system 100 based on the industrial Internet of Things data center includes a management platform 110, a data center 120 and multiple data sub-platforms (such as data sub-platform 131, data sub-platform 132, data sub-platform 133, data sub-platform 134, etc.) that are communicatively connected.

[0013] The management platform refers to a platform for processing information and / or instructions related to the computing scheduling system 100 based on the industrial Internet of Things data center. In some embodiments, the management platform is configured on a server or a processor.

[0014] In some embodiments, the processor includes a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction processor (ASIP), etc., or any combination thereof.

[0015] In some embodiments, the management platform includes a data center 120 .

[0016] The data center is configured to store and manage information and / or instructions related to the computing scheduling system 100 based on the industrial Internet of Things data center. The data center is configured as a storage device, etc.

[0017] Data sub-platforms refer to platforms used to complete business. Different data sub-platforms are used to complete multiple different businesses. Business refers to processing such as calculation or analysis of data. For example, pre-processing data or analyzing the product quality of each product in product data. Pre-processing includes cleaning, normalization, etc.

[0018] In some embodiments, the data sub-platform obtains various data required for executing services from the data center, such as data items to be processed.

[0019] Multiple data sub-platforms can be communicatively connected. In some embodiments, the data sub-platforms are configured on a server or a processor.

[0020] For detailed description of the above, please refer to Figures 2 to 5 Related description.

[0021] Through the computing and scheduling system 100 based on the industrial Internet of Things data center, the business of different data sub-platforms can be split and the business adjustment between different data sub-platforms can be achieved.

[0022] Figure 2 2 is an exemplary flow chart of a computing scheduling method based on an industrial Internet of Things data center according to some embodiments of this specification. In some embodiments, the process 200 of the computing scheduling method based on the industrial Internet of Things data center is executed by a management platform. Figure 2 As shown, the process 200 of the computing scheduling method based on the industrial Internet of Things data center includes the following steps.

[0023] For a description of the computing and scheduling system based on the industrial IoT data center and various platforms, see Figure 1 The corresponding description.

[0024] Step 210, obtaining operation characteristics of multiple data sub-platforms.

[0025] Operational characteristics refer to data related to the operation of the data sub-platform. Each data sub-platform corresponds to an operational characteristic.

[0026] In some embodiments, the operating characteristics include the total CPU usage rate and total memory occupancy rate of the data sub-platform, the CPU usage rate and memory occupancy rate corresponding to each business in the data sub-platform, and the like.

[0027] In some embodiments, the management platform obtains the process running table of the data sub-platform through the data sub-platform, and then obtains the running characteristics of the data sub-platform.

[0028] The process running table is a data table that is automatically generated when the data sub-platform processes business, showing the processing resources used by each business. Processing resources include CPU resources and memory resources.

[0029] Step 220: Determine a first sub-platform and a second sub-platform among a plurality of data sub-platforms based on the operation characteristics.

[0030] The first sub-platform refers to a data sub-platform that needs to dispatch services to other data sub-platforms.

[0031] The second sub-platform refers to data sub-platforms other than the first sub-platform.

[0032] In some embodiments, the management platform may use a data sub-platform that meets the preset screening conditions among multiple data sub-platforms as the first sub-platform, and use other data sub-platforms except the first sub-platform as the second sub-platform. The preset screening conditions include that the total CPU usage and / or total memory occupancy of the data sub-platform is greater than the preset screening threshold. The preset screening threshold is pre-set based on historical experience. The preset screening conditions may also include that the total CPU usage of the data sub-platform is the highest.

[0033] Step 230: Determine at least one business to be split based on the business characteristics and operation characteristics corresponding to the multiple businesses of the first sub-platform.

[0034] A service feature refers to data used to reflect service-related information. One service corresponds to one service feature. In some embodiments, the service feature includes the function item corresponding to the service and the number of data items corresponding to the function item.

[0035] Function items are used to characterize the functions of a business. A business includes one or more function items. Function items include preprocessing each data or analyzing the product quality of each product in the product data. Preprocessing includes cleaning, normalization, etc.

[0036] A data item refers to the data corresponding to the execution function item. For example, if the function item is to preprocess each data, then the data item corresponding to the function item is each data that needs to be preprocessed. One function item corresponds to multiple data items.

[0037] The business to be split refers to the business that needs to be split in the first sub-platform.

[0038] In some embodiments, the management platform calculates the splitting degree of each business based on the business characteristics and operation characteristics corresponding to the multiple businesses of the first sub-platform, sorts the multiple splitting degrees, and determines the business that meets the splitting degree condition as the business to be split. The splitting degree condition includes being ranked in the top N in the splitting degree sorting. N is preset based on historical experience.

[0039] The splitting degree refers to data used to characterize the degree to which a business needs to be split. In some embodiments, the management platform calculates the business characteristics and operation characteristics corresponding to a single business to obtain the splitting degree of the single business. Exemplarily, the management platform calculates the splitting degree of the business using a preset formula. Exemplarily, the preset formula is shown in formula (1): (1) in, is the splitting degree, is the number of functional items, is the number of data items, The CPU usage of the service. Memory usage of the service. ~ is the preset coefficient. ~ It is set based on experience and is greater than 0.

[0040] Step 240: split the at least one business to be split into a plurality of sub-businesses based on the business characteristics of the at least one business to be split.

[0041] A sub-business refers to a business obtained by splitting the business to be split.

[0042] In some embodiments, the number of sub-businesses obtained after splitting is determined in a variety of ways. For example, the number of sub-businesses is equal to or less than the number of data sub-platforms. For another example, the number of sub-businesses is equal to the number of data sub-platforms that meet preset conditions. The preset conditions include that the total CPU usage and the total memory occupancy are respectively less than the corresponding occupancy thresholds. The occupancy threshold is set based on experience.

[0043] In some embodiments, the management platform splits at least one business to be split into multiple sub-businesses in a variety of ways based on the business characteristics of at least one business to be split. For example, the management platform splits each business to be split according to a preset rule to obtain multiple sub-businesses. The management platform identifies each sub-business to distinguish different sub-businesses.

[0044] In some embodiments, the preset rules include splitting the business to be split based on different functional items first, and then splitting the sub-businesses split based on different functional items based on data items to obtain multiple sub-businesses that meet the number of sub-businesses. Wherein, splitting based on data items includes splitting a sub-business whose number of data items is greater than the preset split number into multiple sub-businesses that are not greater than the preset split number based on the number of data items. The preset split number is pre-set based on the number of sub-businesses, for example, the more sub-businesses there are, the fewer the preset split number is.

[0045] Exemplarily, the number of sub-businesses is 3, and the preset number of splits is 10,000. The business to be split includes two functional items, one of which corresponds to 20,000 data items. The business to be split is split into two sub-businesses corresponding to the two functional items, and the sub-business with 20,000 data items is split into two sub-businesses with 10,000 data items, resulting in 3 sub-businesses.

[0046] In some embodiments, the management platform can also split the business to be split based on the target splitting plan to obtain the sub-businesses corresponding to the business to be split. For more information, see Figure 3 and its related description.

[0047] Step 250: Determine processing resource requirements corresponding to the multiple sub-businesses based on the business characteristics and operation characteristics of at least one business to be split and the sub-business characteristics corresponding to the multiple sub-businesses.

[0048] The sub-service feature refers to the service feature corresponding to the sub-service. In some embodiments, the sub-service feature includes the function items and the number of data items corresponding to the sub-service.

[0049] The processing resource requirement refers to the computing resources required to process the sub-service. In some embodiments, the processing resource requirement includes the CPU usage and memory occupancy corresponding to the sub-service.

[0050] In some embodiments, the management platform determines the processing resource requirements corresponding to the sub-business based on the business characteristics, operation characteristics, and sub-business characteristics corresponding to the sub-business of at least one business to be split. For example, the management platform calculates the ratio of the number of data items of the sub-business to the number of data items of the function items corresponding to the sub-business in the business to be split, and multiplies the obtained ratio by the CPU usage and memory occupancy of the business to be split in the operation characteristics to obtain the CPU usage and memory occupancy corresponding to the sub-business.

[0051] Step 260, determining service adjustment parameters based on processing resource requirements and operation characteristics.

[0052] The service adjustment parameter refers to a parameter used to allocate multiple sub-services of at least one service to be split to multiple second sub-platforms. In some embodiments, the service adjustment parameter includes sub-service characteristics and allocation results corresponding to each sub-service.

[0053] The allocation result refers to the second sub-platform to which the sub-business needs to be allocated.

[0054] In some embodiments, the management platform determines the service adjustment parameters based on the processing resource requirements and the operation characteristics. For example, the management platform determines the allocation result of each sub-service in descending order based on the position of the sub-service in the first sorting result.

[0055] The first sorting result refers to the sorting result of sorting the sub-businesses from large to small according to the processing resource requirements. In some embodiments, for each sub-business, the management platform performs a weighted summation of the CPU usage and memory occupancy of the sub-business to obtain a first sum value. The management platform sorts the first sum values ​​corresponding to each sub-business from large to small to obtain the first sorting result.

[0056] In some embodiments, the management platform determines the allocation results of each sub-business in the second sorting results of multiple data sub-platforms from high to low in order based on the first sorting result. For example, the management platform allocates the sub-business ranked first in the first sorting result to the data sub-platform ranked first in the second sorting result. The sub-business ranked second in the first sorting result is allocated to the data sub-platform ranked first in the second sorting result. If the data sub-platform ranked first after allocation meets the preset allocation conditions, the allocation result of the sub-business ranked second is determined as the data sub-platform ranked first. If the data sub-platform ranked first after allocation does not meet the preset allocation conditions, the sub-business ranked second is allocated to the data sub-platform ranked second in the second sorting result, and it is further determined whether the data sub-platform ranked second after allocation meets the preset allocation conditions. If the preset allocation conditions are met, the allocation result of the sub-business ranked second is determined as the data sub-platform ranked second.

[0057] After determining the allocation result of the second-ranked sub-business, the management platform allocates the third-ranked sub-business in the first ranking result to the first-ranked data sub-platform in the second ranking result, and determines the allocation result of the third-ranked sub-business through the above steps. Repeat the above process until the allocation results of all ranked sub-businesses are determined.

[0058] In some embodiments, the preset allocation conditions include that the sum of the CPU usage of the allocated sub-business and the total CPU usage of the data sub-platform does not exceed the CPU allocation threshold of the data sub-platform, and the sum of the memory occupancy of the sub-business and the total memory occupancy of the data sub-platform does not exceed the memory allocation threshold of the data sub-platform. Exemplarily, the preset allocation conditions include that the sum of the CPU usage of the second-ranked sub-business and the total CPU usage of the first-ranked data sub-platform does not exceed the CPU allocation threshold of the data sub-platform, and the sum of the memory occupancy of the second-ranked sub-business and the total memory occupancy of the first-ranked data sub-platform does not exceed the memory allocation threshold of the data sub-platform, etc. At this time, the total CPU usage of the first-ranked data sub-platform refers to the total CPU usage of the sub-business that has been allocated the first-ranked. The same applies to the total memory occupancy. The CPU allocation threshold and the memory allocation threshold are set based on experience.

[0059] The second sorting result refers to the sorting result of sorting the data sub-platforms from small to large according to the operating characteristics. In some embodiments, for each data sub-platform, the management platform performs a weighted sum based on the total CPU usage and total memory occupancy of the data sub-platform to obtain a second sum value. The management platform sorts the second sum values ​​corresponding to each data sub-platform from small to large to obtain a second sorting result.

[0060] The operation characteristics of the first sub-platform participating in the sorting are the remaining total CPU usage and total memory usage after deducting the CPU usage and memory usage of the business to be split.

[0061] The weights of each data in the above weighting process are preset based on experience and are all greater than 0.

[0062] In some embodiments, the management platform can also determine the service adjustment parameters based on the target allocation relationship. For more information, see Figure 5 and its related description.

[0063] Step 270: Generate a service adjustment instruction based on the service adjustment parameter and send it to the first sub-platform and the second sub-platform.

[0064] The service adjustment instruction is an instruction to instruct the first sub-platform to send the sub-service to the service of the second sub-platform.

[0065] In some embodiments, the management platform converts the service adjustment parameters into machine instructions, and sends the machine instructions as service adjustment instructions to the first sub-platform and the second sub-platform respectively.

[0066] In some embodiments, after receiving the service adjustment instruction, the first sub-platform performs data processing on multiple sub-services based on the service adjustment instruction to obtain multiple data packets corresponding to the sub-services, and sends the multiple data packets to the second sub-platform corresponding to the allocation results of the sub-services. Data processing includes data packaging, etc.

[0067] In some embodiments, the data packet includes running program codes, configuration files, and data items to be processed corresponding to the functional items of the sub-service.

[0068] In some embodiments, the management platform indicates the amount of data in the data packet by adding a length prefix to the data packet. The second sub-platform receiving the data packet can obtain the amount of data in the data packet by parsing the length prefix, and then allocate memory for the parsed data packet.

[0069] In some embodiments, after receiving the service adjustment instruction sent by the management platform and the multiple data packets sent by the first sub-platform, the second sub-platform parses the length prefixes of the multiple data packets to determine the data volume corresponding to each data packet. Based on the data volume corresponding to each data packet, the second sub-platform allocates a corresponding parsing memory to each data packet, and parses each data packet to obtain parsed data.

[0070] In some embodiments, the second sub-platform establishes processes corresponding to multiple sub-businesses based on the parsed data, and determines processing resources for processing the processes based on the business adjustment instructions.

[0071] The parsed data includes the running program codes, configuration files and data items to be processed corresponding to the functional items of multiple sub-businesses.

[0072] In some embodiments, for each sub-business, the second sub-platform can establish a process corresponding to the sub-business, and randomly select the required number of processing cores corresponding to the sub-business from the idle processing cores on its own CPU, and bind the process to the selected one or more processing cores.

[0073] By splitting some of the businesses on data sub-platforms with heavy computing loads and allocating the split businesses to other data sub-platforms with lighter loads, the computing resources of multiple data sub-platforms can be used more reasonably, thereby improving business processing efficiency.

[0074] Figure 3 FIG. 1 is an exemplary flow chart of splitting a business to be split according to some embodiments of this specification. Figure 3 As shown, the process 300 of splitting the business to be split includes the following steps.

[0075] In some embodiments, for a single business to be split among at least one business to be split, the management platform may execute steps 310 to 330 to obtain multiple sub-businesses corresponding to the business to be split.

[0076] Step 310: Obtain multiple candidate splitting solutions.

[0077] Step 310 also includes steps 311 to 314. After obtaining multiple candidate splitting schemes, the management platform can execute steps 311 to 314 for a single candidate splitting scheme among the multiple candidate splitting schemes to determine the business operation efficiency and data transmission time corresponding to the candidate splitting scheme.

[0078] A candidate splitting plan refers to a plan for splitting a business to be split into multiple sub-businesses.

[0079] In some embodiments, the management platform obtains candidate splitting solutions in a variety of ways. For example, the management platform sets multiple sub-business numbers, each sub-business number corresponds to a candidate splitting solution, and splits the business to be split in the way of splitting the business to be split in step 240 based on each sub-business number to obtain multiple candidate splitting solutions.

[0080] In some embodiments, the management platform splits the business to be split multiple times by splitting functional items and data items to obtain multiple candidate splitting schemes, wherein the data item splitting includes splitting the data items of the business to be split based on historical call information and data item features of the business to be split.

[0081] For instructions on function items, data items, and the splitting of the business to be split based on function items, see Figure 2 and its related description.

[0082] Historical call information refers to information related to historical data items called at historical times. In some embodiments, the historical call information includes multiple historical data items called at multiple historical time points and historical data item features of different historical data items. Historical data items refer to data items corresponding to historical services.

[0083] Data item characteristics refer to data related to the data item itself. In some embodiments, data item characteristics include data item sources, data item time, and data item types of one or more data items. Data item sources include workshops or production lines. Data item time refers to the time when the data item is uploaded. Data item type refers to the type of function item corresponding to the data item.

[0084] In some embodiments, the management platform may perform normalization and other processing on each data item in the data item feature so as to represent each data item in the data item feature by means of numerical values ​​or other means, so as to facilitate subsequent calculations.

[0085] In some embodiments, the management platform determines the relevance between any two historical data items based on the historical call information, and splits the data items of the business to be split based on multiple relevances and the historical call information.

[0086] Correlation refers to data that measures the degree of association between two historical data items. In some embodiments, the management platform calculates the number of times two historical data items are called simultaneously based on historical call information, and uses the number of times they are called simultaneously as the correlation between the two historical data items.

[0087] In some embodiments, the management platform splits the data items of the business to be split based on the multiple relevances and historical call information, including: S1: Based on multiple relevances and historical data item features, multiple historical data items are divided into multiple first groups.

[0088] In some embodiments, the management platform traverses each historical data item, screens historical data items whose relevance to the currently traversed historical data item is greater than a relevance threshold, and divides such historical data items and the currently traversed historical data item into a first group. The traversal process continues. If the currently traversed historical data item has been divided into a first group, the historical data items whose relevance to the currently traversed historical data item is greater than the relevance threshold are also divided into the same first group. The traversal process continues until all historical data items are traversed. The relevance threshold is set based on experience.

[0089] Exemplarily, the management platform traverses historical data items a, b, c, d, and e, first screens out historical data item b whose correlation with a is greater than the correlation threshold, and divides a and b into a first group. Then screens out historical data item d whose correlation with b is greater than the correlation threshold, and adds d to the first group where b is located. Then screens out historical data item e whose correlation with c is greater than the correlation threshold, and divides c and e into a first group.

[0090] S2: Calculate the grouping feature of each first group. The grouping feature refers to data that can characterize the relevant information of the data items in the group. In some embodiments, the management platform calculates the average of the historical data item features of multiple historical data items in a single first group as the grouping feature of the first group.

[0091] S3: Based on the historical data item features and multiple grouping features, the historical data items that are not grouped in S1 are divided into multiple first groups.

[0092] In some embodiments, for each historical data item in the ungrouped S1, the management platform constructs a data feature vector based on the historical data item features of the historical data item, constructs multiple grouping feature vectors based on multiple grouping features, and calculates the similarity between the data feature vector and the multiple grouping feature vectors. The grouping feature vectors with the greatest similarity and a similarity greater than the similarity threshold are selected, and the historical data item is divided into the first group corresponding to the grouping feature vector. The similarity threshold is set based on experience. Vector similarity is negatively correlated with vector distance. Vector distance includes Euclidean distance, etc.

[0093] S4: Based on the characteristics of the historical data items, the historical data items in the ungrouped state in S3 are divided into a plurality of second groups.

[0094] In some embodiments, the management platform constructs a clustering vector based on the historical data item features of the historical data items in the ungrouped data in S3, performs clustering based on multiple clustering vectors to form multiple cluster clusters, and divides the historical data items corresponding to the clustering vectors in each cluster cluster into a second group. The management platform calculates the grouping features of each second group using the method for calculating the grouping features in S2.

[0095] S5: taking the plurality of first groups and second groups as target groups, and dividing the data items of the business to be split into different target groups based on the characteristics of the data items of the business to be split.

[0096] In some embodiments, for each data item of the business to be split, the management platform constructs a vector to be matched based on the data item features of the data item, calculates the similarity between the vector to be matched and multiple grouping feature vectors, selects the grouping feature vector with the greatest similarity, and divides the data item into the target group corresponding to the grouping feature vector. The above process is repeated until all data items of the business to be split are divided.

[0097] After the division is completed, the management platform will retain the target grouping containing the data items of the business to be split, and delete the historical data items in the retained target grouping, to obtain multiple target groups that only include the data items of the business to be split, and take each target grouping as a sub-business, and form multiple sub-businesses into a candidate splitting plan.

[0098] In some embodiments, when the management platform splits the business to be split multiple times through functional item splitting and data item splitting based on the number of sub-businesses, the data item splitting can be performed through the above method to obtain multiple candidate splitting schemes.

[0099] The data items of the business to be split are split through historical call information and data item characteristics of the business to be split, so that the data items in each sub-business obtained after the data items are split are related, which is conducive to improving the efficiency of subsequent processing of sub-businesses by the data distribution platform after splitting according to the candidate splitting plan.

[0100] Step 311: Determine the sub-business data volumes corresponding to the multiple candidate sub-businesses in the candidate splitting scheme based on the sub-business characteristics corresponding to the multiple candidate sub-businesses and the data characteristics of the business to be split.

[0101] A candidate sub-business refers to a sub-business obtained by splitting the business to be split based on the candidate splitting plan. For more information about the characteristics of sub-businesses, see Figure 2 and its related description.

[0102] The data characteristics of the business to be split refer to data used to characterize data-related information of the business to be split. In some embodiments, the data characteristics of the business to be split include the amount of running program data of each function item, the amount of configuration file data, and the amount of data items to be processed.

[0103] The sub-service data volume refers to the data volume corresponding to the candidate sub-service. In some embodiments, the sub-service data volume includes the running program data volume and configuration file data volume of the function item and the data volume of the data item.

[0104] In some embodiments, for each candidate sub-business, the management platform determines the same functional items based on the sub-business characteristics of the candidate sub-business. The amount of running program data and configuration file data of the same functional items are included in the sub-business data volume. The management platform calculates the ratio of the number of data items of the candidate sub-business to the number of data items to be processed of the same functional items, multiplies the calculated ratio by the data volume of the data items to be processed, and includes the product in the sub-business data volume. The same functional items refer to the functional items in the data characteristics of the business to be split that are the same as the functional items corresponding to the candidate sub-business.

[0105] Step 312: determine a first transmission volume based on the sub-service data volume.

[0106] The first transmission volume refers to the total amount of data transmitted for multiple candidate sub-services.

[0107] In some embodiments, the management platform calculates the sum of the sub-service data volumes based on the sub-service data volumes of the multiple candidate sub-services, and uses the product of the sum and the transmission volume coefficient as the first transmission volume. The transmission volume coefficient is related to the number of the multiple candidate sub-services. For example, the transmission volume coefficient is , is the number of multiple candidate sub-services.

[0108] In some embodiments, the management platform obtains multiple candidate transmission schemes, determines second transmission volumes corresponding to the multiple candidate transmission schemes based on the sub-service data volume, and obtains the first transmission volume based on the multiple second transmission volumes.

[0109] The candidate transmission scheme refers to a scheme for transmitting multiple candidate sub-services to the second sub-platform. In some embodiments, the candidate transmission scheme includes retaining a local service and transmitting multiple candidate sub-services to the second sub-platform. The local service refers to the service processed at the first sub-platform.

[0110] In some embodiments, for each candidate sub-service, the management platform uses the current candidate sub-service as a local service and uses the candidate sub-services other than the current candidate sub-service as candidate sub-services to be transmitted to the second sub-platform, thereby obtaining a candidate transmission scheme. The management platform uses each candidate sub-service as a local service and obtains multiple candidate transmission schemes through the above method.

[0111] The second transmission volume refers to the total amount of data of multiple candidate sub-services transmitted according to the candidate transmission scheme.

[0112] In some embodiments, the management platform calculates the sum of the sub-service data volumes of the candidate sub-services transmitted to the second sub-platform in the candidate transmission scheme based on the sub-service data volumes, and uses the sum as the second transmission volume corresponding to the candidate transmission scheme.

[0113] In some embodiments, the management platform performs a weighted summation of multiple second transmission amounts and uses the sum as the first transmission amount. The weights of different candidate transmission schemes are negatively correlated to the first sum of the local services in the candidate transmission schemes. The larger the first sum, the more processing resources are required for the local service, and the more the local service is inclined to be transferred to the second sub-platform, so the weight is smaller. For an explanation of the first sum, see Figure 2 and its related description.

[0114] Through multiple candidate transmission schemes, the processing resource requirements of different candidate sub-services can be considered when determining the first transmission volume, and then the first transmission volume can be obtained by combining multiple candidate transmission schemes, so that the determined first transmission volume is more in line with the actual data transmission volume under the candidate splitting scheme.

[0115] Step 313: Determine the data transmission time corresponding to the candidate splitting scheme based on the first transmission amount and the available communication bandwidth.

[0116] Data transmission time refers to the time required to transmit multiple candidate sub-businesses under the candidate splitting scheme to the second sub-platform.

[0117] The available communication bandwidth refers to the communication link bandwidth that can be used by the first sub-platform to transmit multiple candidate sub-services. In some embodiments, the management platform calculates the average of the communication link bandwidths between the first sub-platform and multiple second sub-platforms, and uses the average as the available communication bandwidth.

[0118] In some embodiments, the data transmission time is positively correlated with the first transmission amount and negatively correlated with the available communication bandwidth. Exemplarily, the management platform determines the data transmission time based on the first transmission amount and the available communication bandwidth by the following formula (2): (2) Wherein, F is the data transmission time, m is the first transmission amount, is the number of multiple candidate sub-services, and b is the available communication bandwidth.

[0119] Step 314, based on the business characteristics of the business to be split, the sub-business characteristics and operation characteristics corresponding to the multiple candidate sub-businesses, determine the business operation efficiency corresponding to the candidate splitting scheme.

[0120] For a description of the business and operating characteristics, see Figure 2 and its related description.

[0121] The business operation efficiency refers to the estimated efficiency of processing the business to be split after the business to be split is split based on the candidate splitting scheme. In some embodiments, the business operation efficiency is represented by the sum of the operation efficiencies of multiple candidate sub-businesses. The operation efficiency of the candidate sub-business is represented by the number of executions of the candidate sub-business per unit time. When a data item of the candidate sub-business is processed, the execution number is increased by 1.

[0122] In some embodiments, the management platform determines the processing resource requirements of each candidate sub-business based on the business characteristics of the business to be split and the sub-business characteristics corresponding to the multiple candidate sub-businesses. For instructions on determining the processing resource requirements, see Figure 2 and its related description.

[0123] In some embodiments, the management platform determines the business operation efficiency in a variety of ways based on the processing resource requirements and operation characteristics corresponding to multiple candidate sub-businesses. For example, for each candidate sub-business, the management platform queries the reference operation efficiency corresponding to the average operation characteristics and the processing resource requirements of the candidate sub-business in a preset table based on the average operation characteristics and the processing resource requirements of the candidate sub-business, and uses the reference operation efficiency as the operation efficiency of the candidate sub-business. The management platform calculates the sum of the operation efficiencies of multiple candidate sub-businesses as the business operation efficiency. The average operation characteristics include the average of the total CPU utilization rate and the average of the total memory occupancy rate of multiple data sub-platforms.

[0124] In some embodiments, the preset table is constructed based on experimental data. The experimental process includes: under an average operation characteristic, the sub-business with determined processing resource requirements is run on multiple data sub-platforms to obtain multiple operation efficiencies, and the average of the multiple operation efficiencies is used as a reference operation efficiency corresponding to the average operation characteristic and the processing resource requirements.

[0125] In some embodiments, the management platform can determine the business operation efficiency through an efficiency estimation model based on the processing resource requirements and operation characteristics corresponding to multiple candidate sub-businesses. For more information, see Figure 4 and its related description.

[0126] Step 320: Determine a target splitting scheme based on multiple data transmission times and multiple service operation efficiencies corresponding to multiple candidate splitting schemes.

[0127] The target splitting scheme refers to the candidate splitting scheme to be used.

[0128] In some embodiments, the management platform uses the candidate splitting scheme with the highest service operation efficiency among multiple candidate splitting schemes whose data transmission time is less than the transmission time threshold as the target splitting scheme, wherein the transmission time threshold is preset based on experience.

[0129] Step 330: Split the business to be split based on the target splitting plan to obtain multiple sub-businesses corresponding to the business to be split.

[0130] The management platform splits the business to be split into multiple candidate sub-businesses according to the steps of splitting the business to be split into multiple candidate sub-businesses in the target splitting plan, and obtains multiple sub-businesses corresponding to the business to be split.

[0131] By obtaining multiple candidate splitting plans and evaluating the data transmission time and business operation efficiency of the sub-businesses after splitting based on the candidate splitting plans, a better splitting plan can be found, which is conducive to making more appropriate splitting of the split business, avoiding excessive computing pressure on the second sub-platform due to excessive sub-businesses, and balancing the computing pressure of each data sub-platform.

[0132] Figure 4 is an exemplary schematic diagram of an efficiency estimation model according to some embodiments of the present specification.

[0133] In some embodiments, the management platform may also determine the business operation efficiency 440 through an efficiency estimation model 430 based on the processing resource requirements 410 and operation characteristics 420 corresponding to the plurality of candidate sub-businesses.

[0134] For more information about candidate sub-services, processing resource requirements, operating characteristics, and service operating efficiency, see Figure 2 and Figure 3 and its related description.

[0135] The efficiency estimation model refers to a model used to determine the efficiency of business operations. In some embodiments, the efficiency estimation model is a machine learning model. For example, any one or combination of a recurrent neural network (RNN) model or a deep neural network (DNN) model or other custom model structures.

[0136] In some embodiments, the management platform trains the efficiency estimation model based on the training data set by gradient descent method, etc. The training data set includes multiple groups of training samples and labels corresponding to different training samples. Each group of training samples includes sample operation characteristics and sample processing resource requirements corresponding to multiple sample sub-businesses of the sample to be split business, and the label of each group of training samples includes the actual business operation efficiency of the group of sample to be split business.

[0137] In some embodiments, training samples and labels are obtained based on historical data. For example, the management platform can use historical to-be-split businesses as sample to-be-split businesses, and randomly distribute multiple sample sub-businesses after the sample to-be-split businesses are split to multiple historical second sub-platforms, and calculate the actual business operation efficiency of the sample to-be-split businesses after each random distribution. The management platform uses the average of the business operation efficiency corresponding to the multiple random distributions as a label. For the determination of business operation efficiency, see Figure 3 and its related description.

[0138] In some embodiments, the efficiency estimation model is trained by: inputting a plurality of labeled training samples into the initial efficiency estimation model, constructing a loss function through the labels and the prediction results of the initial efficiency estimation model, iteratively updating the initial efficiency estimation model based on the loss function, and completing the efficiency estimation model training when the loss function satisfies a preset training condition. The preset training condition includes the convergence of the loss function, the number of iterations reaching a set value, etc.

[0139] By processing the operating characteristics and the processing resource requirements corresponding to multiple candidate sub-businesses through the efficiency prediction model, we can use the self-learning ability of the machine learning model to find patterns from large amounts of data, obtain the operating characteristics and the correlation between the processing resource requirements corresponding to multiple candidate sub-businesses and the business operating efficiency, and improve the accuracy and efficiency of determining the business operating efficiency.

[0140] Figure 5 FIG. 1 is an exemplary flow chart of determining service adjustment parameters according to some embodiments of this specification. Figure 5 As shown, the process 500 of determining the service adjustment parameter includes the following steps.

[0141] Step 510: determine a plurality of candidate allocation relationships based on processing resource requirements and operation characteristics, and determine estimated operation information corresponding to the plurality of candidate allocation relationships.

[0142] The candidate allocation relationship refers to the allocation relationship between multiple sub-services and sub-platforms. In some embodiments, the candidate allocation relationship includes the allocation results of multiple sub-services. For a description of the allocation results, processing resource requirements and operating characteristics, see Figure 2 and its related description.

[0143] In some embodiments, the management platform determines multiple candidate allocation relationships based on processing resource requirements and operating characteristics. For example, the management platform randomly allocates multiple sub-services to multiple second sub-platforms multiple times, thereby determining multiple candidate allocation relationships. When randomly allocating sub-services to the second sub-platforms, the sum of the total CPU usage of the second sub-platforms does not exceed the CPU allocation threshold. For an explanation of the CPU allocation threshold, see Figure 2 and its related description.

[0144] The estimated operation information refers to information related to the operation of multiple second sub-platforms in the future after allocation based on the candidate allocation relationship. In some embodiments, the estimated operation information includes the estimated total CPU usage and the estimated total memory occupancy of each second sub-platform.

[0145] In some embodiments, the management platform determines the estimated operation information based on the processing resource requirements of each sub-business determined in step 250 and the operation characteristics of multiple second sub-platforms in the candidate allocation relationship. For example, the management platform calculates the sum of the CPU usage of multiple sub-businesses allocated to the second sub-platform, and calculates the sum of the CPU usage and the total CPU usage in the operation characteristics, and uses the sum as the estimated total CPU usage of the second sub-platform. The calculation method of estimating the total memory occupancy is similar to that of estimating the total CPU usage.

[0146] Step 520, determining a target allocation relationship based on the estimated operation information.

[0147] The target allocation relationship refers to the candidate allocation relationship to be used.

[0148] In some embodiments, the management platform calculates the third sum corresponding to the candidate allocation relationship based on the estimated operation information, and selects the candidate allocation relationship with the smallest third sum as the target allocation relationship, wherein the third sum is the sum of the variances of the multiple estimated CPU total usage rates and the variances of the multiple estimated memory total occupancy rates.

[0149] Step 530: Determine the service adjustment parameters based on the target allocation relationship.

[0150] In some embodiments, the management platform determines the allocation result of each sub-business in the target allocation relationship as the allocation result of each sub-business in the business adjustment parameter.

[0151] By evaluating the business processing efficiency of various allocation relationships, it is helpful to find the optimal allocation relationship and effectively improve the subsequent business processing efficiency.

[0152] It should be noted that the above description of the process 200 of the computing scheduling method based on the industrial Internet of Things data center, the process 300 of splitting the business to be split, and the process 500 of determining the business adjustment parameters are only for example and explanation, and do not limit the scope of application of the present invention. For those skilled in the art, various modifications and changes can be made to the process 200 of the computing scheduling method based on the industrial Internet of Things data center, the process 300 of splitting the business to be split, and the process 500 of determining the business adjustment parameters under the guidance of the present invention. However, these modifications and changes are still within the scope of the present invention.

[0153] In some embodiments of the present invention, a computer-readable storage medium is further provided, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the computing and scheduling method based on an industrial Internet of Things data center described in any one of the above embodiments.

[0154] Furthermore, certain features, structures or characteristics in one or more embodiments of the present invention may be appropriately combined.

[0155] In some embodiments, the numerical parameters used in the specification and claims are approximate values, which may vary according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining the digits. Although the numerical domains and parameters used to confirm the breadth of the range in some embodiments of the present invention are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0156] If there is any inconsistency or conflict between the descriptions, definitions, and / or usage of terms in the referenced materials of this invention and the contents of this invention, the descriptions, definitions, and / or usage of terms in this invention shall prevail.

Claims

1. A computing and scheduling system based on an industrial Internet of Things data center, characterized in that: The system includes a management platform, a data center and a plurality of data sub-platforms connected in communication, wherein the management platform is configured as follows: Obtaining operation characteristics of the multiple data sub-platforms; Based on the operation characteristics, determining a first sub-platform and a second sub-platform among the plurality of data sub-platforms; Determine at least one business to be split based on the business characteristics corresponding to the multiple businesses of the first sub-platform and the operation characteristics; Splitting the at least one business to be split into a plurality of sub-businesses based on business characteristics of the at least one business to be split; Determining processing resource requirements corresponding to the multiple sub-businesses based on the business characteristics of the at least one business to be split, the operation characteristics, and sub-business characteristics corresponding to the multiple sub-businesses; Determining a service adjustment parameter based on the processing resource requirement and the operation characteristics; Based on the service adjustment parameters, a service adjustment instruction is generated and sent to the first sub-platform and the second sub-platform.

2. The system according to claim 1, characterized in that The first sub-platform is configured as follows: Based on the service adjustment instruction, data processing is performed on the multiple sub-services to obtain multiple data packets; Sending the plurality of data packets to the second sub-platform; The second sub-platform is configured as follows: receiving the plurality of data packets; Parsing the multiple data packets to determine the data volumes corresponding to the multiple data packets; Based on the data volume, corresponding parsing memory is allocated to the multiple data packets, and the multiple data packets are parsed to obtain parsed data; Based on the parsed data, establishing processes corresponding to the multiple sub-businesses; Based on the service adjustment instruction, a processing resource for processing the process is determined.

3. The system according to claim 1, characterized in that The management platform is further configured to: For a single business to be split among the at least one business to be split: Obtain multiple candidate splitting solutions; Determine a target splitting scheme based on multiple data transmission times and multiple business operation efficiencies corresponding to the multiple candidate splitting schemes; Based on the target splitting plan, the business to be split is split to obtain multiple sub-businesses corresponding to the business to be split.

4. The system according to claim 3, characterized in that The management platform is further configured to: The business to be split is split multiple times by splitting functional items and data items to obtain the multiple candidate splitting solutions.

5. The system according to claim 1, wherein: The management platform is further configured to: Based on the processing resource requirements and the operation characteristics, determining a plurality of candidate allocation relationships, and determining estimated operation information corresponding to the plurality of candidate allocation relationships; Determining a target allocation relationship based on the estimated operation information; Based on the target allocation relationship, the service adjustment parameter is determined.

6. A computing scheduling method based on an industrial Internet of Things data center, characterized in that: The method is executed by a management platform, and includes: Obtain the operating characteristics of multiple data sub-platforms; Based on the operation characteristics, determining a first sub-platform and a second sub-platform among the plurality of data sub-platforms; Determine at least one business to be split based on the business characteristics corresponding to the multiple businesses of the first sub-platform and the operation characteristics; Splitting the at least one business to be split into a plurality of sub-businesses based on business characteristics of the at least one business to be split; Determining processing resource requirements corresponding to the multiple sub-businesses based on the business characteristics of the at least one business to be split, the operation characteristics, and sub-business characteristics corresponding to the multiple sub-businesses; Determining a service adjustment parameter based on the processing resource requirement and the operation characteristics; Based on the service adjustment parameters, a service adjustment instruction is generated and sent to the first sub-platform and the second sub-platform.

7. The method according to claim 6, characterized in that The step of splitting the at least one to-be-split service into a plurality of sub-services based on the service characteristics of the at least one to-be-split service comprises: For a single business to be split among the at least one business to be split: Obtain multiple candidate splitting solutions; Determine a target splitting scheme based on multiple data transmission times and multiple business operation efficiencies corresponding to the multiple candidate splitting schemes; Based on the target splitting plan, the business to be split is split to obtain multiple sub-businesses corresponding to the business to be split.

8. The method according to claim 7, characterized in that The obtaining of multiple candidate splitting solutions comprises: The business to be split is split multiple times by splitting functional items and data items to obtain the multiple candidate splitting solutions.

9. The method according to claim 6, characterized in that The determining of the service adjustment parameter based on the processing resource requirement and the operation characteristic includes: Based on the processing resource requirements and the operation characteristics, determining a plurality of candidate allocation relationships, and determining estimated operation information corresponding to the plurality of candidate allocation relationships; Determining a target allocation relationship based on the estimated operation information; Based on the target allocation relationship, the service adjustment parameter is determined.

10. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the method as claimed in claim 6.

Images