Optimization method and device for resource integration based on pk, and storage medium

Abstract

The application provides a PKS-based resource integration optimization method, device and storage medium, relates to the technical field of resource integration, and the method comprises the following steps: determining the required number of containers and a resource allocation strategy according to the scale and performance requirements of a computer cluster, configuring a virtual network according to the network requirements of a training system, constructing a container cloud platform, establishing a storage resource pool according to the data types of target devices that have been accessed, allocating corresponding computing resources to application programs based on the container cloud platform, configuring a load balancer on the container cloud platform, setting a load balancing algorithm, distributing an operation request to a matched computing node for request analysis when receiving the operation request of a user on the target device, and statistically displaying the working conditions of each application program based on the container cloud platform and the use conditions of each storage resource pool in real time, so that the efficient operation and stable function of the training system are effectively ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource integration, and particularly relates to a PKS-based resource integration optimization method and device and a storage medium. BACKGROUND

[0002] PKS refers to Pivotal Container Service, which is a Kubernetes-based container orchestration platform. Its goal is to simplify the deployment, management and expansion of containers, and provide a highly available and elastic containerized solution.

[0003] In today's training system, there are many severe technical challenges. On the one hand, the information data source of the training system is extremely extensive, covering a variety of different data sources, and the data types are various, including structured data and a large amount of unstructured data, and the data quantity is huge and the content style is rich and varied. On the other hand, the current data storage management is in a scattered state, which makes it difficult to effectively query and access the data. In the decision-making process, the management layer cannot obtain accurate and effective decision-making data support in time, which greatly affects the scientificity and timeliness of the decision-making.

[0004] In addition, the sharing service demand in different scenarios presents the characteristics of diversification, and the technical system is different, which makes the interconnection and interworking ability seriously insufficient. Between the sharing services, resource scheduling is difficult, and is accompanied by the problem of excessive resource consumption. These problems seriously restrict the efficient operation and function of the training system.

[0005] Therefore, the present application provides a PKS-based resource integration optimization method, device and storage medium. SUMMARY

[0006] The present application provides a PKS-based resource integration optimization method, device and storage medium, which realizes resource integration by constructing a container cloud platform, establishing a storage resource pool, allocating computing resources, configuring a load balancer, etc., and effectively ensures the efficient operation and stable function of the training system.

[0007] The present application provides a PKS-based resource integration optimization method, which comprises:

[0008] Step 1: according to the scale and performance requirements of the computer cluster, determine the number of containers required to be deployed and the resource allocation strategy, and configure a virtual network according to the network requirements of the training system, and construct a container cloud platform;

[0009] Step 2: connect different types of target devices to the container cloud platform, and establish a storage resource pool according to the data types of the connected target devices;

[0010] Step 3: Based on the performance requirements of the applications on the target device, allocate corresponding computing resources for the applications based on the container cloud platform, wherein the computing resources of each application are distributed on multiple computing nodes;

[0011] Step 4: Configure a load balancer on the container cloud platform, set a load balancing algorithm, and when receiving a user operation request on the target device, distribute the operation request to the matched computing node for request analysis;

[0012] Step 5: Real-time statistics of each application based on the container cloud platform and the use of each storage resource pool, and visual display.

[0013] Preferably, the number of containers required for deployment is determined, including:

[0014] According to the historical running data of the computer cluster in the historical period, determine the bottleneck rate of the computer cluster under each existing performance indicator;

[0015] ;

[0016] Wherein, represents the bottleneck rate under the corresponding existing performance indicator; represents the number of time points in the historical period that meet corresponding to the existing performance indicators; represents the total number of time points involved in the historical period; represents the maximum set amount of the corresponding existing performance indicator at the i-th time point; represents the actual consumption amount of the corresponding existing performance indicator at the i-th time point; represents the ratio set threshold of the corresponding existing performance indicator at the i-th time point; represents the variance of all corresponding to the existing performance indicators;

[0017] Obtain the historical task set of the computer cluster in the historical period, and simulate the historical task set, and determine the simulated arrival rate Ud and simulated average processing time Ut of the historical task set in the simulation process by using mathematical models such as queuing theory;

[0018] According to the historical processing log of the computer cluster to the historical task set, obtain the historical arrival rate Ld and the historical average processing time Lt, and combine the simulated arrival rate Ud, the simulated average processing time Ut and the bottleneck rate to determine the redundancy coefficient for each existing performance indicator;

[0019] The container quantity based on each existing performance index is obtained by matching from the coefficient-index-quantity table, and the maximum quantity is filtered from all container quantities, and the initial quantity obtained based on the historical arrival rate Ld and the historical average processing time Lt is combined to obtain the required deployed container quantity.

[0020] Preferably, the redundancy coefficient for each existing performance index is determined, including:

[0021]

[0022] Wherein, R represents the redundancy coefficient corresponding to the existing performance index; represents the maximum value of the historical processing time of each task obtained in the historical processing log of the historical task set.

[0023] Preferably, the existing performance index includes: CPU utilization, memory occupancy and network traffic consumption.

[0024] Preferably, the storage resource pool is established according to the data type of the accessed target device, including:

[0025] Determine the data type of each data sample generated by the accessed target device under the specified working period, wherein the data type is structured type and unstructured type;

[0026] Analyze the data structure, field type and data size of each first sample under the structured type, and determine the theoretical required space of the corresponding first sample;

[0027] Analyze the data format, file size and content characteristics of each second sample under the unstructured type, and determine the theoretical required space of the corresponding second sample;

[0028] Based on all the theoretical required spaces and the working frequency of the corresponding accessed target device and the data generation type under the high-frequency working task, the sub-resource pool is configured to the corresponding device;

[0029] Based on the sub-resource pool of all the accessed target devices, the storage resource pool is obtained.

[0030] Preferably, the sub-resource pool is configured to the corresponding device, including:

[0031] ;

[0032] Wherein, represents the optimization function; represents the theoretical required space of the i1th first sample; represents the theoretical required space of the i2th second sample; represents the size of the sub-resource pool configured to the corresponding device; represents the number of historical actual work tasks of the corresponding target device in the specified work cycle; represents the number of saturated work tasks of the corresponding target device in the specified work cycle; n1 represents the number of first samples; n2 represents the number of second samples; represents the maximum value in all . represents the maximum value in all .

[0033] Preferably, the working conditions of each application based on the container cloud platform and the usage of each storage resource pool are counted in real time, including:

[0034] The self information and running state of each application are collected, and the business index data is collected based on the setting of each application, wherein the working conditions are related to the self information, running state and business index data;

[0035] The current usage of the storage resource pool by the working conditions of the corresponding application is determined, and the future usage is predicted according to the working conditions and the called computing resources.

[0036] The application provides an optimization device for resource integration based on PKS, comprising:

[0037] A platform construction module is configured to determine the number of required containers and resource allocation strategies according to the scale and performance requirements of a computer cluster, and to configure a virtual network according to the network requirements of a training system to construct a container cloud platform.

[0038] A resource pool establishment module is configured to connect different types of target devices to the container cloud platform, and to establish storage resource pools according to the data types of the connected target devices.

[0039] A resource allocation module is configured to allocate corresponding computing resources to the applications based on the container cloud platform according to the performance requirements of the applications on the target devices, wherein the computing resources of each application are distributed on multiple computing nodes.

[0040] A request analysis module is configured to configure a load balancer on the container cloud platform, set a load balancing algorithm, and distribute operation requests on the target devices to matched computing nodes for request analysis when receiving the operation requests.

[0041] A visualization module is configured to count the working conditions of each application based on the container cloud platform and the usage of each storage resource pool in real time, and to visually display the working conditions and the usage.

[0042] The application provides a storage medium, which stores a computer program, and the computer program is characterized in that the computer program is executed by a processor to realize the optimization method for PKS-based resource integration.

[0043] Compared with the prior art, the application has the following beneficial effects:

[0044] By constructing a container cloud platform, establishing a storage resource pool, allocating computing resources, and configuring a load balancer, resource integration is realized, and efficient operation and stable function of the training system are effectively ensured. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 is a flowchart of the optimization method for PKS-based resource integration provided by the embodiment of the application;

[0047] Figure 2 is a structural diagram of the optimization device for PKS-based resource integration provided by the embodiment of the application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the application more clear, the technical solutions in the application will be clearly and completely described below in combination with the drawings in the application. Obviously, the described embodiments are some embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0049] The application provides an optimization method for PKS-based resource integration, as shown in Figure 1 , which comprises the following steps:

[0050] Step 1: according to the scale and performance requirements of the computer cluster, the number of containers required to be deployed and the resource allocation strategy are determined, and a virtual network is configured according to the network requirements of the training system, and a container cloud platform is constructed;

[0051] Step 2: different types of target devices are connected to the container cloud platform, and a storage resource pool is established according to the data types of the connected target devices;

[0052] Step 3: Based on the performance requirements of the applications on the target device, allocate corresponding computing resources to each application based on the container cloud platform, where the computing resources of each application are distributed across multiple computing nodes.

[0053] Step 4: Configure a load balancer on the container cloud platform, set a load balancing algorithm, and when receiving a user operation request on the target device, distribute the operation request to the matching computing node for request analysis.

[0054] Step 5: Real-time statistics of each application based on the container cloud platform and the usage of each storage resource pool, and visual display.

[0055] In this embodiment, the computer cluster size refers to the number of physical servers that make up the computer cluster, the hardware configuration of the servers (such as the total number of CPU cores, the total memory capacity, the total disk storage capacity, etc.), and the specifications and number of network devices (such as switches, routers, etc.). For example, a computer cluster consists of 10 servers equipped with 8-core CPUs, 64GB of memory, and 1TB of hard drives, connected through a gigabit network switch. This is one description of the cluster size. The way to obtain it is to statistically summarize the parameters of each physical device in the cluster.

[0056] In this embodiment, according to the functions and quality of service that the training system is expected to achieve, the index requirements of the computer cluster in terms of computing speed, data processing capability, response time, etc. For example, the training system requires the ability to simulate the actions of 1000 combat units simultaneously, and the response time of the simulation calculation is not more than 1 second, which is a specific requirement for performance. Usually determined by the design team of the training system according to business requirements and industry standards.

[0057] In this embodiment, the resource allocation strategy determines the rules for allocating CPU, memory, disk I / O, network bandwidth, etc. of the computer cluster to different containers or tasks. For example, based on task priority, 70% of CPU time slice is allocated to high-priority real-time command and control tasks, and low-priority data statistics tasks share the remaining 30%. The strategy needs to consider task type, resource demand characteristics, and system overall performance optimization, etc.

[0058] The network requirements of the training system include the requirements for network bandwidth, network delay, network reliability, network topology structure, etc. For example, real-time video transmission requires network bandwidth not less than 100Mbps, network delay less than 50ms, and redundant links to ensure reliability. Through the analysis of network traffic and real-time performance of various businesses in the training system (such as simulation, real-time communication, etc.).

[0059] In this embodiment, a virtual network is configured on the container cloud platform, network bandwidth restrictions, access control policies, etc. are set. For example, a virtual network is created, and independent subnets are divided for different types of tasks, such as simulation tasks in subnet A and data transmission tasks in subnet B. A special SDN controller software (such as OpenDaylight, ONOS, etc.) is used for configuration and creation.

[0060] In this embodiment, various hardware devices are connected to the container cloud platform, which have different functions and data generation / transmission characteristics. For example, sensor devices (temperature sensors, pressure sensors, etc.) are used to collect environmental data, and intelligent terminal devices (smartphones, tablets) are used for user interaction and data display. The devices are classified and identified by their functions, data interface types, etc.

[0061] In this embodiment, the time series data generated by the sensor device is usually structured numerical data, and the image data taken by the intelligent terminal device is unstructured data.

[0062] In this embodiment, the storage resource pool is to provide a space for storing related resources for the corresponding device, and also to ensure the normal operation of the device. The application program is running on the target device.

[0063] In this embodiment, the computing node is a physical server or virtual server in the computer cluster that actually performs the computing task.

[0064] In this embodiment, the load balancer can distribute user access requests to the simulation training system to different computing nodes according to the configured algorithm. Hardware load balancing devices (such as F5, A10, etc.) can be used, or software load balancers (such as Nginx, HAProxy, etc.) can be used. For example, a weighted round-robin algorithm is used to set weights according to the CPU performance and memory size of the computing nodes, and the nodes with higher performance are allocated more requests.

[0065] In this embodiment, the operation request is a request generated by the user when operating the simulation training system on the target device, such as clicking a button, inputting an instruction, querying data, etc.

[0066] In this embodiment, the computing node receives a request to query battlefield situation data, analyzes the request parameters to determine which data to query from the database, and performs corresponding data retrieval and processing. The request analysis is completed by the application logic module running on the computing node.

[0067] In this embodiment, the application program running state on the container cloud platform, resource usage, task execution progress and other information. For example, a data processing application CPU utilization is 60%, memory occupation 8GB, is currently processing the 500th data block. Through the monitoring tool of container cloud platform (such as Prometheus, Grafana and so on) collects the performance index and state information of application running time to obtain, namely working condition.

[0068] In this embodiment, the storage resource pool in the storage capacity of the occupation, read and write operation frequency, performance index of storage device. For example, the total capacity of storage resource pool is 10TB, has used 6TB, the current read and write speed is 100MB / s and 50MB / s respectively, namely the use condition.

[0069] In this embodiment, the visualization display is to collect the application working condition and storage resource pool use condition and other data, to intuitive graphics, chart and other forms display, convenient for administrator and user to view and analyze. For example, using Grafana to display the CPU utilization of application program with line chart, the use condition of storage resource pool with column chart. Through the data visualization tool (such as Grafana, Kibana and so on) is connected to the data collection source (such as monitoring tool, storage management software and so on), the corresponding data display panel is configured to realize.

[0070] The beneficial effects of the above technical scheme are: through the construction of container cloud platform, the establishment of storage resource pool, the allocation of computing resources, the configuration of load balancer, the resource integration is realized, and the efficient operation and stable function of the training system are effectively ensured.

[0071] The application provides an optimization method for resource integration based on PKS, determines the number of containers required to be deployed, and comprises the following steps:

[0072] According to the historical running data of the computer cluster in the historical period, the bottleneck rate of the computer cluster under each existing performance index is determined;

[0073] ;

[0074] Among them, Indicates the bottleneck rate under the corresponding existing performance index; Indicates the number of time points in the historical period that meet under the corresponding existing performance index; Indicates the total number of time points involved in the historical period; Indicates the maximum set amount under the corresponding existing performance index at the i th time point; Indicates the actual consumption amount under the corresponding existing performance index at the i th time point; a threshold value is set for the ratio of the corresponding existing performance indicator at the i-th moment; a variance of all corresponding to the existing performance indicators;

[0075] The computer cluster obtains a historical task set in a historical period and simulates the historical task set, and uses a mathematical model such as queuing theory to determine a simulated arrival rate Ud and a simulated average processing time Ut of the historical task set in the simulation process;

[0076] According to the historical processing log of the computer cluster for the historical task set, a historical arrival rate Ld and a historical average processing time Lt are obtained, and in combination with the simulated arrival rate Ud, the simulated average processing time Ut, and the bottleneck rate, a redundancy coefficient for each existing performance indicator is determined.

[0077] The number of containers based on each existing performance indicator is obtained by matching from a coefficient-indicator-number correspondence table, and the maximum number is selected from all container numbers, and in combination with an initial number obtained based on the historical arrival rate Ld and the historical average processing time Lt, a required number of deployed containers is obtained.

[0078] Preferably, the existing performance indicators include CPU utilization, memory occupancy, and network traffic consumption.

[0079] In this embodiment, the historical period can be the past half hour, and each 1 min is recorded in turn, at this time, T0 is 30.

[0080] In this embodiment, the actual consumption of CPU utilization can be 80%;

[0081] The actual consumption of memory occupancy can be 80%;

[0082] The actual consumption of network traffic consumption can be 10 GB.

[0083] In this embodiment, the maximum set amount of CPU utilization can be 88%;

[0084] The maximum set amount of memory occupancy can be 90%;

[0085] The maximum set amount of network traffic consumption can be 15 GB.

[0086] In this embodiment, the coefficient-index-number correspondence table is a pre-established mapping table recording the correspondence between different redundancy coefficients, performance indicators and corresponding recommended container numbers. For example, the table may record that when the CPU redundancy coefficient is 1.1 and the memory redundancy coefficient is 1.05, the number of containers recommended based on the CPU performance indicator is 40, and the number of containers recommended based on the memory performance indicator is 35. This correspondence table is established according to a large amount of experimental data, historical experience and in-depth analysis of the performance of computer clusters, and is used to quickly determine the required number of containers under different combinations of performance indicators and redundancy coefficients.

[0087] In this embodiment, the required number of deployed containers = maximum number + initial number.

[0088] In this embodiment, the initial number is matched from a historical correspondence table, which contains historical arrival rates Ld and historical average processing times Lt under different combinations, as well as the corresponding initial number, such as 80.

[0089] The beneficial effects of the above technical solutions are: through the prediction of the bottleneck rate, the determination of the redundancy coefficient, and the determination of the number of containers, the performance and reliability of the entire system are improved.

[0090] The application provides an optimization method for resource integration based on PKS, determines a redundancy coefficient for each existing performance indicator, and includes:

[0091]

[0092] Wherein, R represents the redundancy coefficient corresponding to the existing performance indicator; represents the maximum value of the historical processing time of each task obtained from the historical processing log of the historical task set.

[0093] The beneficial effects of the above technical solutions are: starting from the arrival rate and the average processing time, the reasonable calculation of the redundancy coefficient is realized, and the basis for subsequent determination of the number of containers is provided.

[0094] The application provides an optimization method for resource integration based on PKS, establishes a storage resource pool according to the data type of the target device accessed, and includes:

[0095] Determine the data type of each data sample generated by the target device accessed under a specified working period, wherein the data type is a structured type and an unstructured type;

[0096] Analyze the data structure, field type and data size of each first sample under the structured type, and determine the theoretical required space of the corresponding first sample;

[0097] Analyze the data format, file size, and content characteristics of each second sample under the unstructured type, and determine the theoretical required space of the corresponding second sample;

[0098] Based on all the theoretical required spaces and the frequency of working on the corresponding target device and the data generation type under high-frequency work tasks, configure a sub-resource pool for the corresponding device;

[0099] Based on the sub-resource pools of all the target devices that have been accessed, obtain the storage resource pool.

[0100] In this embodiment, the target devices that have been accessed refer to various hardware devices that have successfully connected to the container cloud platform.

[0101] In this embodiment, the specified work cycle can be 1 hour.

[0102] In this embodiment, the data samples all contain data collected or generated by the device at a specific time or under specific conditions within the time period. For example, the temperature sensor collects temperature values every 1 minute within a 30-minute work cycle, and these 30 temperature values constitute a data sample.

[0103] In this embodiment, in a relational database, data is stored in the form of tables, which are composed of rows and columns, which is a common data structure. For the employee information table, the data structure defines the table name as "employees", contains columns such as "employee_id", "name", "age", "department", and the data type and constraint conditions of each column (such as "employee_id" is the primary key and cannot be empty).

[0104] Common field types include integer type (such as employee age), character type (such as employee name), floating point type (such as device running voltage value), date type (such as production time), etc.

[0105] The first sample in an employee information table, assuming that the "employee_id" field occupies 4 bytes (integer type), the "name" field occupies an average of 20 bytes (character type, assuming each Chinese character occupies 2 bytes, and the average name length is 10 characters), the "age" field occupies 2 bytes (short integer type), and the "department" field occupies 10 bytes (character type), then the data size of this first sample is approximately 4+20+2+10=36 bytes, and at this time, the theoretical required space of the first sample is 36 bytes+4 bytes required by other factors, totaling 20 bytes.

[0106] In this embodiment, common data formats include JPEG and PNG formats of images, MP3 and WAV formats of audio, MP4 and AVI formats of video, PDF and DOCX formats of documents, etc.

[0107] For image data, the content features can include color distribution, texture features, object shape, etc. of the image; for audio data, the content features can include frequency range, volume size, tone, etc. of the audio; for text data, the content features can include keywords, themes, sentiment orientation, etc.

[0108] In this embodiment, the working frequency is used to evaluate the frequency of use and the intensity of demand of the device on the storage resource.

[0109] In this embodiment, the type of data generated in the process of the target device performing the task with high frequency can be structured or unstructured.

[0110] In this embodiment, the storage resource pool is the integration of all sub-resource pools of the target device that have been accessed.

[0111] The beneficial effects of the above technical solutions are: by setting a space for samples under different data types to determine the size of the storage pool, the generated data can be more reasonably accommodated, and by the storage resource pool, the use of each sub-resource pool can be uniformly monitored and scheduled, when the space of a certain sub-resource pool is insufficient, resources can be allocated from other idle sub-resource pools, improving the utilization rate and flexibility of the entire storage system.

[0112] The present application provides an optimization method for resource integration based on PKS, configuring a sub-resource pool to a corresponding device, comprising:

[0113] ;

[0114] wherein, represents an optimization function; represents the theoretical required space of the i1th first sample; represents the theoretical required space of the i2th second sample; represents the size of the sub-resource pool configured to the corresponding device; represents the number of historical actual working tasks of the corresponding target device in a specified working period; represents the number of saturated working tasks of the corresponding target device in a specified working period; n1 represents the number of first samples; n2 represents the number of second samples; represents the maximum value of all ; represents the maximum value of all .

[0115] The beneficial effect of the above technical solution is that the size of the sub-resource pool is calculated by determining the number of the first sample and the second sample, and combining the optimization function.

[0116] This invention provides an optimization method for resource integration based on PKS, which provides real-time statistics on the operation of each application and the usage of each storage resource pool based on the container cloud platform, including:

[0117] Collect the self-information and running status of each application. At the same time, collect business indicator data based on the settings of each application. The working status is related to the self-information, running status and business indicator data.

[0118] Determine the current usage of the storage resource pool based on the operational status of the corresponding application, and predict future usage based on the operational status and the computing resources invoked.

[0119] In this embodiment, the self-information includes basic attributes such as the application's name, version number, developer information, and function description.

[0120] In this embodiment, common operating states include running, paused, stopped, waiting for resources, etc.

[0121] In this embodiment, setting up tracking points involves inserting specific code snippets into the application's code to collect business-related data. This data reflects user behavior and the execution of business processes during application use. For example, an image processing application might show it's processing a large number of high-resolution images, indicating it's busy, and business metrics suggest a heavy image processing workload. Monitoring the storage resource pool reveals that significant storage space is currently occupied, used to store pending and processed image data. This analysis clarifies the current storage resource pool usage by applications under different workloads, providing a basis for resource allocation and management.

[0122] For example, a big data analytics application is currently performing large-scale data analysis tasks, consuming significant computing resources (such as multiple CPU cores and ample memory). By analyzing its historical data and business trends, if future business volume is expected to continue growing while the application's operating pattern remains unchanged, it can be predicted that it will consume even more computing resources and generate more data, thus increasing the demand for storage resources. This kind of prediction helps in advance planning for storage resources and avoiding future resource shortages.

[0123] The beneficial effects of the above technical solution are as follows: by comprehensively collecting application information, it is possible to accurately understand the storage and computing resource requirements of each application; the collection and analysis of business indicator data helps to discover problems and potential optimization points in business processes, providing rich data support for management and helping to make scientific and reasonable decisions; and by monitoring the operation of applications in real time and analyzing the usage of storage resource pools, potential system problems can be discovered in a timely manner.

[0124] This invention provides a PKS-based resource integration optimization device, such as... Figure 2 As shown, it includes:

[0125] The platform building module is used to determine the number of containers to be deployed and the resource allocation strategy based on the scale and performance requirements of the computer cluster, and to configure the virtual network according to the network requirements of the training system, thereby building a container cloud platform.

[0126] The resource pool creation module is used to connect different types of target devices to the container cloud platform and create storage resource pools based on the data types of the connected target devices.

[0127] The resource allocation module is used to allocate corresponding computing resources to the application based on the container cloud platform according to the performance requirements of the application on the target device, wherein the computing resources of each application are distributed across multiple computing nodes.

[0128] The request analysis module is used to configure the load balancer and set the load balancing algorithm on the container cloud platform. When it receives an operation request from a user on the target device, it distributes the operation request to a matching computing node for request analysis.

[0129] The visualization module is used to statistically analyze the working status of each application based on the container cloud platform and the usage of each storage resource pool in real time, and to display the data visually.

[0130] The beneficial effects of the above technical solution are: by building a container cloud platform, establishing a storage resource pool, allocating computing resources, and configuring a load balancer, resource integration is achieved, effectively ensuring the efficient operation and stable functioning of the training system.

[0131] The present invention provides a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the optimization method for resource integration based on PKS as described in any one of the claims.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An optimization method for resource integration based on PKS, characterized in that, include: Step 1: Based on the scale and performance requirements of the computer cluster, determine the number of containers to be deployed and the resource allocation strategy, and configure the virtual network according to the network requirements of the training system to build a container cloud platform; Determining the required number of containers to be deployed includes: determining the bottleneck rate of the computer cluster under each existing performance metric based on the historical operating data of the computer cluster within a historical period; ;in, This indicates the bottleneck rate under the corresponding existing performance metrics; This indicates that the corresponding existing performance indicators were met within the historical period. The number of moments; This indicates the total number of moments involved in the historical period; This represents the maximum set value of the corresponding existing performance index at time i. This represents the actual consumption at time i, corresponding to the existing performance metric. This indicates that a threshold is set for the ratio of the existing performance metric at time i. This indicates all under the corresponding existing performance metrics. The variance; The computer cluster acquires a set of historical tasks within a historical time period and performs simulation on the set of historical tasks. A queuing theory mathematical model is used to determine the simulated arrival rate Ud and the simulated average processing time Ut of the set of historical tasks during the simulation. Based on the historical processing logs of the computer cluster for the historical task set, the historical arrival rate Ld and the historical average processing time Lt are obtained. Combined with the simulated arrival rate Ud, the simulated average processing time Ut, and the bottleneck rate, the redundancy coefficient for each existing performance indicator is determined. The number of containers is obtained by matching the coefficient-metric-quantity lookup table for each existing performance metric, filtering the maximum number from all container numbers, and combining it with the initial number obtained based on the historical arrival rate Ld and the historical average processing time Lt to obtain the required number of containers to be deployed. Step 2: Connect different types of target devices to the container cloud platform and establish storage resource pools based on the data types of the connected target devices; Step 3: Based on the performance requirements of the application on the target device, allocate corresponding computing resources to the application using the container cloud platform, wherein the computing resources of each application are distributed across multiple computing nodes; Step 4: Configure a load balancer on the container cloud platform and set the load balancing algorithm. When a user's operation request on the target device is received, the operation request is assigned to a matching computing node for request analysis. Step 5: Real-time statistics on the operation of each application based on the container cloud platform and the usage of each storage resource pool are presented visually.

2. The optimization method for resource integration based on PKS according to claim 1, characterized in that, Determine the redundancy factor for each existing performance metric, including: Where R represents the redundancy coefficient corresponding to the existing performance index; This represents the maximum historical processing time for each task obtained from the historical processing logs of the historical task set.

3. The optimization method for resource integration based on PKS according to claim 1, characterized in that, The existing performance metrics include: CPU utilization, memory usage, and network traffic consumption.

4. The optimization method for resource integration based on PKS according to claim 1, characterized in that, Establish a storage resource pool based on the data types of the target devices that have been connected, including: Determine the data type of each data sample generated by the connected target device in a specified working cycle, wherein the data type is a structured type and an unstructured type; Analyze the data structure, field types, and data volume of each first sample under the structured type to determine the theoretical space required for the corresponding first sample; Analyze the data format, file size, and content characteristics of each second sample under unstructured data types to determine the theoretical space required for the corresponding second sample; Based on the space required by all theories, the frequency of operation of the corresponding connected target devices, and the data generation type under high-frequency tasks, sub-resource pools are configured for the corresponding devices. Based on the sub-resource pools of all connected target devices, a storage resource pool is obtained.

5. The optimization method for resource integration based on PKS according to claim 4, characterized in that, Configure sub-resource pools for the corresponding devices, including: ； ； in, Represents the optimization function; This represents the theoretical space required for the i1th first sample; This represents the theoretical space required for the i2th second sample; This indicates the size of the sub-resource pool configured for the corresponding device; This indicates the number of historical actual work tasks performed by the corresponding target device within a specified work cycle. This represents the number of saturated workloads of the target device under a specified work cycle; n1 represents the number of the first sample; n2 represents the number of the second sample. Indicates all The maximum value in; Indicates all The maximum value in.

6. The optimization method for resource integration based on PKS according to claim 1, characterized in that, Real-time statistics are provided on the operational status of each application based on the container cloud platform and the usage of each storage resource pool, including: Collect the self-information and running status of each application. At the same time, collect business indicator data based on the settings of each application. The working status is related to the self-information, running status and business indicator data. Determine the current usage of the storage resource pool based on the operational status of the corresponding application, and predict future usage based on the operational status and the computing resources invoked.

7. A resource integration optimization device based on PKS, characterized in that, include: The platform building module is used to determine the number of containers to be deployed and the resource allocation strategy based on the scale and performance requirements of the computer cluster, and to configure the virtual network according to the network requirements of the training system, thereby building a container cloud platform. Determining the required number of containers to be deployed includes: determining the bottleneck rate of the computer cluster under each existing performance metric based on the historical operating data of the computer cluster within a historical period; ;in, This indicates the bottleneck rate under the corresponding existing performance metrics; This indicates that the corresponding existing performance indicators were met within the historical period. The number of moments; This indicates the total number of moments involved in the historical period; This represents the maximum set value of the corresponding existing performance index at time i. This represents the actual consumption at time i, corresponding to the existing performance metric. This indicates that a threshold is set for the ratio of the existing performance metric at time i. This indicates all under the corresponding existing performance metrics. The variance of the data is calculated; the historical task set of the computer cluster within a historical period is obtained, and the historical task set is simulated. A queuing theory mathematical model is used to determine the simulated arrival rate Ud and the simulated average processing time Ut of the historical task set during the simulation process; based on the historical processing log of the computer cluster for the historical task set, the historical arrival rate Ld and the historical average processing time Lt are obtained, and combined with the simulated arrival rate Ud, the simulated average processing time Ut, and the bottleneck rate, the redundancy coefficient for each existing performance indicator is determined; the number of containers based on each existing performance indicator is obtained by matching from the coefficient-indicator-quantity lookup table, and the maximum number is selected from all container numbers. Combined with the initial number obtained based on the historical arrival rate Ld and the historical average processing time Lt, the required number of containers to be deployed is obtained. The resource pool creation module is used to connect different types of target devices to the container cloud platform and create storage resource pools based on the data types of the connected target devices. The resource allocation module is used to allocate corresponding computing resources to the application based on the container cloud platform according to the performance requirements of the application on the target device, wherein the computing resources of each application are distributed across multiple computing nodes. The request analysis module is used to configure the load balancer and set the load balancing algorithm on the container cloud platform. When it receives an operation request from a user on the target device, it distributes the operation request to a matching computing node for request analysis. The visualization module is used to statistically analyze the working status of each application based on the container cloud platform and the usage of each storage resource pool in real time, and to display the data visually.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the PKS-based resource integration optimization method according to any one of claims 1 to 6.

Images