PKS-based resource integration optimization method and apparatus, and storage medium

Abstract

The invention provides a PKS-based resource integration optimization method and device and a storage medium, and relates to the technical field of resource integration, and the method comprises the steps: determining the number of containers needing to be deployed and a resource distribution strategy according to the scale and performance requirements of a computer cluster, and configuring a virtual network according to the network requirements of a training system, the method comprises the following steps: establishing a container cloud platform, establishing a storage resource pool according to a data type of an accessed target device, allocating corresponding computing resources to an application program based on the container cloud platform, configuring a load balancer and setting a load balancing algorithm on the container cloud platform, and when an operation request of a user on the target device is received, sending the operation request to the target device; the operation request is allocated to the matched computing node for request analysis, the working condition of each application program based on the container cloud platform and the use condition of each storage resource pool are counted in real time, and visual display is performed, so that efficient operation and stable function exertion of the training system are effectively ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of resource integration, and particularly to an optimization method, device, and storage medium for resource integration based on PKS. Background Art

[0002] PKS refers to Pivotal Container Service, which is a container orchestration platform based on Kubernetes. Its goal is to simplify the deployment, management, and scaling of containers, and provide a highly available and elastic containerized solution.

[0003] In today's training systems, many severe technical challenges are faced. On the one hand, the information data sources of training systems are extremely extensive, covering a variety of different data sources, with a large number of data types, including both structured data and a large amount of unstructured data, and the data quantity is huge, and the content styles are rich and diverse. On the other hand, the current data storage management is in a scattered state, which makes it difficult to effectively query and access data. During the decision-making process of the management level, accurate and effective decision-making data support cannot be obtained in a timely manner, which greatly affects the scientificity and timeliness of decision-making. In addition, the shared service requirements in different scenarios show diverse characteristics, and the technical systems are also different, which results in a serious lack of interoperability. Between shared services, resource scheduling is extremely difficult, and at the same time, there is also the problem of excessive resource consumption. These problems seriously restrict the efficient operation and function exertion of training systems.

[0004] Therefore, the present invention proposes an optimization method, device, and storage medium for resource integration based on PKS. Summary of the Invention

[0005] The present invention provides an optimization method, device, and storage medium for resource integration based on PKS, which are used to achieve resource integration by building a container cloud platform, establishing a storage resource pool, allocating computing resources, configuring a load balancer, etc., and effectively ensure the efficient operation and stable function exertion of training systems.

[0006] The present invention provides an optimization method for resource integration based on PKS, including: Step 1: Determine the number of containers to be deployed and the resource allocation strategy according to the scale and performance requirements of the computer cluster, and configure a virtual network according to the network requirements of the training system to build a container cloud platform; Step 2: Connect different types of target devices to the container cloud platform, and establish a storage resource pool according to the data types of the connected target devices; Step 3: According to the performance requirements of the applications on the target device, allocate corresponding computing resources for the applications based on the container cloud platform, where the computing resources of each application are distributed on multiple computing nodes; Step 4: Configure a load balancer on the container cloud platform, set a load balancing algorithm, and when an operation request from a user on the target device is received, allocate the operation request to a matching computing node for request analysis; Step 5: Real-time statistics on the working conditions of each application based on the container cloud platform and the usage of each storage resource pool, and perform visual display.

[0007] Preferably, determining the number of containers to be deployed includes: According to the historical operation data of the computer cluster in the historical period, determine the bottleneck rate of the computer cluster under each existing performance metric; ; where represents the bottleneck rate under the corresponding existing performance metric; represents the number of moments when the corresponding existing performance metric meets in the historical period; represents the total number of moments involved in the historical period; represents the maximum set amount of the corresponding existing performance metric at the i-th moment; represents the actual consumption of the corresponding existing performance metric at the i-th moment; represents the ratio setting threshold of the corresponding existing performance metric at the i-th moment; represents all under the corresponding existing performance metric; Obtain the historical task set of the computer cluster in the historical period, perform simulation on the historical task set, and use mathematical models such as queuing theory to determine the simulated arrival rate Ud and the simulated average processing time Ut of the historical task set during the simulation process; According to the historical processing logs of the historical task set of the computer cluster, obtain the historical arrival rate Ld and the historical average processing time Lt, and combine the simulated arrival rate Ud, the simulated average processing time Ut, and the bottleneck rate to determine the redundancy coefficient for each existing performance metric; Match and obtain the number of containers based on each existing performance metric from the coefficient-metric-quantity comparison table, select the maximum number from all the container numbers, and combine with the initial number obtained based on the historical arrival rate Ld and the historical average processing time Lt to obtain the number of containers to be deployed.

[0008] Preferably, determining the redundancy coefficient for each existing performance metric includes:

[0009] Among them, R represents the redundancy coefficient corresponding to the existing performance index; represents the maximum value among the historical processing times of each task obtained from the historical processing logs of the historical task set.

[0010] Preferably, the existing performance indexes include: CPU utilization rate, memory occupancy rate, and network traffic consumption.

[0011] Preferably, a storage resource pool is established according to the data types of the connected target devices, including: Determine the data types of each data sample generated by the connected target device in a specified working cycle, where the data types are structured types and unstructured types; Analyze the data structure, field type, and data volume size of each first sample under the structured type, and determine the theoretically required space corresponding to the first sample; Analyze the data format, file size, and content characteristics of each second sample under the unstructured type, and determine the theoretically required space corresponding to the second sample; Based on all the theoretically required spaces, the working frequency of the corresponding connected target device, and the data generation type under high-frequency working tasks, configure a sub-resource pool for the corresponding device; Based on the sub-resource pools of all the connected target devices, obtain the storage resource pool.

[0012] Preferably, configuring a sub-resource pool for the corresponding device includes: ; Among them, represents the optimization function; represents the theoretically required space of the i1-th first sample; represents the theoretically required space of the i2-th second sample; represents the size of the sub-resource pool configured for the corresponding device; represents the number of historical actual working tasks of the corresponding target device in a specified working cycle; represents the number of saturated working tasks of the corresponding target device in a specified working cycle; n1 represents the number of first samples; n2 represents the number of second samples; represents all the maximum value among; represents all the maximum value among.

[0013] Preferably, the working conditions of each application program based on the container cloud platform and the usage conditions of each storage resource pool are statistically analyzed in real time, including: Collect the self-information and running status of each application. Meanwhile, based on the settings of each application, collect business metric data by setting breakpoints. Among them, the working conditions are related to the self-information, running status, and business metric data. Determine the current usage of the storage resource pool based on the working conditions of the corresponding application, and predict the future usage according to the working conditions and combined with the computing resources called.

[0014] The present invention provides an optimization device for resource integration based on PKS, including: A platform construction module, which is used to determine the number of containers to be deployed and the resource allocation strategy according to the scale and performance requirements of the computer cluster, and configure the virtual network according to the network requirements of the training system to construct a container cloud platform. A resource pool establishment module, which is used to connect different types of target devices to the container cloud platform and establish a storage resource pool according to the data types of the connected target devices. A resource allocation module, which is used to allocate corresponding computing resources for the application based on the container cloud platform according to the performance requirements of the application on the target device. Among them, the computing resources of each application are distributed on multiple computing nodes. A request analysis module, which is used to configure a load balancer on the container cloud platform, set a load balancing algorithm, and when receiving an operation request from a user on the target device, allocate the operation request to a matching computing node for request analysis. A visualization module, which is used to statistically analyze the working conditions of each application and the usage of each storage resource pool based on the container cloud platform in real time and perform visual display.

[0015] The present invention provides a storage medium on which a computer program is stored. The computer program, when executed by a processor, implements any one of the optimization methods for resource integration based on PKS.

[0016] Compared with the prior art, the beneficial effects of the present application are as follows: Through constructing a container cloud platform, establishing a storage resource pool, allocating computing resources, configuring a load balancer, etc., to achieve resource integration, effectively ensuring the efficient operation of the training system and the stable play of functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a flowchart of an optimization method for resource integration based on PKS provided by an embodiment of the present invention; Figure 2 It is a structural diagram of an optimization device for resource integration based on PKS provided by an embodiment of the present invention. Detailed implementation manners

[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] The present invention provides an optimization method for resource integration based on PKS, as Figure 1 shown, including: Step 1: Determine the number of containers to be deployed and the resource allocation strategy according to the scale and performance requirements of the computer cluster, and configure a virtual network according to the network requirements of the training system to construct a container cloud platform; Step 2: Connect different types of target devices to the container cloud platform, and establish a storage resource pool according to the data types of the connected target devices; Step 3: Allocate corresponding computing resources for the application program based on the container cloud platform according to the performance requirements of the application program on the target device, where the computing resources of each application program are distributed on multiple computing nodes; Step 4: Configure a load balancer on the container cloud platform, set a load balancing algorithm, and when receiving an operation request from a user on a target device, allocate the operation request to a matching computing node for request analysis; Step 5: Real-time statistics on the working conditions of each application program based on the container cloud platform and the usage conditions of each storage resource pool, and perform visual display.

[0021] In this embodiment, the scale of the computer cluster refers to the comprehensive situation of the number of physical servers constituting the computer cluster, the hardware configuration of the servers (such as the total number of CPU cores, the total memory capacity, the total disk storage capacity, etc.), and the specifications and quantities of network devices (such as switches, routers, etc.). For example, a computer cluster consists of 10 servers equipped with 8-core CPUs, 64GB of memory, and 1TB of hard disks, and is connected through a gigabit network switch. This is a description of the scale of the cluster. The acquisition method is through the parameter statistics and summary of each physical device in the cluster.

[0022] In this embodiment, according to the functions and quality of service expected to be achieved by the training system, index requirements are put forward for the computer cluster in terms of computing speed, data processing ability, response time, etc. For example, the training system is required to be able to simulate the actions of 1000 combat units simultaneously, and the response time of the simulation calculation does not exceed 1 second. This is the specific requirement for performance. It is usually determined by the design team of the training system according to business requirements and industry standards.

[0023] In this embodiment, the resource allocation strategy determines the rules for allocating resources such as the CPU, memory, disk I / O, and network bandwidth of the computer cluster to different containers or tasks. For example, based on task priorities, 70% of the CPU time slices are allocated to high-priority real-time command and control tasks, and the remaining 30% is shared by low-priority data statistics tasks. When formulating the strategy, factors such as task type, resource demand characteristics, and overall system performance optimization need to be comprehensively considered.

[0024] The network requirements of the training system include requirements for network bandwidth, network latency, network reliability, network topology structure, etc. For example, real-time video transmission requires a network bandwidth of not less than 100 Mbps, a network latency of less than 50 ms, and redundant links to ensure reliability. It is obtained through the analysis of network traffic and real-time performance of various services (such as simulation, real-time communication, etc.) in the training system.

[0025] In this embodiment, a virtual network is configured on the container cloud platform, and network bandwidth limits, access control policies, etc. are set. For example, a virtual network is created, and independent subnets are divided for different types of tasks. For example, simulation tasks are in subnet A, and data transmission tasks are in subnet B. Special SDN controller software (such as OpenDaylight, ONOS, etc.) is used for configuration and creation.

[0026] In this embodiment, various hardware devices are connected to the container cloud platform, and they have different functions and data generation / transmission characteristics. For example, sensor devices (temperature sensors, pressure sensors, etc.) are used to collect environmental data, and intelligent terminal devices (smartphones, tablets) are used for user interaction and data display, etc. They are classified and identified through the functional uses of the devices, data interface types, etc.

[0027] In this embodiment, the time series data generated by sensor devices is usually structured numerical data; the image data captured by intelligent terminal devices belongs to unstructured data.

[0028] In this embodiment, the storage resource pool is to provide a space for storing relevant resources for the corresponding devices, which is also convenient for ensuring the normal operation of the devices. The application programs run on the target devices.

[0029] In this embodiment, the computing node is a physical server or a virtual server that actually executes computing tasks in a computer cluster.

[0030] In this embodiment, the load balancer can distribute the access requests of users to the training system to different computing nodes according to the configured algorithm. A hardware load balancing device (such as F5, A10, etc.) can be used, or it can also be implemented through a software load balancer (such as Nginx, HAProxy, etc.). For example, the weighted round-robin algorithm is adopted, and weights are set according to the CPU performance and memory size of the computing nodes, and more requests are allocated to the nodes with higher performance.

[0031] In this embodiment, the operation request is a request generated when the user operates the training system on the target device, such as clicking a button, inputting an instruction, querying data, etc.

[0032] In this embodiment, the computing node receives a request to query battlefield situation data. By analyzing the request parameters, it determines which data to query from the database and performs corresponding data retrieval and processing. The request analysis is completed by the application logic module running on the computing node.

[0033] In this embodiment, information such as the running status, resource usage, and task execution progress of the application on the container cloud platform. For example, the CPU utilization rate of a data processing application is 60%, the memory occupancy is 8GB, and it is currently processing the 500th data block. It is obtained by collecting the performance metrics and status information of the application during runtime through the monitoring tools (such as Prometheus, Grafana, etc.) of the container cloud platform, which is the working condition.

[0034] In this embodiment, the occupancy of the storage capacity in the storage resource pool, the frequency of read and write operations, the performance metrics of the storage device, etc. For example, the total capacity of the storage resource pool is 10TB, 6TB has been used, and the current read and write speeds are 100MB / s and 50MB / s respectively, which is the usage situation.

[0035] In this embodiment, the visual display is to display the data such as the working condition of the application and the usage situation of the storage resource pool collected in an intuitive graph, chart, etc., which is convenient for administrators and users to view and analyze. For example, Grafana is used to display the CPU utilization rate of the application as a line chart and the usage situation of the storage resource pool as a bar chart. It is achieved by connecting a data visualization tool (such as Grafana, Kibana, etc.) to the data collection source (such as monitoring tools, storage management software, etc.) and configuring the corresponding data display panel.

[0036] The beneficial effects of the above technical solution are as follows: By constructing a container cloud platform, establishing a storage resource pool, allocating computing resources, configuring a load balancer, etc., resource integration is achieved, effectively ensuring the efficient operation of the training system and the stable performance of its functions.

[0037] The present invention provides an optimization method for resource integration based on PKS, which determines the number of containers to be deployed, including: Determine the bottleneck rate of the computer cluster under each existing performance metric according to the historical operation data of the computer cluster in the historical period; ; Wherein, represents the bottleneck rate under the corresponding existing performance metric; represents the number of moments when the corresponding existing performance metric meets in the historical period; represents the total number of moments involved in the historical period; represents the maximum set amount of the corresponding existing performance metric at the i-th moment; represents the actual consumption amount of the corresponding existing performance metric at the i-th moment; represents the ratio setting threshold of the corresponding existing performance metric at the i-th moment; represents all under the corresponding existing performance metric; Obtain the historical task set of the computer cluster in the historical period, and conduct simulation on the historical task set. Use mathematical models such as queuing theory to determine the simulated arrival rate Ud and the simulated average processing time Ut of the historical task set in the simulation process; According to the historical processing log of the historical task set of the computer cluster, obtain the historical arrival rate Ld and the historical average processing time Lt, and combine the simulated arrival rate Ud, the simulated average processing time Ut and the bottleneck rate to determine the redundancy coefficient for each existing performance metric; Match and obtain the number of containers based on each existing performance metric from the coefficient-index-number comparison table, screen the maximum number from all the number of containers, and combine with the initial number obtained based on the historical arrival rate Ld and the historical average processing time Lt to obtain the number of containers to be deployed.

[0038] Preferably, the existing performance metrics include: CPU utilization rate, memory occupancy rate, and network traffic consumption.

[0039] In this embodiment, the historical period can be the past half hour, and records are taken every 1 minute. At this time, T0 is 30.

[0040] In this embodiment, the actual consumption amount for the CPU utilization rate can be 80%; The actual consumption rate for memory occupancy can be 80%; The actual consumption rate for network traffic can be 10 GB.

[0041] In this embodiment, the maximum set value for CPU utilization rate can be 88%; The maximum set value for memory occupancy can be 90%; The maximum set value for network traffic consumption can be 15 GB.

[0042] In this embodiment, the coefficient - metric - quantity correspondence table is a pre - established mapping relationship table that records the correspondence between different redundancy coefficients, performance metrics, and the corresponding recommended container quantities. For example, the table may record that when the CPU redundancy coefficient is 1.1 and the memory redundancy coefficient is 1.05, the recommended container quantity based on the CPU performance metric is 40, and the recommended container quantity based on the memory performance metric is 35. This correspondence table is established based on a large amount of experimental data, historical experience, and in - depth analysis of the performance of computer clusters, and is used to quickly determine the required container quantity under different combinations of performance metrics and redundancy coefficients.

[0043] In this embodiment, the required number of containers to be deployed = maximum quantity + initial quantity.

[0044] In this embodiment, the initial quantity is obtained by matching from the historical correspondence table, which contains the historical arrival rate Ld, the historical average processing time Lt, and the corresponding initial quantity under different combinations, for example, it is 80.

[0045] The beneficial effects of the above - mentioned technical solution are: By predicting the bottleneck rate, determining the redundancy coefficient, and determining the number of containers, the performance and reliability of the entire system are improved.

[0046] The present invention provides an optimization method for resource integration based on PKS, which determines the redundancy coefficient for each existing performance metric, including:

[0047] where R represents the redundancy coefficient corresponding to the existing performance metric; represents the maximum value among the historical processing times of each task obtained from the historical processing logs of the historical task set.

[0048] The beneficial effects of the above - mentioned technical solution are: Starting from the arrival rate and the average processing time, a reasonable calculation of the redundancy coefficient is realized, providing a basis for subsequent determination of the number of containers.

[0049] The present invention provides an optimization method for resource integration based on PKS, which establishes a storage resource pool according to the data types of the connected target devices, including: Determine the data type of each data sample generated by the connected target device in a specified working cycle, where the data type is a structured type and an unstructured type; Analyze the data structure, field type, and data volume size of each first sample under the structured type, and determine the theoretical required space for the corresponding first sample; Analyze the data format, file size, and content characteristics of each second sample under the unstructured type, and determine the theoretical required space for the corresponding second sample; Based on all the theoretical required spaces, the working frequency of the corresponding connected target device, and the data generation type under high-frequency working tasks, configure a sub-resource pool for the corresponding device; Based on the sub-resource pools of all the connected target devices, obtain a storage resource pool.

[0050] In this embodiment, the connected target device refers to various hardware devices that have successfully connected to the container cloud platform.

[0051] In this embodiment, the specified working cycle can be 1 hour.

[0052] In this embodiment, the data samples all contain the data collected or generated by the device at a specific moment or under specific conditions during this time period. For example, in a 30-minute working cycle of the above temperature sensor, the temperature value is collected once every 1 minute, then these 30 temperature values constitute a data sample.

[0053] In this embodiment, in a relational database, data is stored in the form of tables, and tables are composed of rows and columns, which is a common data structure. For an employee information table, its data structure defines the table name as "employees", contains columns such as "employee_id", "name", "age", "department", etc., as well as the data type and constraints of each column (such as "employee_id" being the primary key and not being null).

[0054] Common field types include integer type (such as employee age), character type (such as employee name), floating-point type (such as the voltage value of device operation), date type (such as production time), etc.

[0055] A first sample in an employee information table. Assume that the "employee_id" field occupies 4 bytes (integer type), the "name" field occupies an average of 20 bytes (character type, assuming each Chinese character occupies 2 bytes and the average name length is 10 characters), the "age" field occupies 2 bytes (short integer type), and the "department" field occupies 10 bytes (character type). Then the data volume of this first sample is approximately 4 + 20 + 2 + 10 = 36 bytes. At this time, the theoretically required space for the first sample is 36 bytes + 4 bytes required due to other factors, totaling 40 bytes.

[0056] In this embodiment, common data formats include JPEG and PNG formats for images, MP3 and WAV formats for audio, MP4 and AVI formats for video, PDF and DOCX formats for documents, etc.

[0057] For image data, content features may include the color distribution of the image, texture features, object shapes, etc.; for audio data, content features may include the frequency range, volume, timbre, etc. of the audio; for text data, content features may include keywords, themes, sentiment tendencies, etc.

[0058] In this embodiment, the frequency of work is used to evaluate the usage frequency and demand intensity of the device for storage resources.

[0059] In this embodiment, the type of data generated during the process of tasks with a relatively high execution frequency on the target device may be a structured type or an unstructured type.

[0060] In this embodiment, the storage resource pool is to integrate all the sub - resource pools that have been connected to the target device.

[0061] The beneficial effects of the above - mentioned technical solution are as follows: By setting spaces for samples under different data types to determine the size of the storage pool, it can reasonably accommodate the generated data. And through the storage resource pool, it is possible to uniformly monitor and schedule the usage of each sub - resource pool. When the space of a certain sub - resource pool is insufficient, resources can be allocated from other idle sub - resource pools, improving the utilization rate and flexibility of the entire storage system.

[0062] The present invention provides an optimization method for resource integration based on PKS, which configures sub - resource pools for corresponding devices, including: ; Among them, represents the optimization function; represents the theoretically required space for the i1 - th first sample; represents the theoretically required space for the i2 - th second sample; represents the size of the sub - resource pool configured for the corresponding device; represents the number of historical actual work tasks of the corresponding target device in the specified working cycle; represents the number of saturated work tasks of the corresponding target device in the specified working cycle; n1 represents the number of the first sample; n2 represents the number of the second sample; represents all the maximum value in; represents all the maximum value in.

[0063] The beneficial effects of the above technical solution are: by determining the number of the first sample and the second sample, and combining with the optimization function to comprehensively calculate the size of the sub-resource pool.

[0064] The present invention provides an optimization method for resource integration based on PKS, which real-time statistically analyzes the working conditions of each application program and the usage conditions of each storage resource pool based on the container cloud platform, including: Collect the self-information and running status of each application program. At the same time, based on the set data points inserted in each application program, collect business metric data, where the working conditions are related to the self-information, running status, and business metric data; Determine the current usage conditions of the storage resource pool by the working conditions of the corresponding application program, and predict the future usage conditions according to the working conditions and combined with the invoked computing resources.

[0065] In this embodiment, the self-information includes basic attributes such as the name, version number, developer information, and function description of the application program.

[0066] In this embodiment, common running statuses include running, paused, stopped, waiting for resources, etc.

[0067] In this embodiment, setting data points is to insert specific code snippets into the code of the application program for collecting business-related data. These data can reflect the behaviors of users during the use of the application program and the execution of the business process. For example, for an image processing application program, its working conditions show that it is processing a large number of high-definition pictures. From the running status, it can be known that it is in a busy state, and the business metric data indicates that the amount of image processing tasks is large. Through the monitoring of the storage resource pool, it is found that the storage space in the current storage resource pool is occupied in large quantities for storing the to-be-processed and processed picture data. Through this analysis, it is possible to clarify the current occupancy of the storage resource pool by application programs with different working conditions, providing a basis for resource allocation and management.

[0068] For example, a big data analysis application is currently working on a large-scale data analysis task, which calls a large amount of computing resources (such as multiple CPU cores and a large amount of memory). By analyzing its historical work data and business development trends, if it is predicted that the future business volume will continue to grow and the working mode of this application remains unchanged, then it can be predicted that it will call more computing resources in the future, and at the same time, more data will be generated, resulting in a corresponding increase in the demand for the storage resource pool. This prediction helps to plan the storage resources in advance and avoid the situation of insufficient resources in the future. The beneficial effects of the above technical solution are as follows: By comprehensively collecting information about the application, it is possible to accurately understand the demand for storage resources and computing resources of each application. The collection and analysis of business metric data help to discover problems and potential optimization points in the business process, providing rich data support for the management and facilitating the formulation of scientific and reasonable decisions. By real-time monitoring the working conditions of the application and analyzing the usage of the storage resource pool, potential system problems can be detected in a timely manner.

[0069] The present invention provides an optimization device for resource integration based on PKS, as Figure 2 shown, including: A platform construction module, which is used to determine the number of containers to be deployed and the resource allocation strategy according to the scale and performance requirements of the computer cluster, and configure a virtual network according to the network requirements of the training system to construct a container cloud platform; A resource pool establishment module, which is used to connect different types of target devices to the container cloud platform and establish a storage resource pool according to the data types of the connected target devices; A resource allocation module, which is used to allocate corresponding computing resources for the application based on the container cloud platform according to the performance requirements of the application on the target device, where the computing resources of each application are distributed on multiple computing nodes; A request analysis module, which is used to configure a load balancer on the container cloud platform, set a load balancing algorithm, and when receiving an operation request from a user on the target device, allocate the operation request to a matching computing node for request analysis; A visualization module, which is used to statistically analyze the working conditions of each application based on the container cloud platform and the usage of each storage resource pool in real time and perform visual display.

[0070] The beneficial effects of the above technical solution are as follows: By constructing a container cloud platform, establishing a storage resource pool, allocating computing resources, configuring a load balancer, etc., resource integration is realized, effectively ensuring the efficient operation of the training system and the stable performance of its functions.

[0071] The present invention provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the optimization method for resource integration based on PKS described in any one of the above is implemented.

[0072] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0073] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An optimization method for resource integration based on PKS, characterized in that, Including: Step 1: According to the scale and performance requirements of the computer cluster, determine the number of containers to be deployed and the resource allocation strategy, and configure the virtual network according to the network requirements of the training system to build a container cloud platform; Step 2: Connect different types of target devices to the container cloud platform and establish a storage resource pool according to the data types of the connected target devices; Step 3: According to the performance requirements of the application programs on the target devices, allocate corresponding computing resources for the application programs based on the container cloud platform, where the computing resources of each application program are distributed on multiple computing nodes; Step 4: Configure a load balancer on the container cloud platform, set the load balancing algorithm, and when receiving an operation request from a user on a target device, allocate the operation request to a matching computing node for request analysis; Step 5: Real-time statistics on the working conditions of each application program based on the container cloud platform and the usage conditions of each storage resource pool, and perform visual display.

2. The optimization method for resource integration based on PKS according to claim 1, wherein Determining the number of containers to be deployed includes: According to the historical operation data of the computer cluster in the historical period, determine the bottleneck rate of the computer cluster under each existing performance index; ； Among them, represents the bottleneck rate under the corresponding existing performance indicators; represents the number of moments when the corresponding existing performance indicators are satisfied within the historical period ; represents the total number of moments involved in the historical period; represents the maximum set amount of the corresponding existing performance indicator at the i-th moment; represents the actual consumption amount of the corresponding existing performance indicator at the i-th moment; represents the ratio setting threshold of the corresponding existing performance indicator at the i-th moment; represents all of the corresponding existing performance indicators variance; Obtain the historical task set of the computer cluster in the historical period, perform simulation on the historical task set, and use mathematical models such as queuing theory to determine the simulated arrival rate Ud and the simulated average processing time Ut of the historical task set in the simulation process; According to the historical processing logs of the historical task set of the computer cluster, obtain the historical arrival rate Ld and the historical average processing time Lt, and combine the simulated arrival rate Ud, the simulated average processing time Ut and the bottleneck rate to determine the redundancy coefficient for each existing performance index; Match and obtain the number of containers based on each existing performance index from the coefficient-index-number comparison table, screen the maximum number from all the container numbers, and combine with the initial number obtained based on the historical arrival rate Ld and the historical average processing time Lt to obtain the number of containers to be deployed.

3. The optimization method for resource integration based on PKS according to claim 2, characterized in that, Determining the redundancy coefficient for each existing performance index includes: wherein, R represents a redundancy coefficient corresponding to an existing performance index; represents the maximum value among the historical processing times of each task obtained from the historical processing logs of the historical task set.

4. The optimization method for resource integration based on PKS according to claim 2, wherein The existing performance indexes include: CPU utilization rate, memory occupancy rate, and network traffic consumption.

5. The optimization method for resource integration based on PKS according to claim 1, characterized in that Establishing a storage resource pool according to the data types of the connected target devices includes: Determine the data type of each data sample generated by the connected target device in the specified working cycle, where the data type is structured type and unstructured type; Analyze the data structure, field type, and data volume size of each first sample under the structured type, and determine the theoretically required space for the corresponding first sample; Analyze the data format, file size, and content characteristics of each second sample under the unstructured type, and determine the theoretically required space for the corresponding second sample; Based on all the theoretically required spaces, the working frequency of the corresponding connected target device, and the data generation type under the high-frequency working tasks, configure a sub-resource pool for the corresponding device; Based on the sub-resource pools of all the connected target devices, obtain the storage resource pool.

6. The optimization method for PKS-based resource integration according to claim 5, characterized in that Configuring a sub-resource pool for the corresponding device includes: ； Among them, represents the optimization function; represents the theoretically required space of the i1-th first sample; represents the theoretically required space of the i2-th second sample; represents the size of the sub-resource pool configured for the corresponding device; represents the number of historical actual work tasks of the corresponding target device in the specified working cycle; represents the number of saturated work tasks of the corresponding target device in the specified working cycle; n1 represents the number of first samples; n2 represents the number of second samples; represents all the maximum value in; represents all the maximum value in.

7. The optimization method for PKS-based resource integration according to claim 1, wherein Real-time statistics on the working conditions of each application and the usage of each storage resource pool based on the container cloud platform, including: Collect the self-information and running status of each application. At the same time, collect business metric data based on the set data points of each application. Among them, the working conditions are related to the self-information, running status, and business metric data; Determine the current usage of the storage resource pool by the working conditions of the corresponding application, and predict the future usage based on the working conditions and combined with the invoked computing resources.

8. An optimization device for resource integration based on PKS, characterized in that, Including: A platform construction module for determining the number of containers to be deployed and the resource allocation strategy according to the scale and performance requirements of the computer cluster, and configuring a virtual network according to the network requirements of the training system to construct a container cloud platform; A resource pool establishment module for connecting different types of target devices to the container cloud platform and establishing a storage resource pool according to the data types of the connected target devices; A resource allocation module for allocating corresponding computing resources for the application based on the container cloud platform according to the performance requirements of the application on the target device, where the computing resources of each application are distributed on multiple computing nodes; A request analysis module for configuring a load balancer on the container cloud platform, setting a load balancing algorithm, and when receiving an operation request from a user on the target device, allocating the operation request to a matching computing node for request analysis; A visualization module for real-time statistics on the working conditions of each application and the usage of each storage resource pool based on the container cloud platform and performing visual display.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the optimization method for PKS-based resource integration according to any one of claims 1 to 7.

Images