64 results about "Service level objective" patented technology

The invention relates to the technical field of computer deep learning, in particular to a deep learning task hybrid deployment method and system. The method comprises the steps that S1, a user submits a task through a Kubernetes native interface; s2, a task scheduling optimizer analyzes tasks according to resource requirements and service levels and allocates the tasks to proper nodes; s3, the SLO manager tracks real-time data of various performance indexes of the system and compares the real-time data with a preset service grade target; s4, the Koordlet component executes a specific resource allocation operation on the target node; and S5, the traffic safety monitor monitors the data streams among the nodes in real time. According to the method, a mixed deployment strategy for different types of resources is provided by analyzing the periodical rule of deep learning task resource use, dynamic resource sharing of online and offline tasks is achieved, and meanwhile it is ensured that the system can still run stably and efficiently under the high-load condition.

As described herein, various embodiments of the present invention improve computational efficiency of performing reliable persistent monitoring of a computer system. Persistent monitoring of the operations of a computer system has various operational reliability benefits for various computer systems and is an important part of service level objectives for highly maintenance-critical computer systems. However, doing the noted persistent monitoring operations in a reliable manner requires access to large amounts of labeled training data that are not always available for more customized computer systems with unique operational / behaviorial pattern signatures. In response, various embodiments of the present invention address the noted challenges by generating training data for an anomalous operational state detection machine learning model whose training data may be generated using a ground-truth validation criterion that is defined based at least in part on an inferred outlier score for a decomposed residual component of a given operational monitoring timeseries trend.

Various approaches to efficiently allocating and utilizing hardware resources in data centers while maintaining compliance with a service level objective (SLO) specified for a computational workload is translated into a hardware-level SLO to facilitate direct enforcement by the hardware processor, e.g., using a feedback control loop or model-based mapping of the hardware-level SLO to allocations of microarchitecture resources of the processor. In some embodiments, a computational model of the hardware behavior under resource contention is used to predict the application performance (e.g., as measured in terms of the hardware-level SLO) to be expected under certain contention scenarios. Scheduling of workloads among the compute nodes within the data center may be based on such predictions. In further embodiments, configurations of microservices are optimized to minimize hardware resources while meeting a specified performance goal.

Various systems and methods for implementing intent-based orchestration in heterogenous compute platforms are described herein. An orchestration system is configured to: receive, at the orchestration system, a workload request for a workload, the workload request including an intent-based service level objective (SLO); generate rules for resource allocation based on the workload request; generate a deployment plan using the rules for resource allocation and the intent-based SLO; deploy the workload using the deployment plan; monitor performance of the workload using real-time telemetry; and modify the rules for resource allocation and the deployment plan based on the real-time telemetry.

The invention discloses a server-free workflow dynamic resource configuration system based on resource decoupling, and relates to the field of cloud computing. Comprising a workflow sampling scheduling module, a configuration decoupling search module and a dynamic configuration module. The dynamic configuration module shunts the workload in combination with the input characteristic data, the scheduling workflow sampling scheduling module obtains the decoupled optimal resource configuration, and a mapping relation between the input characteristic and the decoupled resource configuration is established by training a random forest model; the workflow sampling scheduling module analyzes a workflow structure through input, generates a weighted directed acyclic graph and identifies a key path, preferentially searches for optimal configuration for the key path, and then iteratively optimizes a sub-path on the premise of not violating the consistency of the key path; the configuration decoupling search module allocates decoupled CPU / memory resources based on a critical path priority policy. According to the method, the cost is minimized while the service level target is met through an automatic searching method for decoupling the memory and the CPU.

Techniques and mechanisms are described for enabling a user to run heavy deep learning workloads on standard edge networks without off-loading computation to a cloud, leveraging the available edge computing resources, and efficiently partitioning and distributing a Deep Neural Network (DNN) over a network. The techniques enable the user to split a workload into multiple parts and process the workload on a set of smaller, less capable compute nodes in a distributed manner, without compromising on performance, and while meeting a Service Level Objective (SLO).

Disclosed herein are a method for GPU resource management using reinforcement learning and an apparatus for the same. The method performed by the apparatus includes deriving a Multi-Instance GPU (MIG) instance configuration that meets a Service Level Objective (SLO) condition and a request rate assigned to a workload by utilizing a pretrained reinforcement learning model and reorganizing MIG resources of a GPU device to correspond to the workload by transferring the MIG instance configuration to the GPU device.

A computer-implemented method for trace-driven call-graph-aware proactive coordinated autoscaling of component microservices in an application includes generating performance-resource elasticity models of endpoints of the component microservices of the application. Workload levels of the endpoint of the component microservices is predicted based on user traffic observed at a front end service. A trace-level performance of the application is predicted for different microservice replica scaling based on the performance-resource elasticity models at end points, the ends points on the trace call graph and the predicted workload levels. A microservice replica scaling is recommended for each of the component microservices to meet predefined trace-level user service level objectives.

A computer-implemented method for trace-driven dependency-set-aware proactive coordinated autoscaling of component microservices in an application includes generating performance-resource elasticity models at a trace-level for traces of the application using dependency set of microservices for each trace. The method predicts workload levels of each of the traces, and also predicts a trace-level performance of the application for different microservice replica scaling based on the dependency set of microservices for each trace, performance-resource elasticity models and the predicted workload levels. The method uses distributed computing to recommend a microservice replica scaling for each of the component microservices to meet one or more predefined trace-level user service level objectives.

Embodiments of the invention provide a model input request scheduling method and device, a storage medium and a program product. The method comprises the steps of receiving an input request of a preset language model; determining prediction time consumption of the input request in a pre-filling stage in a preset language model processing process; determining the latest execution time of the pre-filling stage of the input request according to the receiving time of the input request, the predicted consumed time of the pre-filling stage and the target first lexical element delay; and according to the latest execution time of the pre-filling stage, sending the input request to a service node of a preset language model for processing. According to the embodiment of the invention, the latest execution time of the pre-filling stage is determined based on the predicted time consumption of the pre-filling stage of the input request to schedule the input request, and the system throughput and the calculation load balance are optimized while the service level target is met, so that the resource utilization efficiency and the service quality guarantee capability of the service node of the preset language model can be improved.

Various systems and methods for implementing intent-based orchestration in heterogenous compute platforms are described herein. An orchestration system is configured to: receive, at the orchestration system, a workload request for a workload, the workload request including an intent-based service level objective (SLO); generate rules for resource allocation based on the workload request; generate a deployment plan using the rules for resource allocation and the intent-based SLO; deploy the workload using the deployment plan; monitor performance of the workload using real-time telemetry; and modify the rules for resource allocation and the deployment plan based on the real-time telemetry.

Implementing Service Level Agreement (SLA) requirements for batch processes. An enterprise workflow platform receives the SLA requirements via a batch administration form for scheduling the batch processes. The enterprise workflow platform persists the SLA requirements in a knowledge base. A batch monitoring tool is integrated with the enterprise workflow platform to retrieve the SLA requirements from the knowledge base and then measure performance of the schedule batch processes against the SLA requirements.

The invention discloses a multi-target execution plan optimization method, system and product oriented to intelligent application, and belongs to the technical field of intelligent application. A logic application diagram and context information are obtained, and the context information comprises a service level target and a load feature; the method comprises the steps that a logic application diagram, a service level target, load characteristics and a pre-constructed operator performance model are integrated to obtain a constraint optimization problem, a multi-target constraint optimization solver is used for solving the constraint optimization problem, an optimal physical execution plan is generated, and an execution engine is used for executing the optimal physical execution plan. According to the method, the multi-dimensional service target can be understood and adjusted in real time by obtaining the service level target set by the user, the hysteresis is avoided by generating the optimal physical execution plan before plan execution, the self-adaptive capability and the quality of the generated result are improved by using the pre-constructed operator performance model and the multi-target constraint optimization solver, and the method has the advantages of being high in practicability and the like. And the scientificity, the globality and the reproducibility of a scheduling decision are ensured.

This disclosure relates to methods, devices, and computer-readable media for use in developing data products. One such method comprises receiving an existing build of the data product, identifying a data product version associated with the existing build, receiving a user-specified modification for the data product, in response to a user input, automatically determining a compatibility result for the modification with the identified data product version, based on the existing build of the data product, and in response to the determined compatibility result being a negative compatibility result, triggering a failure event in relation to the identified data product version. The user specified modification may comprise one or more of: modification of a schema or addition modification or deletion of one or more tables, columns, service level objectives, service level indicators, constraints. Determination of the compatibility result may comprise determining a negative compatibility result by identifying that a table, column, service level objective, constraint or other object present in the existing build is removed by the modification.

The present invention relates to the technical field of computer deep learning, and particularly relates to a method and system for hybrid deployment of deep learning tasks. The method includes: S1. A user submits a task through the native interface of Kubernetes; S2. A task scheduling optimizer analyzes the task according to resource requirements and service levels and allocates it to a suitable node; S3. An SLO manager tracks the real-time data of various performance indicators of the system and compares the real-time data with the pre-set service level objectives; S4. The Koordlet component performs specific resource allocation operations on the target node; S5. A traffic security monitor monitors the data flow between nodes in real time. By analyzing the periodic laws of resource usage of deep learning tasks, the present invention proposes a hybrid deployment strategy for different types of resources, realizes dynamic resource sharing between online and offline tasks, and at the same time ensures that the system can still operate stably and efficiently under high load conditions.

A method and apparatus for utilizing microslices in an enterprise wireless communication network to manage and control network performance. The microslice instances are monitored during network operation, for example at communication nodes, functional blocks, and end-to-end to provide Key Performance Indicators (KPIs). These KPI are compared with performance objectives, which may be Service Level Objectives (SLOs). If the performance objectives are not met by the KPIs, then one or more of the microslice instances may be dynamically adjusted until the performance objectives are sufficiently met. Alternatively, the lower priority microslice instances may be dropped (i.e., terminated) until the performance objectives are sufficiently met.

Systems and methods for implementing power oversubscription in graphic processing unit (GPU) servers are provided. An increase to a quantity of servers allocated to a group of GPU servers in an inference cluster is applied. Based on the power consumption of the group of GPU servers exceeding a first threshold, a frequency of low priority inference workloads is capped, and based on the power consumption of the group of GPU servers exceeding a second threshold, the frequency of the low priority inference workloads are capped and a frequency of high priority inference workloads are capped, enabling an increase in allocated server capacity in the existing inference clusters while maintaining service level objectives (SLOs).

Techniques are disclosed that enable a self-regulating process to meet a service level objective (SLO). In some embodiments, a self-regulating process is a background process comprising a regulator that receives background job requests and historical information related to the background process for evaluation to determine actions (e.g., speed up, slow down, or maintain the same speed), enabling the background process to adjust its pace gradually and smoothly even when encountering unexpected big changes in load.

A Service Level Objective (SLO)-based Continuous Integration / Continuous Development (CICD) framework is disclosed, which enables SLO-based Canary deployments, SLO-based chaos engineering, and / or SLO-based performance testing. The SLO-based CICD framework enhances and simplifies CICD pipelines and brings the pipelines in line with Site Reliability Engineering (SRE) best practices. By relying on SLOs rather than SLIs, the SLO-based CICD framework avoids the use of threshold-based alerts, which can be noisy and error prone, and the thresholds associated with threshold-based alerts can change over time due to changes in production volume. The SLO-based CICD framework uses multi-window burn rates when evaluating SLOs, which removes the reliance on a static, predefined thresholds of SLI threshold-based alerting, and removes sensitivity of such evaluations to changes in production volume.

Systems and methods for optimizing edge-assisted augmented reality (AR) devices. To optimize the AR devices, frame capture timings of AR devices can be profiled (110) that capture relationships between the AR devices. Requests from the AR devices can be analyzed (120) to determine accuracy of the frame capture timings of the AR devices based on a service level objective (SLO) metric. A frame timing plan that minimizes overall timing changes of the AR devices can be determined (130) by adapting the accuracy of the frame capture timings to optimal adjustments generated based on a change in device metrics for requests below an accuracy threshold. Current frame capture timings of cameras of the AR devices can be adjusted (140) based on the frame timing plan by generating a response pocket for the AR devices.

The techniques describe network slicing in computer networks. For example, a node receives a slice policy definition. The slice policy definition comprising a slice selector to identify packets belonging to one or more network slices, referred to as a “slice aggregate,” and one or more network resource requirements for the slice aggregate to meet one or more Service Level Objectives (SLOs). The node configures, based on the slice policy definition, a path for the slice aggregate that complies with the one or more network resource requirements. In response to receiving a packet, the node determines whether the packet is associated with the slice aggregate and, in response to determining that the packet is associated with the slice aggregate, forwards the packet along the path for the slice aggregate.

A Service Level Objective (SLO)-based Continuous Integration / Continuous Development (CICD) framework is disclosed, which enables SLO-based Canary deployments, SLO-based chaos engineering, and / or SLO-based performance testing. The SLO-based CICD framework enhances and simplifies CICD pipelines and brings the pipelines in line with Site Reliability Engineering (SRE) best practices. By relying on SLOs rather than SLIs, the SLO-based CICD framework avoids the use of threshold-based alerts, which can be noisy and error prone, and the thresholds associated with threshold-based alerts can change over time due to changes in production volume. The SLO-based CICD framework uses multi-window burn rates when evaluating SLOs, which removes the reliance on a static, predefined thresholds of SLI threshold-based alerting, and removes sensitivity of such evaluations to changes in production volume.

Techniques described herein relate to a method for managing a distributed multi-tiered computing (DMC) environment. The method includes obtaining, by a local controller associated with a DMC domain, service level objective (SLO) metrics; applying the SLO metrics to a predictive anomaly detection transformer to perform anomaly detection; making a first determination that an anomaly is detected; in response to the first determination: attempting basic remediation to resolve the anomaly; making a second determination that the basic remediation is unsuccessful; in response to the second determination: making a third determination that the anomaly is associated with a silent failure; and in response to the third determination: performing service impairment isolation to obtain a collection of services correlated to the anomaly; and performing root cause analysis to identify causal services.

The invention relates to the technical field of cloud computing, and provides a deep reinforcement learning cloud platform resource adjustment method based on user workload prediction. The method aims at solving the problem of adjustment hysteresis caused by the fact that load changes cannot be predicted in a traditional response type resource adjustment strategy, so that the rule violation rate of a service level target (SLO) is effectively reduced, and the resource utilization rate is improved. According to the main scheme, the method comprises the following steps: firstly, predicting a future workload trend through a long short-term memory (LSTM) network; combining a prediction result with a current system state (such as response delay and copy number) to serve as input of a deep reinforcement learning agent; the intelligent agent outputs a resource adjustment action through a strategy network; and finally, optimizing the decision model according to a composite revenue function integrating the service quality, the resource cost and the adjustment stability. The method can be widely applied to cloud native application under a micro-service architecture, and automatic and prospective elastic resource scaling management is realized.

A method is provided, comprising: obtaining a plurality of measured response times for a given storage group, each of the measured response times corresponding to a different instance of a same time window; classifying the plurality of measured response times with a machine learning model to obtain a first predicted response time and a second predicted response time, the first predicted response time corresponding to a first future instance of the time window, and the second predicted response time corresponding to a second future instance of the time window; detecting whether the first predicted response time and the second predicted response time satisfies a service level objective for the given storage group; and generating a notification when at least one of the first predicted response time and the second predicted response time fails to satisfy the service level objective.

The invention provides a computing power scheduling method and device, communication equipment, a medium and a product, and the method comprises the steps: carrying out the data preprocessing and topology awareness of a computing task, and obtaining a computing graph of the computing task; performing sub-graph division on the calculation graph to obtain calculation sub-graphs, and packaging the calculation sub-graphs into schedulable units; the calculation benefit of the schedulable unit is obtained by determining the execution delay, the power consumption and the heterogeneous coefficient of the calculation nodes which distribute the schedulable unit to different areas; constructing a task scheduling model meeting a service level target according to a maximum and minimum algorithm and the calculation benefit; the optimal allocation of the schedulable units on the computing nodes is determined through the task scheduling model, the schedulable units are scheduled to the computing nodes in parallel according to the optimal allocation, large computing tasks can be scheduled to a proper area according to delay and power consumption, the problem of hardware resource heterogeneity in the scheduling process can be solved, and the scheduling efficiency is improved. And the total power consumption of the whole task is reduced through refined scheduling.

The invention provides a job intention-oriented data arrangement method and device, and the method comprises the steps: compiling a job intention, and obtaining a multi-level arrangement strategy; the multi-level arrangement strategy comprises at least two of an application layer arrangement strategy, a transmission layer arrangement strategy and a system storage layer arrangement strategy; and executing the data transmission operation based on the multi-level arrangement strategy. According to the method provided by the invention, the operation intention containing the global demand and the constraint is acquired and compiled into the multi-level arrangement strategy covering multiple levels of application, transmission, storage and the like, and finally the data transmission operation is executed based on the strategy, so that the full-link resource configuration can be flexibly adjusted according to the service intention; the problem that downstream data receiving is blocked due to the fact that the network is not matched with the storage performance is effectively solved, and the comprehensive performance and the resource utilization rate of data transmission are remarkably improved while the service level target is guaranteed and complex constraints are met.

The invention provides a service level target violation diagnosis method, a service level target violation diagnosis device, service level target violation diagnosis equipment and a medium, and the method realizes the service level target violation diagnosis method in a data plane, and comprises the following steps: carrying out compression grouping on equipment on a network path based on path information, and collecting location-based diagnosis information in a probabilistic manner, the location-based diagnostic information includes state information of a specific point on the network path; probabilistically detecting packet loss port information of all devices on the network path based on the compression grouping result to obtain detail-based diagnosis information which refers to specific data required by diagnosis of the suspicious area; and performing native path coding according to the media access control code of each device in the network to determine a target device corresponding to a violation event contained in the location-based diagnostic information and / or the detail-based diagnostic information. The method does not depend on the control plane capability, alleviates the problem of high violation detection overhead, and has complete knowability.

Managing the resource demand load for edge systems is significantly more complex than for other systems, such as cloud environments. Edge resource demand load scheduling systems and methods are disclosed that can ensure that edge systems operate smoothly and efficiently while balancing multiple scheduling objectives. Scheduling techniques disclosed herein may utilize heuristic rules for candidate edge system selection (e.g., utilizing ARMA / ARIMA averages and / or service level objectives) and modified best fit decreasing (mBFD) assignment / allocation techniques.

The technology described herein is directed towards deploying a node cluster, e.g., in a cloud environment, based on abstract user storage solution inputs / service level objective input data, including user-specified storage capacity and performance characteristics. An intelligent system converts the input data into cluster topology data (e.g., virtual machines, storage devices / volumes and other information) that can be used for initial cluster deployment or infrastructure updates to existing node clusters in multi-cloud environments. The input data can be directed to an initial deployment of a new node cluster or an update to an existing node cluster. Further, the system can take customer rules as input data, monitor cluster conditions corresponding to those rules, and update the cluster based on a condition in the rules being met, by taking an action associated with the rule's condition to automatically generate and handle a cluster update request.