719 results about "Load balancing (computing)" patented technology

The invention discloses a big data distributed storage and parallel processing collaboration method based on cloud computing. The method comprises the following steps: sensing data stream characteristics in real time through a self-adaptive dynamic partitioning engine, dynamically adjusting a partitioning strategy and generating a metadata label; constructing a node selection model through a comprehensive evaluation algorithm, selecting storage nodes to form an optimal storage cluster, and dynamically adjusting a resource matching weight coefficient based on a load state through a load optimization module; decomposing a data processing task into parallel subtask units, and constructing a dual-objective optimization model; triggering a dynamic rebalance mechanism through a distributed monitoring agent in combination with a hierarchical early warning strategy; and constructing a multi-level cache system to optimize a data access path, outputting a final result, pushing the final result to the user terminal, and updating the knowledge base. Through collaborative optimization of dynamic partitioning, multi-dimensional resource scheduling, elastic scaling and intelligent caching technologies, the resource utilization rate, the load balancing capacity and the stability in a high-concurrency scene of the system are remarkably improved.

An AI orchestration system dynamically manages multiple artificial intelligence (AI) agents within a cloud computing environment to efficiently process user requests. A model orchestration subsystem determines whether a request is handled locally using a domain-specific database or by invoking one or more AI agents. The system maintains AI agents in active and inactive states, provisioning computing resources for inactive agents as needed. Real-time model metrics guide the selection of target AI agents, and if a degrading performance trend is detected, the system preemptively spins up additional AI instances. The system provisions processor cycles, memory, and network bandwidth through a cloud-based resource manager, instantiates containerized execution environments or virtual machines, and performs automated load balancing among AI instances.

The invention relates to a multi-modal large model dynamic compression and reasoning optimization method based on a MoE architecture. The method comprises the following steps: establishing an edge computing system conforming to medical equipment specifications, constructing a medical image analysis network based on an improved hybrid expert MoE architecture, and adopting a three-layer cascade structure of a feature coding layer, a dynamic routing layer and an expert execution layer; executing expert module dynamic loading and video memory optimization; executing knowledge graph compensation and domain knowledge injection; executing hardware instruction level optimization and calculation acceleration; executing multi-expert feature fusion and decision weighting; performing diagnosis result generation and confidence evaluation; performing real-time data return and model iterative optimization; executing multi-device cooperation and load balancing; executing system security monitoring and exception handling; and generating a structured diagnostic report. The problem that the precision loss of a multi-modal large model is difficult to meet actual requirements is solved, and medical feature adaptive dynamic compression, medical hardware collaborative energy efficiency optimization and cross-modal compensation of medical knowledge enhancement are realized.

The invention relates to the technical field of data processing, and discloses an automobile manufacturing industrial data distributed processing method and system based on digital twinning. The method comprises the following steps: monitoring and collecting production abnormal events, decision emergency degree parameters and simulation task types based on service states to generate a service priority evaluation data set, and performing priority ranking on a plurality of simulation tasks to form a dynamic priority queue and a resource demand matrix, the queue is input into a distributed simulation engine for CPU, memory and GPU load balancing processing to generate a node resource allocation scheme, and preemptive scheduling is executed to migrate low-priority tasks to idle nodes to form a task execution mapping table; and monitoring simulation progress and resource consumption through an adaptive scheduling mechanism to obtain scheduling performance feedback data, and updating the dynamic priority queue. According to the method and the device, the problem of lack of an intelligent scheduling mechanism during concurrent execution of multiple simulation tasks in the prior art is solved, and the response speed of the key simulation task and the utilization efficiency of computing resources are improved.

The invention relates to the field of distributed machine learning, provides a big data-oriented distributed model training optimization method, and solves the problems of load imbalance, low resource utilization rate, large communication overhead, insufficient fault-tolerant efficiency and the like caused by data fragmentation staticization in the prior art. Load balancing is realized through intelligent clustering and overlapping control; the multi-dimensional heterogeneous resource evaluation model monitors calculation / storage / network indexes in real time, and realizes adaptive scheduling in combination with a task prediction and optimization algorithm; the hierarchical gradient synchronization mechanism adopts a tree-shaped parameter server and a dynamic compression technology, so that the communication traffic is reduced by 50%, and the precision loss is less than 0.8%; the incremental checkpoint system uses erasure code coding and parallel recovery to shorten the fault recovery time from 15 minutes to within 2 minutes, the resource utilization rate reaches 85% or above, the convergence speed is improved by 30%-40% in ResNet, BERT and other model training, and the large-scale training efficiency and the system stability are remarkably optimized.

The invention discloses a virtual machine scheduling method in a distributed environment based on deep reinforcement learning, and belongs to the technical field of cloud computing resource scheduling. According to the method, the defects of a traditional method in multi-objective optimization and mixed action space collaborative decision-making are overcome by constructing a mixed action space joint decision-making mechanism. The method specifically comprises the following steps: establishing a mixed action space containing discrete node selection and continuous resource allocation, filtering invalid nodes by adopting a dynamic mask mechanism, and ensuring resource ratio constraint through projection gradient descent; designing a hierarchical reward function to realize multi-target dynamic balancing, and dynamically adjusting the priorities of energy consumption, load balancing and SLA guarantee based on an adaptive weight strategy; a multi-agent collaborative framework is provided, cross-node topological dependence is captured by using a graph attention network, and dynamic fusion of spatio-temporal characteristics is realized through cross attention in combination with LSTM coding time sequence load characteristics; a course learning strategy and a priority experience playback mechanism are introduced to improve training efficiency and strategy robustness.

The invention relates to a competition management system and method based on cloud computing, and belongs to the technical field of resource allocation. The number of virtual machines, physical resources and competition levels are determined by dynamically acquiring competition information, resource priorities are automatically associated, stable allocation of high-level competition item resources is guaranteed, a scheduling model is constructed through a deep reinforcement learning algorithm, a virtual machine allocation probability matrix is generated, and long-term income is evaluated in combination with a reward function. Autonomous learning optimization of a scheduling strategy is realized; a multi-dimensional server state matrix is constructed, matching tasks and hardware characteristics are quantified, and the task interruption risk is reduced; a dynamic label system is generated in combination with static code analysis and runtime data, and key competition resource allocation is ensured; and staged competition items are staged, and resource demand vectors are defined, so that staged dynamic resource adjustment is realized. By remarkably improving the resource utilization rate and the load balancing capacity, the fairness of competition management, the system stability and the intelligent level are improved.

The invention discloses a performance test and verification method, system and device for a multi-core heterogeneous chip and a medium, and the method comprises the steps: configuring a test parameter set according to a target application scene, the test parameter set comprising a processor core type recognition parameter, a communication topology weight parameter and a load balancing threshold parameter; generating a test case covering the heterogeneous computing core cooperative working mode; the method comprises the following steps: acquiring characteristic data during testing through a reconfigurable monitoring sub-module, wherein the characteristic data comprises inter-core communication delay information, shared cache hit rate information and power consumption distribution thermodynamic information; and performing quantitative analysis on the feature data through a multi-dimensional evaluation model to obtain a performance deviation index and an architecture optimization suggestion parameter. By optimizing the chip performance verification process under the multi-core heterogeneous architecture, the verification precision is effectively improved, and the chip performance verification method can be widely applied to the technical field of chip design.

The present disclosure provides connection management techniques based on graph neural networks (GNN) and deep reinforcement learning (DRL) to optimize user association and load balancing. A graph structure of a communication network is considered for the GNN architecture and DRL is used to learn parameters of the GNN algorithm / model. Connection management is defined as a combinatorial graph optimization problem, and the DRL mechanism uses the underlying graph to learn weights of the GNN for an optimal user connections or associations. The connection management techniques can consider local network features to make better decisions to balance network traffic load while network throughput is also maximized. Implementations are provided based on edge computing frameworks include the Open RAN (O-RAN) architecture. Other embodiments may be described and / or claimed.

The invention relates to the technical field of digital computing and resource scheduling, in particular to a computing power resource intelligent scheduling and load balancing method. The method comprises the following steps: collecting performance indexes of a computing power node on different resource dimensions, and constructing a resource capability vector of the computing power node; extracting demand features of the calculation task on different resource dimensions, and constructing a task feature vector; matching the resource capability vector of the computing power node with the task feature vector to obtain an adaptation degree score; operation indexes of the computing power nodes are collected, comprehensive load scoring is carried out on the computing power nodes, and the current load states of the computing power nodes are obtained; and distributing calculation tasks based on the current load state of the computing power node and the adaptation degree score, and updating the current load state of the computing power node in real time. The problems that the resource utilization rate is low, the task response delay is high and the node load is unbalanced due to the fact that the computing power resources cannot be dynamically matched according to the task characteristics and the computing power node states in a traditional computing power resource scheduling method are solved.

The invention discloses an artificial intelligence large model training method in a heterogeneous multi-machine and multi-card environment, and belongs to the technical field of artificial intelligence large model training. Load balancing of heterogeneous equipment is realized by constructing a uniform interface, and the communication efficiency is optimized by adopting hierarchical pipeline aggregation and dynamic quantization compression; the node dynamic adjustment is realized in combination with the elastic topological structure, the problems of poor equipment compatibility, high communication delay and rigid topological structure in the prior art are effectively solved, and the method has the remarkable advantages of improving the utilization rate of heterogeneous computing resources, reducing the cross-node communication overhead and enhancing the fault-tolerant capability of the system.

The invention discloses an Internet of Things operation management scheduling system based on cloud edge collaboration, relates to the technical field of task scheduling optimization, and aims to solve the problem of adaptability of edge computing and cloud edge collaboration architecture, the Internet of Things perception access technology is adapted and fused with heterogeneous equipment through an architecture adaptation module, various IOT data are stably acquired, and the cloud edge collaboration is realized. The task execution position is dynamically adjusted in combination with the task distribution unit; real-time stream calculation and an ETL processing mechanism are combined to fit a comprehensive scheduling index Zdzh, the utilization rate of calculation resources is increased, and task scheduling is optimized; internet of Things equipment data, internet data and government affair data are analyzed based on the data lake and warehouse architecture, a data access strategy is optimized, and the data fusion efficiency is improved; and calculating and evaluating a sudden anomaly prediction coefficient Tycs, triggering an anomaly early warning mechanism or a sudden anomaly emergency mechanism, optimizing task migration, calculating node load balancing and data traceability analysis, so as to improve the stability and intelligent level of an Internet of Things operation management scheduling system.

The invention belongs to the technical field of mobile communication, and particularly relates to a task unloading and resource allocation method for edge computing. According to the method, a three-layer network structure is established, a distributed decision framework is constructed through reinforcement learning, the task emergency degree is dynamically evaluated, and computing resources are distributed in a differentiated mode; and task unloading and resource allocation are optimized in combination with an edge-cloud collaborative architecture, so that calculation load balancing is realized. According to the method, aiming at a cloud edge-end collaborative edge calculation model, the total cost of a system is defined as a joint optimization problem of task unloading time delay and energy consumption, the problem model is converted into a Markov decision process, and multi-agent and multi-user oriented deep reinforcement learning algorithm agent near-end strategy optimization (MAPPO) is designed; and obtaining an optimal unloading decision through mutual learning among multiple agents. According to the method, the total cost of the system can be effectively reduced, the rationality of edge computing task unloading and resource allocation decision is realized, and meanwhile, the use experience of a user can be improved.

The invention discloses an energy-efficient task scheduling method for an edge computing system, which comprises the following steps of: acquiring a task state, a computing node resource state, a link state and an energy consumption state according to a unified time slot under a computing power network control domain, and constructing a system state vector; performing priority evaluation on the to-be-scheduled task based on the residual delay budget, the candidate node energy efficiency coefficient and the queue position to obtain a target task set; inputting a system state vector and a target task set into an energy efficiency perception deep reinforcement learning scheduling model, outputting a task-node allocation decision under the constraint of computing node resources and task time delay, and introducing a system-level energy consumption ratio, self-adaptive energy consumption penalty and exploration bias facing high-energy-efficiency nodes into rewards; and scheduling tasks according to the allocation decision, recording state transition and instant rewards, updating a double-commentator and actor network, and performing iterative execution in continuous time slots. According to the method, task success rate, time delay, load balancing and energy-saving performance are considered, and system energy consumption is reduced.

The invention discloses an elastic management and optimal scheduling method and system for cloud computing resources, and belongs to the technical field of cloud computing resource management.The method comprises the steps that a cluster composed of multiple types of intelligent agents is constructed, and the intelligent agents comprise a resource evaluation intelligent agent, an elastic telescopic intelligent agent, a load balancing intelligent agent and a fault recovery intelligent agent; the intelligent agents are distributed and deployed on different nodes of a cloud computing data center, and information interaction is realized through a communication interface based on RESTful API and a distributed message bus constructed by a gRPC protocol; a unified agent management platform is set up and is responsible for registration, monitoring, scheduling and dynamic adjustment of agents, and orderly operation and efficient cooperation of the whole multi-agent system are ensured. According to the method, the functions of efficient elastic allocation, intelligent load balancing, rapid fault recovery, continuous optimization evolution and the like of resources in the cloud computing data center can be realized, and the quality and competitiveness of cloud computing services are effectively improved.

The invention is applicable to the technical field of computing power distribution, and provides a power equipment edge gateway computing power dynamic distribution method, which comprises the following steps of: determining a system state representation vector based on multi-source data of power equipment; the system state representation vector comprises a real-time computing power occupancy rate, a task queue length, a network bandwidth utilization rate and an equipment energy consumption index; constructing target parameters and constraint conditions of the calculation power dynamic distribution model; taking the system state representation vector as input, solving the computing power dynamic allocation model through a reinforcement learning algorithm, and generating a task scheduling path and a resource allocation proportion; and optimizing a preset hierarchical buffer queue based on the task scheduling path and the resource allocation proportion, and generating a target dynamic allocation strategy. According to the method, the high-real-time task response speed and the resource utilization rate are improved, energy consumption is reduced, meanwhile, backlog tasks are efficiently processed in the idle period through an event-driven mechanism, and system load balance is guaranteed.

The invention relates to the technical field of cloud computing, and discloses an enterprise resource planning data efficient processing method based on cloud computing, and the method comprises the steps: collecting system data of finance, supply chain, production management and the like through a multi-source interface module, and carrying out the preprocessing through a hierarchical fusion framework, thereby obtaining a standardized data set; the method comprises the following steps of: selecting a multi-target scheduling model, inputting the multi-target scheduling model into a task decomposition model, generating task decomposition parameters based on a block processing mechanism and a service association rule, constructing the multi-target scheduling model aiming at the shortest processing duration and the minimum resource occupation, and outputting an optimal scheduling scheme by adopting a dynamic allocation algorithm. A hierarchical execution control model comprising a strategy layer, a coordination layer and an execution layer is established, the strategy layer performs global task planning, the coordination layer performs local resource adjustment, and the execution layer tracks tasks based on a load balancing algorithm and outputs a processing control instruction to realize efficient processing. The method improves the data processing efficiency and the resource utilization rate, and provides support for enterprise decision making.

The invention belongs to the field of high-performance computing, and relates to an integer parallel computing method and device based on distributed storage and computer equipment, and the method comprises the steps of collecting real-time resource indexes, dynamically identifying fault nodes, triggering task migration, and performing data verification and hard disk fault detection. The weight value of each node is calculated, the nodes are arranged according to the descending order of the weight values, and the nodes with high load capacity are selected to distribute tasks; dynamically distributing a data generation task to a computing node, executing parallel computing, and performing distributed storage on a result; obtaining an operand, converting the operand into a first-order tensor form of a basic operand, serializing tensor data, and sending the serialized tensor data to a parallel computing layer; distributing a search task to a computing node, retrieving storage data in parallel, reading effective data from a storage layer, and combining search results into a partial sum; and summarizing and then outputting. The system has dynamic resource management and fault-tolerant capabilities, and can realize efficient task allocation and load balancing.

Aspects of the disclosure are directed to improving load balancing for serving mixture of experts (MoE) machine learning models. Load balancing is improved by providing memory dies increased access to computing dies through a 2.5D configuration and / or an optical configuration. Load balancing is further improved through a synchronization mechanism that determines an optical split of batches of data across the computing die based on a received MoE request to process the batches of data. The 2.5D configuration and / or optical configuration as well as the synchronization mechanism can improve usage of the computing die and reduce the amount of memory dies required to serve the MoE models, resulting in less consumption of power and lower latencies and complexity in alignment associated with remotely accessing memory.

The invention provides a data processing method and system based on cloud computing, belongs to the technical field of data processing, and realizes high concurrent processing and automatic load balancing by dynamically scheduling computing power through elastic resource allocation and a distributed computing framework and combining a storage and computing separation framework. The system supports cross-node disaster recovery backup and multi-layer encryption, and data security is guaranteed; and an on-demand payment mode is adopted, so that the hardware input cost is reduced, the resource utilization rate is improved, and the large-scale data processing efficiency and the system expandability are remarkably improved.

The present application relates to the technical field of large-scale data processing, and particularly relates to a parallel computing method and system suitable for large-scale data processing. The system comprises a task management unit, a distributed load balancing module, an elastic expansion architecture, an intelligent communication optimization module, and a resource monitoring unit, wherein the task management unit divides large-scale data into a plurality of sub-tasks by means of a task decomposer, and distributes the sub-tasks to computing nodes by means of a task scheduler and a priority distributor; the distributed load balancing module achieves global load balancing by means of a load sensing unit, a dynamic adjustment unit and a balance optimization unit; the elastic expansion architecture dynamically adjusts system resources by means of a node manager, a resource pool controller and an expansion decision-making device; the intelligent communication optimization module optimizes inter-node communication by means of a communication path planning unit, a bandwidth distribution unit and a delay compensation unit; and the resource monitoring unit monitors the system performance in real time by means of a performance collector, a state analyzer and an anomaly detector.

The invention provides an intelligent computing fusion network routing system and a routing method thereof. The system comprises an entrance gateway, a routing forwarding node, a computing node and a computing network controller, the computing network controller is used for carrying out path selection and computing resource allocation in combination with a real-time network state and load information of the computing node, dynamically generating and updating a computing network routing table through a computing network fusion routing algorithm, and issuing the computing network routing table to the entry gateway and the routing forwarding node; and the entry gateway receives and stores the computing network routing table issued by the computing network controller as a data stream access point, receives a computing power data stream which is input from the outside and carries a computing power service request, executes computing power routing table matching according to the computing power service request, and forwards the computing power data stream to a proper routing forwarding node according to a matching result. According to the invention, by optimizing the joint scheduling of the computing power service paths and the computing nodes, a plurality of paths of the anycast computing power routing and the weights of the paths are generated, the service flow level load balancing is realized, and the end-to-end time delay is reduced.

The invention relates to the technical field of big data, and discloses a resource pool planning method based on a computing power network, comprising the following steps: S1, global resource modeling; s2, intention-driven resource slicing; s3, intelligent scheduling and load balancing; s4, predictive fault-tolerant self-healing is carried out; s5, energy efficiency optimization resource recovery; according to the method, a safe and efficient distributed computing power network resource awareness framework is constructed by fusing a dynamic key agreement protocol and a federated learning mechanism; according to the new method, the resource discovery accuracy, the state updating timeliness and the abnormal node recognition capability are remarkably improved, and meanwhile, the privacy leakage risk in cross-domain collaboration is effectively avoided through the national secret algorithm encryption, block chain auditing and zero-knowledge certification technologies; according to the method, greenization and high efficiency of resource allocation are realized through a carbon perception dynamic scheduling strategy and a multi-objective optimization model; tasks are preferentially scheduled to green energy nodes, energy consumption of idle nodes is reduced in combination with a dynamic frequency modulation technology, and system energy consumption and carbon emission are reduced.

The invention relates to an industrial Internet of Things cloud edge collaboration method based on distributed reasoning, and the method comprises the steps: segmenting a deep learning reasoning model into a plurality of sub-models according to a reasoning task, and packaging each sub-model into a lightweight WASM model through format conversion, operator detection and structured pruning in combination with containerized deployment and WasmEdge operation, the memory and CPU occupation of the edge node is obviously reduced, the loading and reasoning time delay is shortened, and the network and computing resource utilization rate is improved; and then, distributing each Docker container mirror image to each edge node according to a resource distribution strategy to perform distributed reasoning, thereby realizing multi-node load balancing and reliable scheduling, and improving the throughput and the overall reasoning rate of the industrial Internet of Things.

The invention discloses a multi-cluster heterogeneous computing power scheduling method oriented to an AI training reasoning task, and belongs to the technical field of computers. According to a scheduling mechanism with issuing of the AI task as a core, the state of a resource pool is monitored in real time, resources are dynamically allocated, it is ensured that the AI task is efficiently executed, the use efficiency of the whole resource pool is improved, and the load state of an AI application example is detected in real time; through an elastic telescoping function, AI application examples are automatically expanded and shrunk, stable operation of AI tasks is ensured, resources of a plurality of clusters are integrated into a unified resource pool, centralized management and cooperative scheduling of the resources are achieved, the load condition of each cluster is monitored in real time, load balancing is automatically carried out among the clusters, and the AI tasks are reasonably distributed to different clusters. And the GPU resources of the specified model are accurately allocated to the AI task according to the actual demand of the AI task, so that the situation that the AI task application runs on non-optimal resources is avoided, and the efficient running of the AI task is ensured.

The invention discloses a resource collaborative awareness-based Storm flow calculation dynamic scheduling method, and provides the following technical scheme aiming at the problem that the performance of a Storm default scheduling algorithm is reduced in node abnormity, resource fluctuation and rescheduling scenes: firstly, dynamically sensing node busy, downtime and data flow fluctuation events through a real-time monitoring module; triggering a rescheduling process; secondly, constructing a resource collaborative optimization model based on historical task instance resource requirements, node performance indexes and communication overhead, and generating a task instance allocation scheme by taking minimization of inter-node communication cost as a target and combining CPU / memory dynamic threshold constraints; further, a greedy algorithm is adopted to sort high-relevance task instances, the high-relevance task instances are preferentially distributed to nodes with the optimal historical performance, and it is ensured that the node resource utilization rate does not exceed a dynamic threshold value; meanwhile, node load balancing parameters are corrected in real time through time window smoothing processing, and the remaining resource state is updated; and finally, outputting an optimized topology division result, so that the system delay after rescheduling is remarkably reduced, and the throughput is improved. According to the method, rapid recovery and stable operation of the heterogeneous cluster are realized through resource collaborative modeling, historical data driven dynamic scheduling and load balancing optimization.

The invention discloses a switch load balancing method based on fuzzy logic control, and the method comprises the following steps: S1, monitoring a network state in real time through quantum sensing equipment, and obtaining related parameters; s2, transmitting the parameters to a fuzzy logic controller for processing; s3, generating a load distribution strategy according to a predefined rule; s4, transmitting the preliminary strategy to each switch node in the block chain network, and recording related data; s5, verifying the data by using a distributed consensus mechanism of the block chain; s6, dynamically evaluating a node trust value based on an enhanced fuzzy trust model, and selecting nodes meeting requirements to participate in a load balancing game decision; s7, carrying out cooperative calculation on the selected nodes by utilizing a game theory cooperation mechanism, and generating a globally optimized load distribution strategy; and S8, optimizing a load distribution strategy through a quantum genetic algorithm, and performing automatic execution through an intelligent contract. According to the method, the advantages of fuzzy logic, the block chain and quantum computing are combined, and the intelligence and safety of load balancing are improved.

The invention relates to a cluster-oriented large model parallelization method and device and an electronic device.The method is applied to the field of large models.The method comprises the steps that operator information of a cluster-oriented large model in a preset microprocessing batch and a preset parallelization mode is obtained, the operator information comprises operator time information and operator memory information of operators in the cluster-oriented large model; the cluster comprises one or more types of accelerators; determining an initial operator parallel configuration strategy of a plurality of assembly lines in the large model based on the operator information, model memory information required by the large model and a memory extreme value of an accelerator; and performing recursion processing on the initial operator parallel configuration strategy according to a preset load balancing mode to obtain a target operator parallel configuration strategy, and running the cluster-oriented large model based on the target operator parallel configuration strategy. According to the method and the device, the utilization efficiency of chip calculation performance during large model parallel configuration is improved, and high efficiency and wide application range of large model training are realized.

The invention relates to the technical field of deep learning model training optimization, in particular to a fine-grained resource scheduling method and system in DNN model parallel training, and the method comprises the steps: converting a DNN model into calculation graph representation, and constructing a calculation graph in combination with the topological information of calculation equipment; modeling calculation stage division in a DNN model training process and resource mapping from each calculation stage to calculation equipment as a stage division and resource mapping combined optimization problem; and iteratively solving the combinatorial optimization problem by using a heuristic algorithm, and optimizing the computing equipment resource mapping of each computing stage by dynamically adjusting the weight of constraint conditions and minimizing the overhead and load difference of the computing equipment. According to the method, the calculation cost, the communication overhead, the overall load balancing and the solving time are comprehensively considered in the target function, efficient calculation resource allocation and task scheduling optimization are achieved, the training and reasoning efficiency of a large-scale deep learning model is improved, and the method has a good application prospect in the field of deep neural network distributed parallel training.

The invention discloses a computer resource allocation management system and method based on data analysis, and relates to the technical field of computer resource management.The method comprises the steps that data are collected in real time, and historical data are collected; constructing a spatio-temporal data warehouse based on the real-time data and the historical data; extracting static features and dynamic features of the task, and combining the static features and the dynamic features to generate a high-dimensional task feature vector; constructing a resource semantic knowledge graph, and defining a computing resource entity, a relationship and a constraint rule; combining the spatio-temporal data warehouse and the resource semantic knowledge graph to construct a demand prediction model, and outputting probability distribution prediction of future resource demands; and in combination with real-time data, calculating resource use efficiency and a load balancing index in real time, and dynamically adjusting a resource allocation decision. The demand prediction model is constructed based on the spatio-temporal data warehouse and the resource semantic knowledge graph, resource allocation is carried out actively, resource shortage or idle is avoided, and the resource utilization rate is improved.