161 results about "Heterogeneous cluster" patented technology

According to at least one embodiment, a method comprises receiving, into a capacity planning system, workload information representing an expected workload of client accesses of streaming media files from a site. The method further comprises receiving, into the capacity planning system, at least one service parameter that defines a desired service characteristic to be provided by a heterogeneous cluster of servers under the expected workload. The capacity planning system evaluates whether the heterogeneous cluster, having a plurality of different server configurations included therein, is capable of supporting the expected workload in compliance with the at least one service parameter.

A system and method for scheduling client-server applications onto heterogeneous clusters includes storing at least one client request of at least one application in a pending request list on a computer readable storage medium. A priority metric is computed for each application, where the computed priority metric is applied to each client request belonging to that application. The priority metric is determined based on estimated performance of the client request and load on the pending request list. The at least one client request of the at least one application is scheduled based on the priority metric onto one or more heterogeneous resources.

The invention provides a method and a system for the fusion of multi-source monitoring data based on a heterogeneous clustering wireless sensor network. The system comprises a plurality of monitoring nodes, a plurality of cluster head nodes, a plurality of trunk nodes, a gateway node and an upper computer. The method comprises the following steps: after the gateway node switches on the upper computer, a PAN network in conformity with the ZigBee protocol is established; after the cluster head nodes and the trunk nodes are electrified, the cluster head nodes and the trunk nodes search for and join in the PAN network to form a wireless meshed network; the cluster head nodes receive broadcast packets of the monitoring nodes to carry out correlation calculation, and provide the calculation result to the upper computer through the wireless meshed network to carry out clustering processing; and then, the monitoring nodes in the cluster acquire environmental data according to the clustering result and send the data to the cluster head nodes in the cluster, and the cluster head nodes send the acquired environmental data to the upper computer through the wireless meshed network to be subjected to corresponding processing based on different sources of the environmental data. Thus, the heterogeneous clustering wireless sensor network and the wireless meshed network are combined, and the advantages of the both can be fully used for data acquisition.

The invention discloses a scalable monitoring system supporting hybrid clusters, comprising a monitoring agent program, a main monitoring program and a graphical interface; the monitoring agent program operates above a monitored node computer and broadcasts the performance monitoring data of the computer to all nodes in a group; the main monitoring program operates above a monitoring host computer, connects with all data resource nodes, periodically acquires the performance monitoring data of all the nodes of the group in which a data resource exists from the data resource nodes, and stores the acquired data into a database; a backend module of the graphical interface acquires relevant data from the database and sends the data to a frontend display module, and the frontend display module carries out visual presentation on the data and prompts alarm information. The invention can favorably adapts to the expansion of system function and scale, support isomerous and hybrid architectures and simultaneously adopt various monitoring strategies to monitor a plurality of nodes and equipment with different architectures and types, thereby having wide application prospect.

The invention relates to an atmospheric pollution monitoring system based on a wireless sensor network, which comprises wireless sensor network nodes, wireless sensor network sink nodes, local monitoring subcenters, a remote control terminal and a mobile phone terminal. Besides the atmospheric pollution monitoring system based on the wireless sensor network, the invention also customizes a wireless sensor network multi-level heterogeneous clustering routing protocol at the stage of network self-organization; the design of the system is reasonable, the system is easy to deploy, and the intelligent degree is high; when being applied on a large scale, the system can balance the energy consumption of the wireless sensor network nodes, accurately and timely reflect the situation of atmospheric pollution and give the alarm, the wireless sensor network nodes within monitored areas can be conveniently self-networked, and are highly scalable, a large-scale monitoring system can be constructed, and the system is suitable for popularization and application.

The invention belongs to the technical field of concurrent calculation and concretely relates to a concurrent calculation frame system based on Spark and a GPU (Graphics Processing Unit). Based on YARN resource management platform, a resource manager and a node manager are improved so that the concurrent calculation frame system can effectively sense GPU resources of a heterogeneous cluster, thereby supporting management and scheduling of the cluster GPU resource; and then in a YARN deployment mode, a job scheduling mechanism and a task execution mechanism of the Spark are improved so that the concurrent calculation frame system supports scheduling and execution for a GPU task. Identification for the GPU resource is introduced in stages such as resource application, resource distribution, DAG (Directed Acyclic Graph) generation, stage division and task execution so that an execution engine can sense the GPU task and effectively executes the GPU task in the heterogeneous cluster; and meanwhile, an effective programming model under the framework is provided by use of the characteristics of efficient memory calculation of Spark in combination with the advantages of multi-core concurrent calculation of the GPU. The system can effectively process data-intensive and compute-intensive jobs and greatly improves the job processing efficiency.

The embodiment of the invention provides a task scheduling method and device in a heterogeneous cluster and electronic equipment, and the method can comprise the steps: determining each task scheduledto a computing node for each computing node; dividing each sub-task into a plurality of sub-tasks; when the parent-child tasks corresponding to the child tasks are executed, adding the sub-task intoa ready child task queue; for each first sub-task in the ready sub-task queue, when the execution rate of the first sub-task in a GPU computing unit is greater than that of the first sub-task in a CPUcomputing unit, adding the first sub-task to the GPU sub-task queue; and scheduling each second sub-task to a target GPU corresponding to the second sub-task according to an interference relationshipbetween each second sub-task in the GPU sub-task queue and the sub-task being executed by the GPU. Therefore, the resource utilization rate can be improved, and the system throughput of the heterogeneous cluster is improved.

The invention discloses a heterogeneous cluster and a task processing method and apparatus. The heterogeneous cluster comprises multiple calculation nodes, a cluster resource management module, an Ethernet exchanger and an IB network exchanger, wherein the calculation nodes are used for performing parallel calculation; each calculation node comprises a GPU and a CPU; the cluster resource management module is used for allocating the calculation nodes in the cluster and GPU resources on the calculation nodes; the calculation nodes are connected with the cluster resource management module throughthe Ethernet exchanger; and the calculation nodes are connected through the IB network exchanger. According to the scheme, the heterogeneous cluster can be used for performing the parallel calculation, so that the acceleration ratio is increased and the executive efficiency of a deep learning task is improved.

The invention discloses an adaptive data loading method for heterogeneous cluster storage, and belongs to the field of computer system storage. The method comprises the following specific steps: (1) basic load balance: at the initial establishment of a system, distributing data on each node of clusters according to the maximum load capacity of each node; (2) adaptive distributed incremental load balance: during running of the system, collecting the resource occupation situation of each node in real time, and adjusting a data distribution situation adaptively and dynamically; and (3) distributing data loads among the clusters in a balanced manner at the initial establishment of the system by considering the load size of data as identical and taking the performance of a heterogeneous server as a weight through a weight-based hash method. The method is suitable for processing heterogeneous data loads in a high-stress, high-concurrent-reading-writing, multi-user and heterogeneous large-scale distributed storage system, and adaptively adjusting the distribution of data among servers according to loads.

A heterogeneous parallel processing system that includes a computer cluster system and a device cluster system is provided. The computer cluster system includes a computer cluster node that includes at least one cluster node processor. The device cluster system includes a cluster device that includes a cluster device processor, and is substantially different from the computer cluster nodes. The system includes a heterogeneous cluster node that includes a computer cluster node from the computer cluster system and a cluster device from the device cluster system. The heterogeneous cluster node includes a runtime environment module for transmitting data between the computer cluster node and the cluster device in the heterogeneous cluster node, such that at least one process executing on the device cluster system is transparent in its operation to a plurality of processes executing in the computer cluster system.

The invention belongs to the technical field of cloud computing, and provides a big data appliance realizing method based on a CPU-GPU heterogeneous cluster. The method includes the steps that the computer cluster is set up and comprises a Master node provided with a CPU and other Slave nodes provided with CPUs and GPUs; a CUDA is installed on each Slave node; a MapReduce model provided by a Hadoop is selected, a Map task is started for each task block, and the Map tasks are transmitted to the Slave nodes to be calculated; the Slave nodes divide the received Map tasks into corresponding proportions, dispatch the proportioned Map tasks to the CPUs or the GPUs to execute Map and Reduce operation and transmit operation results to the Master node; the Master node receives the operation results fed back by all the Slave nodes and finishes all task processing.

Various embodiments of methods and systems for intelligent thermal power management implemented in a portable computing device (“PCD”) are disclosed. To mitigate or alleviate unwanted workload migration that could exacerbate a thermal energy generation event in a processing component having heterogeneous processing core clusters, embodiments of the solution apply mitigation measures in a predetermined order to the large cluster before applying any thermal mitigation measures to the small cluster.

The invention discloses a large-scale data processing system and method, which are applied to a hybrid heterogeneous cluster of central processors and graphics processor. The large-scale data processing system comprises at least one shared storage node, a control node and multiple calculating nodes, the shared storage node, the control node and the multiple calculating nodes are connected via a high-speed network, the control node overall plans and allocates basic data blocks in a data packet to the calculating nodes, to ensure load balance of the calculating nodes, the calculating nodes directly read corresponding basic data blocks from the shared storage node for calculating and return computed results to the control node, the data transmission process is optimized, and the calculating time is shortened, so that the performance of the existing system is fully used, the efficiency of the entire system is greatly improved, such situations as insufficient network bandwidth, small internal memory capacity and the like are not limited, and the requirements of high-performance applications for processing large-scale data are satisfied.

The invention relates to a task scheduling method and a task scheduling system facing mixed load in a heterogeneous cluster. The method comprises the following steps: a resource scheduler is used for receivingthe beats of machines and maintaining the cluster attributes of the machines; a job manager is used for receiving and resolving operation to obtain a plurality of tasks; the job manager is used for setting the attribute cluster and restrain demand for the task, and transmitting the task information to a resource manager; the resource scheduler is an optimal machine which is used for meeting the restraint of task matching, and returning a matching relation between the task and the machine to the job manager; and the job manager is used for transmitting the task to an actuator on the matched machine for executing tasks. The attribute and task requirement of the heterogeneous machine can be indicated by an easily expandable description method, and the optimal machine is distributed for the task based on taking the hard constraint as a filtration standard and the soft constraint as a selection standard, therefore the executing efficiency of the task and the integral performance of the system can be obviously improved.

Embodiments of the present invention provide a heterogeneous cluster system for CFD simulation calculation and a CFD calculation method. The heterogeneous cluster system for CFD simulation calculation comprises a central processing unit (CPU) and a hardware accelerator. The CPU is configured to divide the received CFD task into task blocks, according to the attribute of the CFD task, the CFD boundary condition and the performance parameter of the hardware accelerator, determine the hardware accelerator for calculating the task blocks, and transmit the task blocks to the hardware accelerator; and the hardware accelerator is configured to receive and calculate task blocks transmitted by the CPU and transmit the calculation result to the CPU. According to the technical scheme of the embodiments of the present invention, by constructing the heterogeneous cluster system integrating the CPU, the GPU and the FPGA, calculation resources are intelligently deployed according to the attribute of the CFD task, so that the bottleneck problem of CFD simulation calculation is solved and the running efficiency of the CFD application is improved.

The invention relates to the field of parallel computation, and discloses a parallel computation dispatch method in the heterogeneous environment. According to the method, a plurality of JVM task slots having different internal storages, and idle task slot sets are built, tasks in the parallel computation are divided into I/O intensities and CPU intensities, the tasks are appointed to suitable task slots for calculation, and parallel calculation efficiency in the heterogeneous environment is optimized. The parallel computation dispatch method has the advantages that the sizes and the types of the internal storages required by the tasks are determined dynamically, resource using efficiency of heterogeneous clusters is improved, total operation time of the parallel calculation work is shortened, and the situation that the internal storages overflow in a task operation process is avoided.

The invention discloses a method of data flow programming oriented to CPU/GPU heterogeneous clusters. The method comprises the steps of utilizing a data flow program to obtain a SDF diagram, conducting hierarchical secondary task division and scheduling on the SDF diagram, setting a stage number for each task unit to obtain the execution order of the task units; utilizing a CPU kernel or GPU mapping from the task unit to heterogeneous cluster nodes in the SDF diagram and a heterogeneous cluster multitask data communication model to obtain data communications between the task units, and obtaining target codes according to the execution order of the task units and the data communications between the task units. According to the method and system of data flow programming oriented to CPU/GPU heterogeneous clusters, the data communications are optimized with the parallelization of the data flow programs, and the overall execution performance of the programs is improved.

The invention discloses a distributed job scheduling method suitable for a heterogeneous computational capability cluster. The method comprises the following five steps: (1) automatic discovery of node resources, wherein each node in the cluster transmits resource information to a network in a form of a multicast message; and a scheduler receives the multicast message and automatically discovers cluster nodes; (2) a job scheduling mechanism based on first input first output (FIFO) and resource utilization maximization; (3) a data set-based dynamic rescheduling mechanism, wherein a job manager dynamically adjusts the data sets of various tasks; (4) a delay scheduling mechanism, wherein partial tasks of the job are deployed on all nodes; and the remaining tasks are submitted to a delay queue to be scheduled; and (5) use of a redundant mutual preparation mechanism, wherein the scheduler deploys a backup task for each task to ensure that the backup task still can provide a computation result after the task is in a fault or off-line. According to the distributed job scheduling method, the distributed processing capacity of the cluster is improved; the resource utilization rate of the system is improved; and the job processing reliability is ensured.

The invention provides a data distribution method on the basis of a heterogeneous cluster, which is applied to a plurality of communicated equipment nodes. The method comprises the following steps of reading data block read information of each equipment node and determining required task data, wherein the data block read information comprises data blocks locally calculated when a local task is executed, data blocks read out by other equipment nodes and data blocks read in from the other equipment nodes; according to the read data block read information, predicting calculating ability of each equipment node; according to a predicting result, distributing the determined task data into the local task of each equipment node. According to the invention, by utilizing distribution of the task data on a bottom layer to guide dispatching of tasks on an upper layer and reasonably distributing the data, the calculating ability of each equipment node is matched with the distributed data; moreover, according to the invention, the determined task data is distributed into the local task of each equipment node, so that the problems of network expenses and aggravated network resource scramble which are caused by remote tasks and data movement are solved.

A multi-heterogeneous cluster job unified scheduling method comprises the steps that S1, job proxy services are deployed on multiple clusters, the job proxy services unify input and output data formats of different job scheduling systems, and a unified data interface view is provided; S2, a job service gateway is deployed at the upper layer of the cluster, a plurality of job proxy services are summarized and managed, and a plurality of clusters are uniformly managed; S3, the user obtains the authorization of the operation proxy service, and achieves the unified access to the cluster through the operation proxy service; S4, the user submits jobs, the job service gateway allocates the jobs to specific clusters according to rules, and job files are transmitted locally and among the clusters or among the clusters through job proxy service; and S5, the job proxy service converts the request into a command of a job scheduling system in the cluster for execution, converts a result into unified job management information and returns the unified job management information. Unified job scheduling on the multi-heterogeneous cluster is realized, the job scheduling operation of the multi-heterogeneous cluster is simplified, and computing resources are fully utilized.

The invention provides a heterogeneous cluster service migration transfer system and transfer method based on a container. According to the invention, a service transfer node is arranged in a heterogeneous cluster, a container technology and a process migration technology are combined, and a service operation environment container mirror image library is generated on the service transfer node through calculating service operation environment description on the node in the heterogeneous cluster. When a node in the cluster needs migration service, the service is migrated to a service migration transfer node directly, the service migration transfer node selects a container mirror image with the service environment from a service operation environment libarary, a corresponding container is generated through the container mirror image, and the service is recovered in the container. When communication load is low in the cluster, the service migration transfer node migrates the service to other nodes with the service operation environment in the cluster to continue to operate. The transfer mechanism can effectively reduce the time cost of selecting an available code and the cost for arranging a backup node in service migration, and the continuity of the service migration can be enhanced.

A hierarchical multi-level heterogeneous cluster data system having processing nodes at each of a plurality of cluster levels configured for different data tiers having different availability, accessibility and protection requirements. Each cluster level comprises groups of processing nodes arranged into a plurality of failover domains of interconnected nodes that exchange heartbeat signals to indicate that the nodes are alive and functioning. A master node of each failover domain is connected to a master node of a parent failover domain for exchanging heartbeat signals to detect failures of nodes at lower cluster levels. Upon a network partition, the nodes of the failover domain may be merged into another failover domain at the same or a higher cluster level to continue providing data services. The cluster has a global namespace across all cluster levels, so that nodes that are moved to different failover domains can be accessed using the same pathname.

The invention relates to a heterogeneous rock core three-dimensional reconstruction method based on super dimension. The heterogeneous rock core three-dimensional reconstruction method based on superdimension is based on a three-dimensional modeling problem of a porous medium two-dimensional image, utilizes the advantage that the super dimension uses a real rock core to establish a dictionary, and reconstructs a heterogeneous rock core through prior information. The heterogeneous rock core three-dimensional reconstruction method based on super dimension includes the specific steps: bringing forward introduction of a probability-based image block selection mechanism, establishing the corresponding relationship between a two-dimensional image block and the number of 0 element in a three-dimensional image block in the real rock core in the super dimension reconstruction dictionary, and selecting the elements in the dictionary through a probability selection method so as to maintain the heterogeneous information; at the same time, introducing a mechanism of randomly reconstructing an origin to eliminate the phenomenon that the heterogeneous cluster is diffused to the homogeneous cluster when a heterogeneous rock core is established through an original super dimension reconstruction algorithm; and based on the above steps, providing a heterogeneous reconstruction algorithm. The reconstructed three-dimensional heterogeneous rock core structure can preferably maintain the heterogeneous characteristic of the two-dimensional reference image and provide a good explanation for a realrock core microstructure.