102 results about "Computer cluster" patented technology

In some embodiments, a computer cluster system comprises a plurality of nodes and a software package comprising a user interface and a kernel for interpreting program code instructions. In certain embodiments, a cluster node module is configured to communicate with the kernel and other cluster node modules. The cluster node module can accept instructions from the user interface and can interpret at least some of the instructions such that several cluster node modules in communication with one another and with a kernel can act as a computer cluster.

An end-cloud cooperative data mining method, comprising: determining a target text and a task configuration file according to a business requirement by a cloud end; encoding the target text by a text encoder to obtain a text feature; placing the text feature in the task configuration file and issuing it to a vehicle end together; encoding image data by a first picture encoder to obtain an image feature by the vehicle end; calculating a value of a similarity of the text feature and the image feature; determining a target picture according to the value of the similarity and the task configuration file; and uploading the target picture to the cloud end; wherein the first picture encoder is obtained by compressing and optimizing a second picture encoder; and the text encoder and the second picture encoder are two modules of a picture-text multimodal large model. The application is applied to an automatic driving shadow mode, can complete data mining of any interesting target text by using one large model, and does not need to design and develop detection rules for each type of interesting target. The picture encoder of the vehicle end can be continuously updated and optimized, and the model iteration efficiency is improved.

According to one aspect, a method includes: receiving, by a node of a computer cluster, a command for a health status report, the node having a plurality applications and being one of a plurality of nodes of the cluster, the command indicating a class of a health checks to be executed and a scope; identifying, from a configuration, one or more health checks to be executed based on the indicated class; executing the one or more health checks on one or more of the applications and on one or more of the nodes of the cluster according to the indicated scope, where different ones of the health checks are configured to determine state of different ones of the applications; aggregating, by the node, results of the executed health checks; and outputting the aggregated results.

In some embodiments, a computer cluster system comprises a plurality of nodes and a software package comprising a user interface and a kernel for interpreting program code instructions. In certain embodiments, a cluster node module is configured to communicate with the kernel and other cluster node modules. The cluster node module can accept instructions from the user interface and can interpret at least some of the instructions such that several cluster node modules in communication with one another and with a kernel can act as a computer cluster.

The invention relates to the technical field of simulation, in particular to a simulation method and system for expanding a meta-model of a virtual integrated system, equipment and a medium, by adopting the method provided by the invention, a mechanism for expanding the meta-model is plug-in, simulation modeling and operation scheduling interfaces are separated, and the coupling degree between plug-in function modules is reduced. The modeling scheduler and the engine scheduler can be integrated into the simulation modeling platform and the scheduling platform respectively, and after modeling work is completed in the simulation modeling platform, local simulation can be carried out, and distributed simulation in a computer cluster can also be carried out. The modeling scheduler and the engine scheduler perform interface interaction with the resolver and the actuator in the expansion plug-in respectively, the modeling scheduler and the engine scheduler have respective function responsibilities, corresponding instance handles are created and destroyed through different derived functions, and interface expansion can be performed according to requirements on the premise of being compatible with old interfaces to realize function iteration.

The invention discloses a computer cluster security protection method and system. The method comprises the following steps: collecting multi-dimensional operation data in real time through an agent program deployed at each node; feature parameters are extracted from the data, and a safety evaluation model is adopted to calculate a safety score and map the safety score into a discrete safety level; gradient protection strategies including basic monitoring, enhanced protection and strict isolation strategies are automatically matched and loaded according to the security level; and monitoring a strategy execution effect, and performing closed-loop optimization on the model parameters based on feedback data. The system correspondingly comprises a data acquisition module, a safety evaluation module, a strategy management module, a strategy execution module and a feedback optimization module. According to the method, the problems of static strategy stiffness, inaccurate risk quantification, single response means and lack of self-learning are solved, dynamic self-adaptive protection is realized, the evaluation accuracy is improved, false alarms and missing alarms are reduced, and the cluster environment is ensured to be safe and reliable.

The invention discloses a cluster computing power configuration management and automatic deployment system, which belongs to the technical field of computer cluster management, and comprises a resource state tracking module used for monitoring resource state information of each computing node in a cluster in real time and establishing a resource pool, an exclusive resource pool and a shared resource pool according to the resource state information, the resource pool comprises a first idle node; the resource allocation arbitration module is used for generating a second idle node and a third idle node according to the first idle node; the task resource matching module is used for screening a matched third idle node from the shared resource pool; and the resource recovery coordination module is used for recovering the low-utilization-rate computing node and the first idle node, the second idle node and the third idle node which have completed the computing task into a resource pool. The multi-level resource pool is established through the resource state tracking module, the problem of rigidity of resource allocation of a traditional system is effectively solved, and the computing resource allocation flexibility and utilization rate are remarkably improved.

A method, computer system, and a computer program for managing computer jobs in a queue is provided. This comprises extracting metadata from a new job received for processing and upon determining when a similar enriched metadata exists in a database. A job score and storage footprint may then be determined for the new job from the extracted metadata. It is then determined whether the new job can be grouped for processing with any other jobs already placed on a queue. The new job is then added to the queue based on the new job's score and footprint, and whether it can be grouped with other jobs. The queue is then updated and sent to a scheduler for further processing.

The invention provides a PKS-based resource integration optimization method and device and a storage medium, and relates to the technical field of resource integration, and the method comprises the steps: determining the number of containers needing to be deployed and a resource distribution strategy according to the scale and performance requirements of a computer cluster, and configuring a virtual network according to the network requirements of a training system, the method comprises the following steps: establishing a container cloud platform, establishing a storage resource pool according to a data type of an accessed target device, allocating corresponding computing resources to an application program based on the container cloud platform, configuring a load balancer and setting a load balancing algorithm on the container cloud platform, and when an operation request of a user on the target device is received, sending the operation request to the target device; the operation request is allocated to the matched computing node for request analysis, the working condition of each application program based on the container cloud platform and the use condition of each storage resource pool are counted in real time, and visual display is performed, so that efficient operation and stable function exertion of the training system are effectively ensured.

The embodiment of the invention discloses a computer cluster parallel processing method for a reverse time migration program, which comprises the following steps that: a main node carries out parallel processing according to a list file containing to-be-processed shot information in a global disk, a flag file in a first global file directory and a flag file in a second global file directory; determining a specified gun and distributing a gun data processing task corresponding to the specified gun to the computing node; the first global file directory is used for storing mark files of shots which are subjected to reverse time migration processing on each computing node, and the second global file directory is used for storing mark files of shots which are subjected to reverse time migration processing on each computing node; and the computing node responds to the received shot data processing task, calls a single-shot reverse time migration program to execute reverse time migration processing for the corresponding specified shot, and updates the flag file in the first global file directory and / or the second global file directory according to the processing state. In this way, the processing efficiency of the multi-shot reverse time migration task can be improved.

The invention relates to the technical field of computer cluster communication, in particular to a set communication method suitable for a core particle equipment cluster with unified bus interconnection, which comprises the following steps of: acquiring hierarchical topology and link information of the core particle equipment cluster, and constructing a global topology information base according to the acquired hierarchical topology and link information; optimizing the core size level primitive, the node level primitive and the super node level primitive; a link monitoring unit is deployed to monitor a link state index in real time, an alarm is triggered when a link fault is detected, an optimal backup link is screened based on a global topology information base, a communication path is updated, and a dynamic resource scheduling algorithm is adopted to allocate computing power and communication resources; and dynamically adjusting a communication path and bandwidth allocation according to a set communication task requirement and a link dynamic parameter. The method is suitable for a three-level link architecture of a core particle equipment cluster, can improve the communication efficiency and the bandwidth utilization rate, enhances the cluster reliability and the resource utilization rate, and effectively meets the requirements of high computing power and high communication intensity of large model training.

Provided is a processing system that executes a process flow in which computation results are aggregated after data is divided and processed. This processing system comprises: a computer cluster 10 that is composed of a plurality of computation units which each process divided data; a DPU cluster 20 that is composed of at least one DPU which aggregates computation results; and a cross-connect switch 30 that connects devices. A setting device enumerates process flows which can be established by the processing system while changing the number l of interfaces of a computation unit to be used and the number m of DPUs to be used, combines the process flows such that data communication time in the processing system is minimized, and sets the cross-connect switch 30 such that a combination of the process flows can be executed.

In some embodiments, a computer cluster system comprises a plurality of nodes and a software package comprising a user interface and a kernel for interpreting program code instructions. In certain embodiments, a cluster node module is configured to communicate with the kernel and other cluster node modules. The cluster node module can accept instructions from the user interface and can interpret at least some of the instructions such that several cluster node modules in communication with one another and with a kernel can act as a computer cluster.

The application provides a resource fragment consolidation method, a controller, a control node and a computer cluster, and relates to the technical field of data processing. The method simulates resource scheduling according to a first resource capacity and a first task currently being executed to obtain a plurality of candidate resource scheduling paths; determines a target resource scheduling path from the plurality of candidate resource scheduling paths; and schedules the first task based on the target resource scheduling path to perform resource fragment consolidation. After each candidate resource scheduling path is used to perform resource fragment consolidation, the residual resource capacity of at least one node can reach the first resource capacity. That is, the method can actively eliminate resource fragments in the computer cluster by scheduling the first task to perform resource fragment consolidation. Compared with the method of slowing down the generation of new resource fragments from the source, the scheduling reliability of a task with high resource demand can be effectively improved.

The application relates to the technical field of data processing, and discloses a computer cluster data processing method and device, the method comprises the following steps: acquiring cluster data, wherein the cluster data is any one of continuous storage data and dispersed storage data; performing chain layer storage and analysis on a computer cluster according to the cluster data to obtain a cluster data block; wherein the computer cluster refers to a computer executable program running on a cluster storage structure; performing vectorization coding according to the cluster data block to obtain a data block vector; performing mapping according to the data block vector to obtain mapping data which can be operated on a computer; and inputting the mapping data into a data calculation formula to perform calculation processing, so as to obtain a processed data linked list. The method can solve the problem of low data processing efficiency in the prior art due to the limited capacity of structured modeling for mass data processing.

The application discloses a dual-computer high-availability implementation method and device, equipment and a medium, relates to the technical field of computers, and is applied to a Keepalived service tool of any service node in a dual-computer cluster, and comprises the following steps: acquiring a current node state of a local service node; determining a current script corresponding to the current node state from a plurality of preset scripts; wherein the plurality of preset scripts comprise a first preset script corresponding to a node state being a master node state, a second preset script corresponding to the node state being a backup node state, and a third preset script corresponding to the node state being a Keepalived service hanging state; and executing the current script. When the node state is the master node state, the backup node state and the Keepalived service hanging state respectively, the corresponding current script is automatically executed, so that the operation and maintenance of the local service node is implemented, and the high availability and stability of the dual-computer cluster can be ensured.

A device-cloud collaborative data mining method includes: A cloud determines a target text and a task configuration file based on a service requirement; encodes the target text by using a text encoder to obtain a text feature; and places the text feature in the task configuration file and delivers the task configuration file to a vehicle. The vehicle encodes image data by using a first picture encoder to obtain an image feature; calculates a value of a similarity between the text feature and the image feature; determines a target picture based on the value of the similarity and the task configuration file; and uploads the target picture to the cloud. The first picture encoder is obtained by compressing and tuning a second picture encoder, and the text encoder and the second picture encoder are two modules of a large image-text multimodal model. This application is applied to an autonomous driving shadow mode, and data mining of any target text of interest can be completed by using a large model, without designing and developing a detection rule for each type of target of interest. In this way, a picture encoder on the vehicle may be continuously updated and optimized, to improve model iteration efficiency.

A system including a management application and a satellite container. The management application is configured to manage workload operations of a customer computer cluster and a plurality of third-party compute systems. The satellite container and a customer computer cluster are on a customer network. Further, the satellite container is configured to: provide configuration data of the local computer cluster and authorization data to the management application; provide a first data request to the management application; receive workload data from the management application in response to the first data request; and convey the workload data from the satellite to the local computer cluster. The management application is outside the customer network.

The invention discloses a server-based data synchronization method and system and a storage medium. The method comprises the following steps: each group of quantum security privacy computers in a quantum security privacy computer cluster transmits generated encrypted data to a switch through respective connected terminals; the switch sends the encrypted data to a server cluster, a server in the server cluster receives the encrypted data and forwards the encrypted data to a main server network element, the main server network element decrypts the encrypted data and forwards the decrypted data to the server, and the server forwards the decrypted data to a business server; and the main server network element also classifies and stores the decrypted data, and synchronizes the data to the hot standby server network element. According to the method, the data are classified firstly, and then different modes are selected according to the types of the data to synchronize the data, so that the workload of direct synchronization of all the data is reduced; and the data and the application part are decoupled and backed up respectively, and the data and the application part do not influence each other, so that the data backup efficiency is improved.

Examples of currently disclosed technology include end-to-end systems for automatically training, validating, selecting, and deploying predictive models in response to rapidly changing data trends. Such an end-to-end system includes edge computing resources that stream data directly from customer data sources and detect drifts between predictions from predictive models deployed at the edge computing resources and corresponding time series data derived from the streamed data. The end-to-end system also includes a central computing resource (e.g., a centralized, cloud-based computer cluster) that responds to the drift detections by automatically training and validating instances of stored predictive models using fresh time series data derived from the streamed data.The fresh time series data can be logically grouped into subsets of time series data metrics, with each subset of time series data metrics associated with a common customer subsystem.

A plurality of different virtualization system images are configured for deployment to a plurality of nodes in heterogeneous environments. Individual ones of the virtualization system images are configured such that once deployed, the nodes form a computer cluster having a storage pool that is shared across the nodes. When configuring the virtualization system images, information that describes the heterogeneous computing environments is accessed, and constraints pertaining the heterogeneous computing environments are reconciled in advance of configuring the different virtualization system images. A common subnet across the heterogeneous environments is established. The plurality of different virtualization system images are configured to access the common subnet once deployed. The common subnet serves as a storage I / O communication path over which a cluster-wide storage pool is implemented. The virtualization system images are configured to correspond to address portions of a contiguous address space that is used to access data in the storage pool.

The embodiment of the invention provides a distributed query method and related equipment. The distributed query method comprises the following steps: in response to an SQL task submitted by a submitting end, splitting the SQL task to obtain a plurality of SQL sub-tasks; executing each SQL sub-task in a distributed manner through each cluster node to obtain a corresponding sub-task data result, the cluster nodes being nodes of the computer cluster; and aggregating the subtask results to obtain a final query result. According to the technical scheme, the SQL query task submitted by the submitting end is intelligently split, the complex SQL query task is decomposed into a plurality of subtasks capable of being processed in parallel, and the query speed is remarkably increased through the parallel computing capacity of a computer cluster. When the subtasks are executed, the split SQL subtasks are distributed to a plurality of computing nodes to be executed, parallel execution mechanisms of some cluster frames can process a plurality of tasks at the same time, and therefore the overall processing time is shortened.

The invention discloses a code optimization method and device, and relates to the technical field of computers. The code optimization method comprises the steps that a computing device obtains a code comprising at least one first loop statement, then determines a data division mode according to a dependency relationship among data indicated by the first loop statement, and converts the first loop statement into a second loop statement according to the division mode to obtain an optimized code. Wherein the division mode comprises one or a combination of the following modes: slice data division used for indicating to divide the data in a single dimension, and block data division used for indicating to divide the data in any multiple dimensions. The second loop statement can be processed in parallel by at least two processes on a computer cluster, and different processes are used for respectively processing slice data or block data obtained after data division, so that the data division efficiency is improved, and the development efficiency of parallel codes is improved.

The present application provides a method for allocating test tasks, a computer-readable storage medium, and a processor. The method includes: obtaining multiple scripts to be tested; generating a test task using at least one script to be tested, wherein the test task includes at least one script to be tested with a label; and allocating the script to be tested with a label in the test task to a predetermined server. In this solution, by defining allocation rules for specific scripts by adding labels to the scripts to be tested, the scripts to be tested with labels can be automatically allocated, reducing the labor cost and time cost of manual allocation. In addition, when there are idle servers, the tasks to be tested without labels can be allocated to idle servers, thereby realizing dynamic and automated allocation of test tasks, reducing the waiting time for automated execution of test tasks, and ensuring that computer cluster resources can be fully and reasonably used.

In some embodiments, a computer cluster system comprises a plurality of nodes and a software package comprising a user interface and a kernel for interpreting program code instructions. In certain embodiments, a cluster node module is configured to communicate with the kernel and other cluster node modules. The cluster node module can accept instructions from the user interface and can interpret at least some of the instructions such that several cluster node modules in communication with one another and with a kernel can act as a computer cluster.

The invention relates to a rail transit scheduling method based on multi-computer cluster parallel offline solution, and the method comprises the steps: generating a to-be-updated urban rail train arrival time set and a to-be-updated urban rail train departure time set according to an urban rail train arrival time initial value, an urban rail train departure time initial value and a preset value range; forming a power supply optimization model according to the to-be-updated urban rail train arrival time set and the to-be-updated urban rail train departure time set; forming a passenger flow optimization model according to the passenger flow number, the to-be-updated urban rail train arrival time set and the to-be-updated urban rail train departure time set; a host acquires a to-be-updated urban rail train arrival time set and a to-be-updated urban rail train departure time set, and sends the sets to a plurality of slaves; and the plurality of slaves carry out parallel calculation on the power supply optimization model and the passenger flow optimization model. According to the method and the device, the problems of low speed and low efficiency of urban rail train operation scheduling optimization are solved, and rapid optimization of operation scheduling is realized.

The application provides a data processing method and device, equipment, a storage medium and a program product. The method comprises: obtaining a first logical view of an input first tensor, wherein a first logical dimension arrangement of the first logical view comprises a batch dimension, a sequence dimension and a hidden state dimension in sequence; performing first rearrangement of the first logical view based on a second logical dimension arrangement to obtain a second logical view of the first tensor, wherein the second logical dimension arrangement comprises the sequence dimension and a merged dimension in sequence, and the merged dimension is a product of the batch dimension and the hidden state dimension; and performing set communication on the first tensor in the sequence dimension based on the second logical view. The application can reduce the memory usage of data processing in a computer cluster, improve the speed of data processing, and improve the performance of the computer cluster.