47 results about "Computer cluster" patented technology

An end-cloud cooperative data mining method, comprising: determining a target text and a task configuration file according to a business requirement by a cloud end; encoding the target text by a text encoder to obtain a text feature; placing the text feature in the task configuration file and issuing it to a vehicle end together; encoding image data by a first picture encoder to obtain an image feature by the vehicle end; calculating a value of a similarity of the text feature and the image feature; determining a target picture according to the value of the similarity and the task configuration file; and uploading the target picture to the cloud end; wherein the first picture encoder is obtained by compressing and optimizing a second picture encoder; and the text encoder and the second picture encoder are two modules of a picture-text multimodal large model. The application is applied to an automatic driving shadow mode, can complete data mining of any interesting target text by using one large model, and does not need to design and develop detection rules for each type of interesting target. The picture encoder of the vehicle end can be continuously updated and optimized, and the model iteration efficiency is improved.

The invention discloses a computer cluster security protection method and system. The method comprises the following steps: collecting multi-dimensional operation data in real time through an agent program deployed at each node; feature parameters are extracted from the data, and a safety evaluation model is adopted to calculate a safety score and map the safety score into a discrete safety level; gradient protection strategies including basic monitoring, enhanced protection and strict isolation strategies are automatically matched and loaded according to the security level; and monitoring a strategy execution effect, and performing closed-loop optimization on the model parameters based on feedback data. The system correspondingly comprises a data acquisition module, a safety evaluation module, a strategy management module, a strategy execution module and a feedback optimization module. According to the method, the problems of static strategy stiffness, inaccurate risk quantification, single response means and lack of self-learning are solved, dynamic self-adaptive protection is realized, the evaluation accuracy is improved, false alarms and missing alarms are reduced, and the cluster environment is ensured to be safe and reliable.

The invention discloses a cluster computing power configuration management and automatic deployment system, which belongs to the technical field of computer cluster management, and comprises a resource state tracking module used for monitoring resource state information of each computing node in a cluster in real time and establishing a resource pool, an exclusive resource pool and a shared resource pool according to the resource state information, the resource pool comprises a first idle node; the resource allocation arbitration module is used for generating a second idle node and a third idle node according to the first idle node; the task resource matching module is used for screening a matched third idle node from the shared resource pool; and the resource recovery coordination module is used for recovering the low-utilization-rate computing node and the first idle node, the second idle node and the third idle node which have completed the computing task into a resource pool. The multi-level resource pool is established through the resource state tracking module, the problem of rigidity of resource allocation of a traditional system is effectively solved, and the computing resource allocation flexibility and utilization rate are remarkably improved.

A method, computer system, and a computer program for managing computer jobs in a queue is provided. This comprises extracting metadata from a new job received for processing and upon determining when a similar enriched metadata exists in a database. A job score and storage footprint may then be determined for the new job from the extracted metadata. It is then determined whether the new job can be grouped for processing with any other jobs already placed on a queue. The new job is then added to the queue based on the new job's score and footprint, and whether it can be grouped with other jobs. The queue is then updated and sent to a scheduler for further processing.

The embodiment of the invention discloses a computer cluster parallel processing method for a reverse time migration program, which comprises the following steps that: a main node carries out parallel processing according to a list file containing to-be-processed shot information in a global disk, a flag file in a first global file directory and a flag file in a second global file directory; determining a specified gun and distributing a gun data processing task corresponding to the specified gun to the computing node; the first global file directory is used for storing mark files of shots which are subjected to reverse time migration processing on each computing node, and the second global file directory is used for storing mark files of shots which are subjected to reverse time migration processing on each computing node; and the computing node responds to the received shot data processing task, calls a single-shot reverse time migration program to execute reverse time migration processing for the corresponding specified shot, and updates the flag file in the first global file directory and / or the second global file directory according to the processing state. In this way, the processing efficiency of the multi-shot reverse time migration task can be improved.

The invention relates to the technical field of computer cluster communication, in particular to a set communication method suitable for a core particle equipment cluster with unified bus interconnection, which comprises the following steps of: acquiring hierarchical topology and link information of the core particle equipment cluster, and constructing a global topology information base according to the acquired hierarchical topology and link information; optimizing the core size level primitive, the node level primitive and the super node level primitive; a link monitoring unit is deployed to monitor a link state index in real time, an alarm is triggered when a link fault is detected, an optimal backup link is screened based on a global topology information base, a communication path is updated, and a dynamic resource scheduling algorithm is adopted to allocate computing power and communication resources; and dynamically adjusting a communication path and bandwidth allocation according to a set communication task requirement and a link dynamic parameter. The method is suitable for a three-level link architecture of a core particle equipment cluster, can improve the communication efficiency and the bandwidth utilization rate, enhances the cluster reliability and the resource utilization rate, and effectively meets the requirements of high computing power and high communication intensity of large model training.

In some embodiments, a computer cluster system comprises a plurality of nodes and a software package comprising a user interface and a kernel for interpreting program code instructions. In certain embodiments, a cluster node module is configured to communicate with the kernel and other cluster node modules. The cluster node module can accept instructions from the user interface and can interpret at least some of the instructions such that several cluster node modules in communication with one another and with a kernel can act as a computer cluster.

The application relates to the technical field of data processing, and discloses a computer cluster data processing method and device, the method comprises the following steps: acquiring cluster data, wherein the cluster data is any one of continuous storage data and dispersed storage data; performing chain layer storage and analysis on a computer cluster according to the cluster data to obtain a cluster data block; wherein the computer cluster refers to a computer executable program running on a cluster storage structure; performing vectorization coding according to the cluster data block to obtain a data block vector; performing mapping according to the data block vector to obtain mapping data which can be operated on a computer; and inputting the mapping data into a data calculation formula to perform calculation processing, so as to obtain a processed data linked list. The method can solve the problem of low data processing efficiency in the prior art due to the limited capacity of structured modeling for mass data processing.

A device-cloud collaborative data mining method includes: A cloud determines a target text and a task configuration file based on a service requirement; encodes the target text by using a text encoder to obtain a text feature; and places the text feature in the task configuration file and delivers the task configuration file to a vehicle. The vehicle encodes image data by using a first picture encoder to obtain an image feature; calculates a value of a similarity between the text feature and the image feature; determines a target picture based on the value of the similarity and the task configuration file; and uploads the target picture to the cloud. The first picture encoder is obtained by compressing and tuning a second picture encoder, and the text encoder and the second picture encoder are two modules of a large image-text multimodal model. This application is applied to an autonomous driving shadow mode, and data mining of any target text of interest can be completed by using a large model, without designing and developing a detection rule for each type of target of interest. In this way, a picture encoder on the vehicle may be continuously updated and optimized, to improve model iteration efficiency.

A plurality of different virtualization system images are configured for deployment to a plurality of nodes in heterogeneous environments. Individual ones of the virtualization system images are configured such that once deployed, the nodes form a computer cluster having a storage pool that is shared across the nodes. When configuring the virtualization system images, information that describes the heterogeneous computing environments is accessed, and constraints pertaining the heterogeneous computing environments are reconciled in advance of configuring the different virtualization system images. A common subnet across the heterogeneous environments is established. The plurality of different virtualization system images are configured to access the common subnet once deployed. The common subnet serves as a storage I / O communication path over which a cluster-wide storage pool is implemented. The virtualization system images are configured to correspond to address portions of a contiguous address space that is used to access data in the storage pool.

The embodiment of the invention provides a distributed query method and related equipment. The distributed query method comprises the following steps: in response to an SQL task submitted by a submitting end, splitting the SQL task to obtain a plurality of SQL sub-tasks; executing each SQL sub-task in a distributed manner through each cluster node to obtain a corresponding sub-task data result, the cluster nodes being nodes of the computer cluster; and aggregating the subtask results to obtain a final query result. According to the technical scheme, the SQL query task submitted by the submitting end is intelligently split, the complex SQL query task is decomposed into a plurality of subtasks capable of being processed in parallel, and the query speed is remarkably increased through the parallel computing capacity of a computer cluster. When the subtasks are executed, the split SQL subtasks are distributed to a plurality of computing nodes to be executed, parallel execution mechanisms of some cluster frames can process a plurality of tasks at the same time, and therefore the overall processing time is shortened.

In some embodiments, a computer cluster system comprises a plurality of nodes and a software package comprising a user interface and a kernel for interpreting program code instructions. In certain embodiments, a cluster node module is configured to communicate with the kernel and other cluster node modules. The cluster node module can accept instructions from the user interface and can interpret at least some of the instructions such that several cluster node modules in communication with one another and with a kernel can act as a computer cluster.

The application provides a data processing method and device, equipment, a storage medium and a program product. The method comprises: obtaining a first logical view of an input first tensor, wherein a first logical dimension arrangement of the first logical view comprises a batch dimension, a sequence dimension and a hidden state dimension in sequence; performing first rearrangement of the first logical view based on a second logical dimension arrangement to obtain a second logical view of the first tensor, wherein the second logical dimension arrangement comprises the sequence dimension and a merged dimension in sequence, and the merged dimension is a product of the batch dimension and the hidden state dimension; and performing set communication on the first tensor in the sequence dimension based on the second logical view. The application can reduce the memory usage of data processing in a computer cluster, improve the speed of data processing, and improve the performance of the computer cluster.

The invention discloses a multi-modal high-speed data storage method for a potassium collecting machine, which relates to the technical field of data storage, and comprises the following steps: selecting key value storage according to the storage demand of multi-modal data of the potassium collecting machine, and defining the structure of a key matched with the type of the multi-modal data, each multi-modal data item needing to be stored has a unique corresponding key; uniformly distributing the multi-modal data items on different nodes in a computer cluster matched with the potassium collecting machine by using a consistent Hash algorithm so as to realize load balancing; after the multi-modal data item is allocated to the target node, writing an operation log record of the multi-modal data item into a disk; writing the multi-modal data item into a memory cache; and finally, flashing the multi-modal data in the memory cache into a disk according to a preset period. The number of the virtual nodes can be intelligently adjusted according to the change of the system, so that better load balancing and system performance are realized, and the flexibility, the stability and the reliability of the system are improved.

A process receives a batch of one or more first job requests to be executed by a high-performance computing (HPC) cluster. This batch of first job requests is received by a container orchestration platform. The batch of one or more first job requests is translated into one or more second job requests. The second job requests are interpretable by a scheduler corresponding to the HPC cluster. The second job requests are then sent to the scheduler.

The application relates to a cluster type safety computer platform, which is centrally deployed, can realize centralized maintenance and is favorable for improving system availability. The safety computer platform is deployed on a track side and does not need to bear the severe working environment of a vehicle-mounted device, is high in reliability and can adopt high-performance hardware, thus being favorable for realizing more intelligent functions. The safety computer units in the safety computer platform communicate based on a wired network, and compared with the wireless communication between a vehicle-mounted safety computer platform and a track-side safety computer platform in a traditional train control system, the wired communication is better in reliability. A master and standby database cluster is introduced, the statelessness of safety applications can be realized, data synchronization during master and standby switching of the safety applications is avoided, the switching speed is accelerated, and the consistency of data during the master and standby switching process is ensured. The large-scale safety computer cluster platform is realized, and the problems existing in real-time performance and determination of virtualization on the safety computer platform do not need to be faced.

The embodiment of the application provides a database operation method, system and computer device, the method is applied to the cloud platform, and comprises the following steps: a coordination node receives a data operation request sent by a user end through a target session; the coordination node determines whether a session temporary table created for a global temporary table and belonging to the target session exists at present, wherein metadata corresponding to the session temporary table is stored in a disk of a data node; in the case that it is determined that the session temporary table does not exist, the coordination node creates the session temporary table in a target data node in a plurality of data nodes, and sends the data operation request to the target data node; in the case that it is determined that the session temporary table exists, the coordination node sends the data operation request to the target data node, and the data operation request is used to instruct the target data node to perform data operation on the session temporary table according to the target data node. By adopting the database operation method provided by the application, the stability of the distributed database can be improved, and resource waste can be avoided.

A system and method of processing datasets that are compliant to a layer 2 (L2) protocol of an open systems interconnection (OSI) model in a cloud computing environment on behalf of an edge computing environment. The method includes receiving, from the edge computing device, a set of L2 frames. The method includes detecting one or more errors in the set of L2 frames based on an L2 protocol of the OSI model. The method includes correcting the one or more errors in the set of L2 frames to generate a corrected set of L2 frames. The method includes generating, by a processing device of a cloud computing system, a processed dataset based on the corrected set of L2 frames.

The application discloses a kind of based on loop current method's power electronic system cluster simulation implementation method, it is related to power electronic technical field.The application includes: based on loop current method's power electronic model cluster distribution initialization method and cluster computing scheduling method, wherein model distribution initialization method will large-scale power electronic model be distributed to each computing node according to the local circuit after decoupling of loop current method and complete the initialization of local circuit model simulation on each node, wherein cluster computing scheduling method realizes the multi-level parallel computing scheduling and synchronous process of main node to computing node in cluster.The application is suitable for large-scale power electronic simulation based on supercomputer or high-performance computer cluster, fully utilizes the advantage of large memory capacity of computer cluster to expand the scale of power electronic model simulation, fully utilizes the advantage of many processor core quantity of computer cluster to improve the efficiency of power electronic model simulation.

The invention discloses a large-scale high-throughput task query request processing method and system, and relates to the field of computer cluster scheduling. The large-scale high-throughput task query request processing system comprises a system architecture design module, a core processing mechanism module, a key configuration parameter setting module and a service operation process module. The large-scale high-throughput task query request processing system has the following technical advantages that the scheduling performance is improved through an architecture decoupling and resource isolation mechanism; the performance interference of lock competition, thread switching and frequent I / O operation in a traditional scheme on a scheduler is eliminated through architecture decoupling, the actual measurement scheduling delay is reduced by 40%-60%, the process creation overhead is reduced through a main control process merging request mechanism and maxrequest sonfork parameter regulation and control, the system throughput is improved, the resource utilization rate is optimized, and the system performance is improved. The dual memory cleaning strategy periodically cleans and keeps the timeliness to balance the query real-time performance and memory resource occupation, and the completed task is prevented from residing for a long time.

The invention discloses an optimization and principle system of an ODS management platform, and belongs to the technical field of bank data management. The system comprises a database server, a computer cluster, a data stream processing center, a data access service module, a data processing assembly and an infrastructure assembly, and an ODS serves as a database system center point and is connected with application servers through ODSH online data service. According to the system, a perfect ETL control flow mechanism is established and comprises three key steps of extraction, conversion and loading, and unified management of metadata and configuration information is achieved through SDM, FDM and ADM management libraries of an ODS processing module. Event driving is adopted as a job scheduling core, and various event types such as data file arrival, data stream arrival, database table readiness and service request arrival are supported. According to the method, the system operation quantity can be reduced by more than 40%, the data landing frequency is remarkably reduced, the bank information security requirement is completely met, and the operation efficiency and the management level of the ODS system are effectively improved.

The present invention discloses a method for processing urban rainstorm waterlogging simulation data based on distributed computing, which includes the following steps: uploading a digital surface model to a control node server, the control node server splitting the digital surface model into data chunks with the same number as the computing node servers according to the number of computing node servers, and distributing N data chunks to N computing node servers for processing; the computing node servers completing simulation calculations according to the rainstorm waterlogging model tool and feeding back the results to the node server; the control node server summarizing and completing the fusion and splicing of the rainstorm waterlogging simulation results according to the processing results of each computing node server. The present invention overcomes the problem of high time consumption of massive geographical data in the application of urban rainstorm waterlogging simulation and calculation, utilizes the rich computing resources provided by the computer cluster, realizes the parallel preprocessing of geographical data and the collection of calculation results, and solves the most time-consuming calculation link in waterlogging simulation.

The invention provides a method for designing a large-aperture large-view-field broadband achromatic super lens, which comprises the following steps of: by simulating the structure of a traditional fisheye lens, mapping incident light with different view field angles to different unit areas on the surface of the super lens after passing through a diaphragm, and optimizing an equivalent lens system by utilizing optical design software to correct large-view-field aberration. On the basis, a hybrid algorithm framework and a forward mapping algorithm are constructed, a super-large-scale unit structure library is screened and arranged, the calculation complexity of a traditional point-by-point compensation method is reduced from O (N2) to O (N), consumption of calculation resources is greatly reduced, the calculation speed is increased, the calculation precision is improved, and the calculation efficiency is improved. And the design of the large-aperture broadband large-view-field achromatic super lens independent of a large computer cluster is realized.

The present application relates to the technical field of computer cluster communication, and discloses a collection communication method suitable for a core grain device cluster interconnected by a unified bus, which comprises the following steps: collecting hierarchical topology and link information of the core grain device cluster, and constructing a global topology information database according to the collected hierarchical topology and link information; optimizing core grain level primitives, node level primitives and supernode level primitives; deploying a link monitoring unit to monitor link state indicators in real time, triggering an alarm when a link fault is detected, screening an optimal backup link based on the global topology information database and updating a communication path, and allocating computing power and communication resources by using a dynamic resource scheduling algorithm; and dynamically adjusting the communication path and bandwidth allocation according to link dynamic parameters according to the collection communication task demand. The present application is suitable for a three-level link architecture of a core grain device cluster, can improve communication efficiency and bandwidth utilization, enhance cluster reliability and resource utilization, and effectively adapt to the high computing power and high communication intensity demand of large model training.

The invention provides a data processing method and device, equipment, a storage medium and a program product. The method comprises the steps that a first logic view of an input first tensor is acquired, and first logic dimension arrangement of the first logic view sequentially comprises a batch dimension, a sequence dimension and a hidden state dimension; the first logic view is subjected to first rearrangement based on second logic dimension arrangement, a second logic view of the first tensor is obtained, the second logic dimension arrangement sequentially comprises a sequence dimension and a merging dimension, and the merging dimension is a product of a batch dimension and a hidden state dimension; set communication is performed on the first tensor in the sequence dimension based on the second logical view. According to the method, the memory utilization rate of data processing in the computer cluster can be reduced, the data processing speed is increased, and the performance of the computer cluster is improved.

The embodiment of the invention provides an alarm message notification template processing method and computing equipment. The method is applied to a computer cluster, a monitoring component runs in the computer cluster, the monitoring component is used for managing a target alarm message notification template, and the method comprises the following steps: monitoring a configuration change event, and when it is monitored that the configuration change event indicates that an instance generated based on CRD is changed, generating a target alarm message notification template based on the configuration change event; executing a target operation on a target configuration file corresponding to the target alarm message notification template; and processing the target alarm message notification template based on the target operation, so that the target alarm message notification template becomes effective. By adopting the method, the CRD definition is introduced, and the alarm notification template is modeled into the native resource object, so that management and full-life-cycle automatic operation and maintenance of the target alarm message notification template are realized, dynamic updating can be realized without manual intervention, and the configuration efficiency of the alarm service is remarkably improved.

The invention discloses a management method of a computer cluster and computer equipment. The computer cluster comprises a plurality of nodes. The method comprises the steps of obtaining state information of a plurality of nodes when receiving a routing inspection instruction of a computer cluster, and if it is determined that a target node in a fault state and a plurality of non-target nodes in a normal state exist in the plurality of nodes based on the state information of the plurality of nodes, when the target node comprises an initial main node, sending the initial main node to the computer cluster; and obtaining a to-be-upgraded slave node and a target slave node from the plurality of non-target nodes based on the state information of the non-target nodes, upgrading the to-be-upgraded slave node into a target master node, and controlling the target master node and the target slave node to perform master-slave replication. According to the technical scheme, the computer node with the fault can be recovered in time, and the recovery hysteresis of the computer node with the fault is reduced.

The invention relates to the technical field of industrial automation control, in particular to an industrial cooperative control system and an industrial cooperative control method, and aims to solve the technical problems of resource redundancy, low data collaboration efficiency and insufficient expansibility of a traditional industrial control system. In order to achieve the purpose, the industrial cooperative control system comprises a centralized industrial personal computer cluster and an edge end real-time control unit, and the centralized industrial personal computer cluster virtualizes a plurality of hardware resources including processors and storages into a uniform resource pool; the uniform resource pool comprises a plurality of virtual industrial control units divided based on a virtual container technology, and the virtual industrial control units are used for generating a cooperative control instruction to realize cooperative control of the industrial production line. And the edge end real-time control unit controls the station to complete a real-time task corresponding to a cooperative control instruction based on the cooperative control instruction of the virtual industrial control unit to the station of the industrial production line, so that the problems of resource redundancy, scattered distribution, high cost, data island and the like of a traditional distributed industrial personal computer are solved.