534 results about "Cluster Node" patented technology

The invention relates to the technical field of data analysis, and discloses a GPU computing power resource scheduling method and system, and the method comprises the steps: collecting node hardware parameters and dynamic load indexes of a GPU cluster to construct a multi-dimensional resource feature vector of the GPU cluster, and constructing a resource portrait of the GPU cluster; establishing a node health degree scoring model of the GPU cluster, and generating a health degree score of a cluster node corresponding to the GPU cluster; analyzing a video memory demand of the GPU task request, and calculating an intensive identifier and a communication dependency relationship; determining the SLA weight of the GPU task request, calculating the resource shortage sensitivity of the GPU task request based on the video memory demand, and calculating the target task priority of the GPU task request in combination with the SLA weight; and determining a resource scheduling node group requested by the GPU task in the resource portrait, generating resource scheduling parameters of the resource scheduling node group, and executing scheduling of computing power resources of the GPU cluster based on the resource scheduling parameters. According to the method, the scheduling efficiency of the GPU computing power resources can be improved.

The invention discloses an unmanned aerial vehicle cluster interaction method and system based on an ad hoc network, and belongs to the technical field of unmanned aerial vehicle communication and cluster control. The method comprises the following steps: acquiring positioning information, energy parameters and task types of an unmanned aerial vehicle group to generate a clustering node set; predicting a cluster moving direction based on the motion acceleration data of the clustering node set, and generating a dynamic path planning instruction; allocating a main communication channel and a standby channel according to the dynamic path planning instruction, and generating a channel allocation result; monitoring a signal quality parameter of the main communication channel according to a channel allocation result, and triggering a link switching instruction when interference is detected; and constructing an interference thermodynamic diagram based on the topological relation of the clustering node set, and generating a power adjustment instruction. According to the invention, through dynamic clustering, mobile prediction, adaptive channel management, interference sensing switching and topology power control, the communication stability, reliability and resource efficiency of the unmanned aerial vehicle cluster in a complex environment are improved.

The invention discloses a cluster computing power energy efficiency perception scheduling and green computing system, which relates to the technical field of computers and comprises a multi-source energy efficiency perception and data acquisition module used for acquiring power consumption, utilization rate, temperature, cooling state, PUE index and environmental data of cluster nodes. According to the invention, through the multi-modal energy efficiency fusion sensing network and the multi-scale convolution and time sequence attention fusion network, multi-source heterogeneous energy efficiency data such as current, voltage, temperature, airflow and the like of a node level can be collected and fused in real time and with high precision, noise is effectively removed, abnormity self-correction is realized, the defect of energy efficiency sensing granularity in the prior art is made up, and the energy efficiency sensing precision is improved. And reliable input is provided for subsequent scheduling decisions. A cross-scale dynamic twinborn collaborative modeling mechanism is adopted, a physical information neural network and a computational fluid mechanics model are coupled, optimization is carried out through a generative adversarial network structure, and accurate prediction of a complex energy consumption evolution curve and a cooling flow field is achieved.

The invention discloses an optimized registration method and system based on a micro-service architecture. The invention relates to the technical field of Nacos systems. The health monitoring center is started and starts to send a health examination request to a downstream service at regular time; the gateway access layer service receives a client request; obtaining a network address of a target service instance through a Nacos service discovery mechanism according to a service name in the request; then forwarding the request to a target service instance for processing; when the configuration information is changed, the Nacos configuration management center pushes an update notification to all subscribed service instances; the service instance pulls the latest configuration information and updates the local cache; according to the method, a secondary hand waving mechanism is implemented in Nacos cluster node, gateway service and downstream service communication, so that the offline condition of the downstream service can be accurately judged, and the problem of data inconsistency caused by node faults or network partition is avoided.

The invention discloses a training reasoning method of a decision model, a product, electronic equipment and a medium, and relates to the technical field of artificial intelligence. In the method, when the training reasoning process of the decision model is divided into a plurality of calculation tasks to be distributed to different cluster nodes, the load pressure on the nodes is reduced; and the plurality of data subsets obtained by splitting the input data corresponding to the calculation task are allocated to different calculation nodes, so that the load pressure on the calculation nodes is reduced, the utilization rate of node resources is improved, parallel processing of the data subsets corresponding to the input data is realized, and the efficiency of training reasoning is improved. When the training reasoning process of the decision model is divided into one calculation task, the plurality of data subsets obtained by splitting the input data corresponding to the calculation task are distributed to different calculation nodes, so that the load balance on the calculation nodes is ensured, the load is reasonably distributed to the nodes in the cluster, and the calculation efficiency is improved. And the utilization rate of node resources is improved.

The invention discloses a user-noninductive cold and hot sensing remote memory exchange system, which belongs to the field of computer storage, and is characterized in that cluster nodes are divided into extensible remote memory service nodes and computing client nodes according to roles; the remote memory node comprises a remote memory exchange server and a memory resource registration module; the computing client node comprises a FrontSwap-based remote memory exchange client, and a remote memory exchange server side is registered as a memory resource which is non-inductive to a user; furthermore, popularity statistics based on a popularity histogram is realized in a kernel mode; the memory exchange behavior is guided through the dynamic decision-making module based on online reinforcement learning, the memory management efficiency and the system performance are remarkably improved, and efficient and non-inductive memory expansion service is provided for user application.

The invention provides a chip system and an access method, relates to the technical field of chips, and improves the data access performance of a high-performance computing system. According to the specific scheme, the chip system comprises a snoop filter, a plurality of processor cluster nodes and a plurality of memories, the processor cluster nodes are coupled through a bus, the snoop filter is coupled with the bus, the processor cluster nodes are in one-to-one correspondence with the memories, and the snoop filter adopts a unified directory format. The snoop filter is used for obtaining an access request initiated by the processor cluster node, and the access request comprises address information. The snoop filter is further used for determining an access mode of the access request based on the address information, wherein the access mode comprises a unified memory access mode and a non-unified memory access mode. The processor cluster node is to access the memory based on the access pattern and the access request. The embodiment of the invention is used for the process of accessing the memory by the processor cluster node.

The invention discloses an unmanned aerial vehicle cluster security collaboration method, system and device, a medium and a product. The unmanned aerial vehicle cluster comprises a plurality of pre-divided clusters, each cluster comprises a pre-selected first cluster head node and a plurality of in-cluster child nodes, and the credibility of the first cluster head node meets a preset condition and is larger than that of the in-cluster child nodes. According to the method, secure communication and data sharing among unmanned aerial vehicle cluster nodes are realized through a consensus mechanism, and the security and reliability of an unmanned aerial vehicle cluster are improved; an unmanned aerial vehicle is divided into a plurality of clusters under an unmanned aerial vehicle cluster hierarchical network, the plurality of clusters work cooperatively, and an unmanned aerial vehicle cluster consensus mechanism is divided into an intra-cluster consensus mechanism and a lightweight inter-cluster consensus mechanism by means of hierarchical network characteristics of an unmanned aerial vehicle cluster. The cluster head communication pressure is relieved, communication congestion between cluster head nodes is relieved, the communication complexity is reduced, and the communication efficiency of the unmanned aerial vehicle cluster is improved.

The invention relates to a cross-region computing power cooperation system and method. The method comprises the following steps: acquiring computing power resource information of computing power cluster nodes of each region in a global range; obtaining a cluster pairing strategy according to the computing power resource information of the computing power cluster nodes of each region in the global range, and obtaining a logical computing power cluster group according to the cluster pairing strategy; computing power cluster nodes in the computing power cluster group are used for mutual cooperative computing; acquiring a task processing request, wherein information in the task processing request comprises a service attribute, a computing power demand and a data volume; and according to the task processing request and the computing power resource information of the computing power cluster nodes in the computing power cluster group, determining a target computing power cluster node matched with the service attribute, the computing power demand and the data volume from the computing power cluster group, and sending the task processing request to the target computing power cluster node. The method can improve the utilization rate of global computing power resources.

The invention discloses an inference service deployment method, electronic equipment and a storage medium, and the method comprises the steps: responding to the creation of an inference service, reading a model information configuration table and an engine information configuration table, and carrying out the matching processing of the model information configuration table and the engine information table according to a pre-configured model name during the creation of the inference service, processing resources, inference engine mirror image identifiers and inference engine starting parameters which can be used for loading inference services on the container scheduling platform are obtained, and workload metadata used for bearing the inference services are created based on the processing resources, the inference engine mirror image identifiers and the inference engine starting parameters; the deployment of the inference service on the container scheduling platform is realized by distributing the workload metadata to the target cluster node. Through the method, the technical problem of relatively low deployment efficiency caused by manually configuring the parameters in the workload metadata depending on manpower in related technologies is solved, and the technical effects of simplifying the deployment process of the model reasoning service and improving the deployment efficiency are achieved.

The invention relates to the technical field of industrial Internet of Things and wireless sensor networks, in particular to a large-scale industrial fault diagnosis system and method based on Bluetooth MESH, and efficient monitoring and fault early warning of industrial equipment are realized through low-power-consumption sensor nodes, edge computing, a Bluetooth Mesh network and a clustered Mesh tree architecture. The system comprises a client, a server, a gateway, a cluster head and a cluster node, the cluster node comprises a sensor node and a relay node, a lightweight neural network model is built in the sensor node, and edge reasoning and fault detection can be realized locally. Stable communication and real-time cooperation of large-scale nodes are ensured through a decentralized architecture of Bluetooth Mesh, establishment of a dynamic cluster head chain and optimization of an RSSI threshold value. And a low-power-consumption mechanism combining event driving and fixed polling is adopted, so that the energy consumption of the system is remarkably reduced. In-cluster communication loads are reduced through a clustering architecture, and the communication efficiency and the anti-interference capability of the system are improved in combination with channel separation of Mesh and BLE protocols.

The invention discloses a K8s-based cluster computing power resource isolation and scheduling method and system. The method comprises the following steps: when creating a cluster account, dividing the cluster account according to resource groups; node Selector directional scheduling is combined to realize tenant cluster resource isolation; the custom controller monitors a cluster user binding resource group event; and the user-defined controller monitors the cluster node update event to modify the resource group. According to the method, the resource group can effectively divide the resources in the current cluster, corresponding tenants are provided according to different computing power requirements, and nodes, accounts, labels and the like in the resource group are effectively managed by relying on a K8s CRD self-defined resource mechanism; the characteristic that a daemon process set ensures that a Pod runs on all nodes under a cluster is utilized, a service runs on the Pod to execute a GPU script to read the card number, and a K8s API is called to create a tag marked by the card number, so that subsequent resource group division is facilitated.

The invention provides a YARN cluster node resource dynamic adjustment method and system based on dual-mode perception, and relates to the technical field of data processing, and the method comprises the steps: collecting system index data and application program index data, respectively constructing two deep neural network models for fusion evaluation, calculating a resource dynamic adjustment coefficient, and obtaining a resource dynamic adjustment result; node resources are updated in real time by adopting a sliding time window mechanism, and a multi-dimensional resource allocation optimization model is constructed by utilizing a genetic algorithm to realize optimal resource allocation. According to the invention, the utilization rate of cluster resources can be improved, resource competition is reduced, and intelligent dynamic adjustment of node resources is realized.

The invention discloses an MQTT message transmission optimization method and system. The method comprises the steps of analyzing a theme, extracting a device type, a data feature and a geographic position triple, calculating a hash value, and mapping a device to a specified Broker fragment cluster node to generate a fragment mapping table; processing the equipment data in the edge domain in the fragment mapping table through an edge calculation layer, removing invalid data according to a preset rule, merging the equipment data in the same fragment node, and embedding a fragment node ID for an aggregation message generated after merging; identifying a fragment node ID and routing to a target fragment node, positioning a corresponding shared memory pool, and writing the message into the shared memory pool; and responding to a direct access request of the client, so that the client directly accesses the data from the shared memory pool through the user mode network stack. According to the method, the problem of uneven load is effectively solved, multiple times of state switching in the data transmission process is avoided, and the MQTT message transmission efficiency is improved while the transmission cost is reduced.

The invention provides an underwater dynamic network resource scheduling and time slot conflict optimization method, which comprises the following steps of: firstly, acquiring network state characteristics by using underwater sensor nodes to perform cluster head election and cluster member allocation; secondly, dynamically optimizing the sending priority of the nodes in the cluster by utilizing a Q-learning theory, and carrying out time slot allocation; constructing a reward function based on intra-cluster node distribution information, and iteratively and dynamically adjusting the sending priority of the nodes; and finally, a time slot allocation conflict detection mechanism is formulated to avoid resource competition and transmission conflicts among the nodes in the cluster, so that the optimal node allocation scheme is gradually converged. According to the method, the problems of communication conflicts among multiple cluster nodes of the underwater network, low resource allocation efficiency, inaccurate dynamic sending queues and the like are comprehensively considered, the node sending priority and time slot allocation are optimized through Q-learning, and the channel utilization rate is effectively improved. In a complex dynamic underwater network environment, the technology has applicability and effectiveness, and an underwater node sending strategy can be optimized more reasonably.

The invention relates to the technical field of power loads, in particular to a power load dynamic optimization prediction method based on reinforcement learning, which comprises the following steps: acquiring real-time voltage frequency power of nodes, tracking frequency difference direction change of adjacent nodes, marking reverse disturbance to generate a distribution map, and identifying a frequent disturbance area according to the distribution map; clustering node groups with consistent fluctuation trends to determine a target area; monitoring power fluctuation inversion to lock energy steering nodes; constructing a prediction input set; comparing prediction and actual trends to extract a deviation interval, adjusting stride and rate of a reinforcement learning model, updating a decision and smoothing a curve, and outputting a power load dynamic prediction curve. According to the method, disturbance space-time dynamic identification is realized through multi-layer correlation analysis, judgment precision is improved through frequency power joint calibration, load path consistency and area extension are reflected through node clustering, power transmission tracking is enhanced through energy node identification, a feedback closed loop is constructed through offset monitoring, and curve continuity and response rate are improved. Stable convergence is predicted to be consistent with the trend under multiple disturbances.

In some embodiments, a computer cluster system comprises a plurality of nodes and a software package comprising a user interface and a kernel for interpreting program code instructions. In certain embodiments, a cluster node module is configured to communicate with the kernel and other cluster node modules. The cluster node module can accept instructions from the user interface and can interpret at least some of the instructions such that several cluster node modules in communication with one another and with a kernel can act as a computer cluster.

The invention relates to the technical field of high-availability clusters, and discloses an edge-side-based cluster node data processing method and system, and the method comprises the steps: obtaining edge-side cluster node resources, querying a hardware performance parameter and an idle time period of each dynamic access cluster node, and constructing a node association information group; extracting network condition prediction information of each dynamic access cluster node, and planning a dynamic leader duration in a target cluster data processing period by considering node dynamic access distribution information corresponding to a node association information group of each dynamic access cluster node and a constraint set constructed by the network condition prediction information; an election strategy is generated and is used for executing free interval election when a target operation service is executed, so that the real-time dynamic adjustment of the master control node of the high-availability cluster considering the influence of multiple factors is realized, and the high-availability cluster is helped to perform free release and node switching of the master control node before a fault occurs; and the operation stability and the processing performance of the high-availability cluster are improved.

The invention relates to the technical field of big data, and discloses a massive small file reading optimization method, system and device and a medium, and the method comprises the following steps: scanning small files in a distributed file system by using a Spark calculation engine to obtain metadata information of the small files; based on the metadata information, small files are classified, and the small files with the same type and data relevance are classified into one group; the resource states of cluster nodes are monitored in real time, the resource states comprise the processor utilization rate, the memory occupancy rate and the I / O load, the load score of each node is calculated, and small file task quotas are dynamically allocated; executing small file merging operation in parallel by utilizing a Spark memory calculation engine to generate a merged file containing a plurality of original small files; creating and maintaining index information in an external database, and recording the position and attribute of each original small file in the combined file; and providing a read-write interface through the special service layer, and positioning and accessing specific small file contents in the combined file.

The invention discloses a distributed database built-in monitoring system, belongs to the technical field of databases, and aims to solve the technical problem of how to overcome the defects of complex monitoring deployment, high data response delay, coarse information acquisition granularity and low system integration in the prior art. According to the technical scheme, a monitoring platform module is integrated with a database kernel, a front-end and rear-end separation framework is adopted, and visual display and centralized management and control of the database operation state are achieved; the data acquisition module is used for simultaneously starting with a database process through a built-in HTTP (Hyper Text Transport Protocol) server to realize native support of a monitoring function; the distributed state query module is used for discovering cluster node states through a Gossip protocol, supporting node health detection and remote calling state service, and supporting gRPC-based streaming communication and concurrent processing; and the time sequence data management module is used for storing, compressing and querying the collected time sequence monitoring data.

The invention relates to a data collection and retrieval distributed system and method and computer equipment. Comprising a front-end application used for collecting initial data through a multi-coroutine concurrent execution database; the ES cluster is used for performing index recovery according to the operation conditions of different cluster nodes and retrieving the target data through an inverted index; the back-end application performs service registration and discovery based on the micro-service framework, and provides information calling associated with the initial data or the target data for the front-end application; and the message middleware is used for dynamically adjusting the process of data collection and data retrieval. According to the method and the device, quick retrieval can be realized, and data consistency under a distributed condition can be ensured.

The invention discloses a distributed task processing method and device based on spatio-temporal data complexity and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: extracting a feature vector of spatio-temporal data of a digital twin model, and inputting the feature vector into a trained random forest model; through the trained random forest model, performing classification prediction on the spatio-temporal data according to the feature vector, and outputting a category label corresponding to the feature vector, the category label representing the complexity of the spatio-temporal data; mapping the category labels of the spatio-temporal data into weight coefficients, and formulating distributed task queues based on the weight coefficients, so that the difference of the sum of weights of the task queues of each processing node is minimum to ensure load balance among the distributed task queues; and distributing the distributed task queue to each cluster node for distributed processing. The technical effect of improving the distributed task processing efficiency is achieved.

The invention discloses a grouping clustering method and application of a distributed storage system. The method comprises the steps that nodes in a target cluster are grouped and belong to M sub-clusters, and M is larger than or equal to 2; respectively distributing corresponding borne services for the M sub-clusters, and limiting service access among the sub-clusters; the metadata service center of the target cluster is deployed in a metadata service sub-cluster, the metadata service sub-cluster is one of M sub-clusters, and the metadata service sub-cluster comprises a main node and a standby node for operating the metadata service center of the target cluster. According to the method disclosed by the invention, the target cluster nodes are grouped, the services are allocated to the sub-clusters, and the access among the sub-clusters is limited, so that the services can be effectively isolated in the target cluster, and the security of important information data of an enterprise is ensured. The metadata service center is deployed in a metadata service sub-cluster and comprises a main node and a standby node, so that the reliability of storage service can be ensured.

This application provides a radio access network architecture. The radio access network architecture includes a cluster node and a serving node. The serving node provides task scheduling and executing functions, and the cluster node provides a region-level centralized collaboration function for the serving node and a collaboration function between cross-region cluster nodes.

The invention discloses a set communication control method and device for distributed training and a medium, and the method comprises the steps: reducing data on all calculation nodes in a first cluster to a specified calculation node when the difference of the number of calculation nodes between any two clusters in clusters participating in data reduction is within a preset range, the number of the specified computing nodes is the same as the number of the computing nodes in the second cluster with the minimum number of the computing nodes. Controlling the specified computing node and the computing nodes in the second cluster to perform data reduction; and controlling data synchronization between the specified computing node and other nodes except the specified computing node in the first cluster. Therefore, other clusters except the cluster with the minimum number of computational nodes are reduced once in the cluster, the data are reduced to the specified computational nodes with the number the same as that of the minimum nodes in each cluster, it is guaranteed that the nodes of each cluster are the same during cross-cluster reduction, and it is avoided that some nodes are reduced with multiple nodes at the same time; and the set communication overhead is reduced.

The invention discloses a Presto task scheduling method and a Presto task scheduling device. According to the method, after a task request is received, tasks are allocated to matched clusters according to user information and task information in the request, then task types are further analyzed, execution modes of the tasks are determined, whether the tasks belong to task types needing to exclusively share resources or not is judged, and if the tasks belong to resource exclusive tasks, the tasks are allocated to the clusters. Tasks are allocated to reserved single tasks and special resource nodes, and when the tasks exclusively share working node resources, no other types of tasks are executed on the nodes, so that the tasks exclusively share the resources on specified cluster nodes, physical isolation of the working nodes in a cluster is realized, the performance problem caused by resource contention is avoided, and the service life of the cluster is prolonged. Therefore, the method is suitable for a complex query environment shared by multiple users, efficient execution of the important tasks can be ensured even when a large number of tasks are processed, and the overall reliability and stability of the system are improved.

The invention discloses a resource allocation system and method based on an MLOps platform, and the system comprises an access controller which obtains a currently executed calculation task, verifies the calculation task, and carries out the access of the calculation task; the scheduler is used for acquiring resource use conditions corresponding to the cluster nodes in multiple dimensions and determining resource load conditions of the cluster nodes according to the resource use conditions; according to the resource load condition, distributing a corresponding cluster node for the calculation task; the control manager is used for managing the execution process of the calculation task; and the command line interface interacts with the user. Through a scheduler, according to a resource load condition, a resource request and limitation of a calculation task (MLOps task) are adjusted, a dynamic resource allocation algorithm is realized, college and optimal utilization of resources are ensured, and full utilization or excessive allocation of resource niches is prevented.

The invention discloses a cluster node management method and device, equipment and a medium in the technical field of computers. According to the invention, after the server accesses the cluster, full-automatic configuration and management from access to operation can be realized. Specifically, after access, hardware configuration information of the server is determined by using a file system command; matching application configuration information according to the hardware configuration information, and performing application configuration on the server according to a configuration file generated by the application configuration information; and monitoring the running state of the server, and managing hardware equipment, an operating system and container applications of the server. According to the scheme, the cluster nodes do not need to be configured manually, the management cost and the manual operation risk of the cluster nodes are reduced, and efficient management of the cluster nodes is achieved.

The invention relates to the technical field of deep learning, and discloses a fine-grained pipeline scheduling method and device based on cluster node memory difference perception. The method comprises the following steps: estimating the ratio of the peak memory occupancy to the memory capacity of each GPU node, comparing the ratio with a preset threshold value, and selecting a re-calculation strategy and a back propagation segmentation strategy: by taking minimization of the end-to-end training time of a model as a target, scheduling and modeling micro-batch data as a flow shop problem, analyzing the dependency constraint of each flow line stage, and calculating the flow shop problem; comprising forward-back propagation sequence dependence, stage dependence, operation dependence and memory limitation, and generating a micro-batch operation sequence scheduling scheme meeting the memory capacity limitation; according to the scheduling scheme, an execution sequence queue is created by executing sorting, and the execution time sequence of calculation blocks, communication blocks and re-calculation operation is dynamically coordinated. According to the method, the GPU equipment utilization rate can be remarkably improved, and the end-to-end training completion time of the model is shortened.

The invention belongs to the field of data processing and finance, and relates to a message processing method based on a pressure test, which comprises the following steps: constructing a message body according to service attributes and request content data extracted from a pressure test request; when it is judged that the processing load of the message body exceeds a preset threshold value according to the historical processing data, the message features of the message body are added to a message construction rule; extracting a message body key feature from the pressure measurement data and the online monitoring data to adjust a message construction rule; generating a message according to the new message construction rule, and initiating a message request; dynamically adjusting the number of cluster nodes according to the concurrency, obtaining messages corresponding to the message requests based on the processing sequence, and distributing the messages to the cluster nodes according to a load balancing algorithm. The invention further provides a message processing device and equipment based on the pressure test and a storage medium. In addition, the invention also relates to a block chain technology, and request content data can be stored in a block chain. The message body is dynamically constructed, and the message production speed is controlled.