37 results about "Consistent hashing" patented technology

The invention relates to the technical field of industrial internet security communication, and discloses a lightweight encryption method and system for an industrial real-time data stream, and the method comprises the following steps: intercepting an original data packet, and carrying out the deep packet detection to extract a network layer quintuple and application layer metadata; matching an optimal strategy in a strategy rule set according to the feature vector and generating a scheduling instruction; in response to the instruction, calling a corresponding pre-compilation password operation pipeline to carry out differential encryption processing on the load; and constructing a fixed-length security policy head containing the algorithm template identifier, and packaging and sending the fixed-length security policy head. According to the invention, task fragmentation is carried out by using a consistent Hash technology so as to ensure processing order-preserving. By adopting a lightweight message structure without handshake negotiation and a multi-level strategy scheduling mechanism, content-based fine-grained security protection is realized, and the communication requirements of low delay and high certainty are met while the security of industrial control data is ensured.

The invention relates to an Internet of Things data security sharing method for blockchain fragmentation. The method comprises the following steps: firstly, performing identity anonymous authentication based on an auditing mechanism combining encryption packaging and zero-knowledge proof; thirdly, performing threshold slicing on the to-be-stored data by adopting an IDA slicing algorithm, performing dynamic allocation of storage nodes based on weighted consistency hash, and performing data storage and distribution based on the allocated storage nodes; afterwards, a PBFT consensus mechanism based on behavior reputation is adopted to carry out security auditing of data recovery and storage, a recovery voucher is generated, and on-chip data recovery is carried out; and finally, querying a target fragment of data distribution based on a trusted relay cross-fragment transaction mechanism, and completing cross-fragment data recovery in the target fragment. According to the invention, high efficiency and security of IoT data sharing and terminal identity privacy protection can be realized, and application requirements in a large-scale IoT environment are well met.

The present application relates to a device scheduling access method and system, a readable storage medium, and a program product. The method comprises: in response to a session request sent by a communication device, performing consistent hashing processing on a device code of the communication device by means of a target forwarding server to obtain a hash value; on the basis of a correspondence between the hash value and a signaling service identifier, determining a target signaling service identifier corresponding to the hash value; and distributing the session request to a target signaling server corresponding to the target signaling service identifier. By using the method, on the basis a finer-grained load policy, requests of a same communication device can be ensured to be all processed on a same signaling service instance, thereby implementing platform statelessness and efficient load balancing.

The application discloses a message distribution method and device, equipment and product, and relates to the technical field of communication. The message distribution method is applied to a non-central architecture publishing and subscribing system, the publishing and subscribing system comprises a subscriber cluster and an arbitration center, and the message distribution method comprises the following steps: determining the subscription priority of a plurality of subscribers in the subscriber cluster corresponding to a subscription topic based on a consistent hashing algorithm and the subscription topic; determining a first type of subscriber according to a preset lease mechanism based on the arbitration center and the subscription priority; and distributing the message of the subscription topic to the first type of subscriber, so that the first type of subscriber performs consumption. Through the above scheme, the efficiency of message distribution can be effectively improved, and efficient load balancing without centralization is realized.

The invention discloses a DNS (Domain Name Server) cache optimization system based on multi-node collaboration, which is suitable for public DNS, operator recursion, CDN (Content Delivery Network) scheduling and cloud native scenes. The system is composed of an edge recursion node, a shared second-level cache layer, a cooperative forwarding and near-source rollback module, a distributed active refreshing and lease cooperation module, a cache key generation and routing module and a toughness mechanism module. The domain name and the client network segment are taken as main keys, and the ECS and the two-stage consistent Hash are combined to realize request viscosity and fragmentation stability; when a local miss occurs, preferentially forwarding to a same network segment / same city-same operator / same region collaborative node, and then returning to share cache and authority; a coordination node is elected and refreshed according to domain name fragments, active refreshing of hotspots and immediate entries is driven by lease + priority, and group frightening is avoided through push / pull diffusion; toughness capabilities such as request merging, fusing degradation, health detection and the like are provided, and automatic near-source fallback and removal of abnormities are achieved. According to the method, the hit rate and the first packet delay can be remarkably improved.

The invention discloses an industrial monitoring intelligent alarm method, device and system based on dynamic speech synthesis, and a storage medium. The method comprises the following steps: receiving structured alarm event data of a front end; generating a to-be-synthesized voice text according to the data and a preset template; calling a locally deployed voice synthesis engine to process the voice text to generate voice audio data; and transmitting the audio data stream to a front end for playing. Wherein when the speech synthesis engine executes the synthesis task, the index file is compiled in real time to support hot update of the term library. And meanwhile, in the speech synthesis process, a consistent Hash algorithm is adopted to allocate tasks and automatically expand a process pool when the load is high, and meanwhile, a three-stage fault-tolerant mechanism of synthesis overtime switching, template error degradation and engine crash restarting is provided. The problems that an existing industrial voice alarm scheme is poor in flexibility and intranet safety, term hot update, high concurrency and low delay cannot be considered at the same time are solved, and real-time and dynamic generation and high-reliability broadcasting of thousand-level concurrency alarm voice in the domestic industrial intranet environment are achieved.

The embodiment of the invention provides a data processing method, electronic equipment, a storage medium and a program product. The data processing method comprises the following steps: acquiring a data volume relationship of to-be-Hash distributed data corresponding to all associated nodes; and balancing the data volume of the to-be-Hash distributed data corresponding to the associated nodes based on the data volume relationship, so that the data volume of the to-be-Hash distributed data of each associated node is kept within a preset range threshold. According to the scheme of the embodiment, it is guaranteed that the data size corresponding to each association node is kept uniform, so that uniform hash distribution is achieved on the basis of consistent hash distribution, and the situation that some association nodes in one association subsystem are overloaded and some association nodes are overloaded is avoided.

The application discloses an information security data storage system and relates to the technical field of data storage.The application solves the technical problem of avoiding the problem of resource waste or performance bottleneck caused by one-size-fits-all storage by intelligently selecting local storage or distributed storage mode according to the type, capacity size and delay requirement of data.The application dynamically selects local or distributed storage based on data capacity and delay requirement, stores small-capacity and low-delay data locally to reduce network overhead, stores large-capacity and high-delay data in a distributed manner to improve scalability, improves resource utilization, automatically deletes temporary data, combines high-frequency / low-frequency data differentiation processing, takes into account data security and space optimization, dynamically adapts different capacity data through a dynamic sharding strategy, ensures balanced distribution of data through consistent hashing and virtual nodes, and realizes dynamic load adjustment through cold / hot data layering and node performance grading, thereby improving access speed and system stability.

The application provides a state compression consistent hashing method supporting arbitrary removal of an evidence storage node, comprising: constructing a state compression layer for recording a removal state of the storage node; when a storage node is removed, updating the removal state through the state compression layer with constant time complexity; based on the removal state recorded by the state compression layer, addressing a data key, and based on a stateless consistent hashing engine, performing primary mapping on the data key to obtain a first node number; when the storage node corresponding to the first node number is in a working state, outputting the first node number as an addressing result; or when the storage node corresponding to the first node number is in a removal state, triggering secondary mapping, recalculating the first node number in a current working storage node set, and outputting a second node number as an addressing result. The method realizes high efficient search performance and low memory overhead while supporting arbitrary random removal of the storage node.

The invention discloses a distributed complex task parallel processing system and method. The system comprises a global task scheduling and coordination module, a task division and distribution module, distributed parallel computing nodes, a result aggregation module and a strong consistency verification and fault tolerance module. According to the system, dynamic fragmentation and node mapping of tasks are achieved by calculating task feature fingerprints and adopting a consistent Hash algorithm, and only data migration in the minimum range is needed during node capacity expansion and contraction; by setting a blocking fence synchronization point and a periodic verification mechanism, high consistency and reproducibility of a global task result are realized. And when node faults or inconsistent results are detected, the system automatically rolls back to the last successful fence check point and performs task rescheduling, so that the completeness and stability of calculation are ensured. The method has remarkable advantages in concurrency performance, consistency guarantee and fault-tolerant recovery, and is suitable for scenes such as large-scale data analysis, scientific simulation and real-time decision calculation.

The application provides a stock selection system based on a distributed efficient index formula, comprising a front-end agent service module, a plurality of sharding calculation service modules and a data source service module. The front-end agent service module receives a user request containing a plate factor and an index formula, acquires a benchmark stock pool list requiring stock selection according to the plate factor, divides the benchmark stock pool list requiring stock selection into a plurality of shards by taking the modulus of an incremental internal code and the total number of shards, and generates a corresponding shard calculation request; a shard calculation node management module in the front-end agent service module distributes the request to the corresponding shard calculation service module according to the shard identifier by using an adaptive consistent hashing algorithm. The application separates the front-end agent and the shard calculation by combining the MapReduce task distribution and the result aggregation mechanism, improves the real-time performance and the concurrent processing capability of stock selection calculation, dynamically adjusts the distribution of virtual nodes and entity nodes by using the adaptive consistent hashing strategy, solves the problem of unbalanced sharding, and supports node elastic scaling.

This invention discloses a task allocation method and device. The method includes: if the target node is the master node, then when the periodic inspection time arrives, acquiring information on defined tasks, information on delegated tasks, information on a first set of nodes with delegated tasks, and information on a second set of available nodes; if the information on defined tasks differs from the information on delegated tasks, and / or the information on the first set differs from the information on the second set, then acquiring a first latest set of available nodes and a first latest set of full tasks; based on the first latest set of available nodes and the first latest set of full tasks, performing a task reallocation operation using a consistent hashing algorithm to obtain a first mapping relationship between nodes and delegated tasks; and writing the first mapping relationship into a database node. This task allocation method reduces task execution latency, improves the throughput of the distributed scheduling system, and enhances the reliability and resource utilization during the task allocation process.

The application discloses an audio integrated network resource virtualization method, and belongs to the technical field of audio network resource management, and comprises the following steps: constructing a distributed virtual switching layer of a covering subnet, embedding a virtual switch instance into a kernel state network stack of each physical host in a cluster, and dynamically mapping cloud hosts in the subnet to the physical host group through a consistent hashing algorithm; when the cloud host sends an audio data packet, the source physical host virtual switch instance performs the following steps: extracting a DSCP field and marking it as EF, encapsulating through a GRE-over-UDP tunnel, and replacing a target MAC address with a virtual MAC address of a next-hop virtual switch instance; after the virtual switch instance of the target physical host is unpacked, the audio packet is delivered to the target cloud host based on a virtual port forwarding table. Through the distributed virtual switching layer and the consistent hashing mapping mechanism, the method realizes efficient virtualization transmission of audio data, supports QoS guarantee and low-delay transmission, and is suitable for audio network resource management in a large-scale cloud environment.

This invention discloses a high-concurrency access control system and method for multi-terminal collaboration in construction management. The system includes: a terminal access authentication module, which performs identity authentication and role classification on multi-terminal devices, derives corresponding terminal classifications, and assigns corresponding role weight coefficients; a dynamic hierarchical rate limiting module, which calculates the token generation rate and token bucket capacity based on a preset dynamic hierarchical token bucket algorithm and performs token verification and rate limiting control on concurrent access requests from multiple terminals according to the role weight coefficients; a distributed scheduling module, which uses consistent hashing distributed scheduling rules to distribute access requests that pass rate limiting verification, and simultaneously completes multi-terminal session state synchronization based on a distributed session cluster; and a data conflict resolution module, which performs two-factor conflict detection on triggered operations on the same data item, performs conflict resolution, and completes data read / write and returns an access response. Its beneficial effects are: making differentiated rate limiting more reasonable and automatically adapting to on-site network fluctuations.

Techniques are described for enabling uninterrupted services by computing resources on nodes of a consistency hash ring (CHR) while adding or removing nodes (i.e., making changes) of the CHR. In some embodiments, a duplicate of the existing CHR (i.e., old version) is created to become a new version for performing the changes. Two versions of consistent hash rings (CHRs) co-exist during the transition period of making changes. In some embodiments, computing resources on the nodes of these CHRs perform version upgrades and data migration while continuing to service client requests without interruption.

A method and device for scaling a distributed database and a storage medium, comprising: in the case of meeting a scaling condition, scaling the distributed database using a consistent hashing algorithm; during the scaling, suspending a write request from a client for a target physical node; obtaining feature information of the suspended write request according to a preset strategy; in the case of the obtained feature information reaching a feature threshold, suspending the scaling, and processing a new write request from the client for the target physical node and the suspended write request whose feature information reaches the feature threshold, until the scaling is resumed in the case of re-meeting a scaling preparation condition. The embodiments of the present disclosure enable the distributed database to still provide normal services during scaling, thereby avoiding service interruption of the distributed database and improving service availability of the business distributed database.

Systems, methods, and network attached storage nodes for secure peer-to-peer file sharing using consistent hashing for distributed ownership data. A plurality of network attached storage nodes may be interconnected by a network and configured for peer-to-peer communication without a centralized server. When a node receives a user data file, it divides the file into data chunks and determines hashes for those data chunks. It uses the chunk hashes to map the data chunks to other nodes for storage. It also sends ownership data for the file that includes an owner identifier, a hash key, the set of chunk hashes.

This application discloses a distributed device time-series data storage method, device, and medium, relating to the field of data storage technology. The method includes: receiving time-series data generated by an IoT device in real time, and acquiring the data generation rate of the IoT device and the real-time storage status of the distributed storage nodes; dynamically planning the write path to the target storage node corresponding to the time-series data based on the data generation rate and real-time storage status; performing data sharding on the time-series data based on a preset consistent hashing algorithm, and writing the time-series data to the target storage node through the target storage node write path based on the data sharding results; creating a backup copy of the time-series data through a multi-replica mechanism, and monitoring the health status of the target storage node to switch the access path of the time-series data to the backup copy when the target storage node fails.

The invention discloses a multi-modal high-speed data storage method for a potassium collecting machine, which relates to the technical field of data storage, and comprises the following steps: selecting key value storage according to the storage demand of multi-modal data of the potassium collecting machine, and defining the structure of a key matched with the type of the multi-modal data, each multi-modal data item needing to be stored has a unique corresponding key; uniformly distributing the multi-modal data items on different nodes in a computer cluster matched with the potassium collecting machine by using a consistent Hash algorithm so as to realize load balancing; after the multi-modal data item is allocated to the target node, writing an operation log record of the multi-modal data item into a disk; writing the multi-modal data item into a memory cache; and finally, flashing the multi-modal data in the memory cache into a disk according to a preset period. The number of the virtual nodes can be intelligently adjusted according to the change of the system, so that better load balancing and system performance are realized, and the flexibility, the stability and the reliability of the system are improved.

The invention discloses a distributed real-time database high availability method, system and device of a new energy centralized control station system and a storage medium. Based on a distributed architecture, a real-time data writing strategy, a real-time data reading strategy of the distributed architecture, a high-availability strategy after host exit of the distributed architecture and a high-availability strategy after host recovery of the distributed architecture are constructed, and respective storage nodes and hot standby nodes are created for a plurality of hosts to form a complete real-time library. According to the method, the acquisition, storage and reading of the large-capacity real-time database of the new energy centralized control station system are realized by utilizing load balancing, consistent hash, node heartbeat monitoring and failover, the high availability of real-time data is realized, and the reliability of the whole monitoring system is improved.

A business rule hot update method based on quantitative analysis and dynamic arbitration, relating to the field of distributed computing, is disclosed. The method includes: quantifying the impact of data changes on business metrics through rule impact strength network analysis, triggering incremental recalculation only for high-impact changes; routing recalculation tasks to a fixed subset of arbitration nodes using consistent hashing, ensuring strong consistency of results through multi-node parallel computing and load-aware arbitration; storing both arbitration and old versions of the results, and dynamically adjusting the canary release ratio based on a PID feedback controller according to real-time risk indicators to achieve adaptive risk control and second-level automatic rollback. This invention solves the problems of service interruption, data inconsistency, and high release risk in traditional solutions, achieving minute-level, accurate, and secure hot updates of business rules, making it particularly suitable for high-requirement scenarios such as e-commerce and finance.

The invention provides an alarm computing load balancing method, which belongs to the technical field of cloud computing and distributed monitoring operation and maintenance, adopts a distributed architecture, and comprises a plurality of Prometheus examples, a scheduling management module, a task distributor, an equal division algorithm executor and a shared storage component. The scheduling management module dynamically fragments the alarm rules according to rule types, data source features or time dimensions, and adjusts the number and size of fragments according to real-time loads. And the equal division algorithm executor combines the load, the performance index and the resource utilization rate of each instance, adopts strategies such as consistent Hash, polling or weighted polling to calculate an assignment relationship, and supports newly-added rule allocation and fragment migration. Dynamic management and real-time synchronization of rules are achieved through the shared storage component, stateless deployment of Prometheus instances is achieved, requests are distributed through the load balancer, and therefore the load balancing capacity, the rule management efficiency and the alarm timeliness are improved.

The invention discloses a flow load balancing algorithm for a multi-node encryption device driven by a control machine, which aims at encryption service requirements in key fields, adopts a'centralized decision-making and distributed execution 'framework and comprises the control machine, a flow distributor and an encryption node cluster. The control machine periodically collects node multi-dimensional state data and calculates a comprehensive load evaluation value; extracting encrypted request features, calculating the matching degree of nodes and requests, and dynamically allocating traffic (state keeping requests are forwarded by using consistent Hash, and stateless requests are allocated according to probability). The system has fault detection isolation and gray recovery mechanisms, and algorithm parameters are optimized through a feedback learning module. According to the algorithm, the resource utilization rate and throughput are improved, the response time delay is reduced, the system reliability and adaptability are enhanced, and the high-reliability and high-throughput encryption service requirements are met.

This invention discloses a hierarchical storage system and method for permissionless blockchains. The system includes: a hot node layer for storing hot data of the blockchain; a cold node layer for storing both hot and cold data of the blockchain; and an on-chain smart contract, including: a dynamic data management module for periodically mapping cold data blocks to the cold nodes for long-term storage using a consistent hashing algorithm; a storage proof verification module for verifying the continuous storage status of the cold data blocks by the cold nodes and obtaining a verification result; and an incentive settlement module for automatically executing economic incentives on the cold nodes based on the verification result and feeding back the results of the economic incentives to the dynamic data management module. This invention addresses the storage bloat problem of permissionless blockchains at the protocol level, significantly reducing storage costs, promoting decentralization, and improving the security and scalability of permissionless blockchain networks.

Techniques are described for enabling uninterrupted services by computing resources on nodes of a consistency hash ring (CHR) while adding or removing nodes (i.e., making changes) of the CHR. In some embodiments, a duplicate of the existing CHR (i.e., old version) is created to become a new version for performing the changes. Two versions of consistent hash rings (CHRs) co-exist during the transition period of making changes. In some embodiments, computing resources on the nodes of these CHRs perform version upgrades and data migration while continuing to service client requests without interruption.

The application relates to a server cluster load balancing method based on improved consistent hashing and Redis, which comprises the following steps of: firstly, constructing a modular cluster architecture containing a user management layer, a load balancing layer and the like; collecting node hardware performance parameters and real-time load data through a cluster state monitoring layer; dynamically calculating the number of virtual nodes based on parameters such as node performance ratio in the load balancing layer, mapping a hash ring by using an improved consistent hashing algorithm, locating a target physical node, and distributing a request in combination with a dynamic scheduling strategy; realizing efficient switching of a master and a backup through an election queue mechanism in a cluster high-availability layer, and completing redundant backup and synchronous sharing of Session data by means of a Redis cluster. The application solves problems such as data tilt, inefficient master-backup switching and inconsistent Session sharing in traditional technologies, improves the cluster load balancing efficiency and high availability, and is suitable for high-concurrency scenes such as medical treatment, finance and e-commerce.

The application provides an airport low-altitude flight track playback analysis method and system, and relates to the technical field of air traffic management. The method comprises the following steps: acquiring low-altitude flight track data, constructing a hybrid neural network model to filter and process the low-altitude flight track data to generate intermediate flight track data; identifying key track feature data and non-key track data in the intermediate flight track data through an SVM classifier and respectively performing lossless compression and lossy compression to generate playback flight track data; designing a collaborative computing framework of CPU, GPU and FPGA, and distributing the playback flight track data to multiple track computing nodes for collaborative processing; adopting a storage architecture based on consistent hashing to partition and store the playback flight track data, and designing a multi-level index mechanism to support the retrieval of the playback flight track data, so that the track playback precision can be improved, the data processing efficiency can be optimized, and the system storage and computing cost can be reduced.

The embodiment of the invention discloses a data storage method and a related device, the method can be applied to a data storage system, the data storage system comprises a center node and at least two storage nodes, and the storage nodes are used for correspondingly storing data identifiers and data. In the method, a central node can obtain first data and determine a first group identifier based on a consistent hash algorithm according to a first data identifier, the first group identifier is a group identifier of a first group and is used for indicating that the first data belongs to the first group, the first data identifier is an identifier of the first data, and then the first group identifier is used for indicating that the first data belongs to the first group based on a random selection algorithm. And a first storage node corresponding to the first group identifier is determined from the at least two storage nodes, and the center node sends the first data identifier and the first data to the first storage node for corresponding storage. The flexibility of capacity expansion or capacity reduction of the system can be improved, and the actual capacity expansion or capacity reduction effect is improved.

The invention discloses a block chain collaborative storage method and system based on weighted consistency Hash, and relates to the technical field of distributed data management, and the method comprises the steps: electing an anchor node in a group; the anchor node allocates the number of virtual nodes according to the available storage capacity of each physical node, virtual node coordinates are generated, and a consistent hash ring and a mapping table are constructed; setting a backup number parameter n, calculating n hash values for a new block according to a new block identifier and a backup sequence number, performing clockwise search and duplicate removal to supplement n target physical nodes, and distributing, writing and recording mapping of the block and a node set; and when the detection node fails, updating the hash ring, screening an affected block, recalculating a target node, determining an incremental node set, complementing and writing from the existing copy, and updating the mapping. According to the technical scheme of the invention, load balancing and copy discrete distribution driven by capacity weight are realized, the node change migration volume is reduced, the write availability and the repair efficiency are improved, and the ledger continuity is ensured.

Embodiments of the present disclosure relate to intelligent flow state synchronization that improves availability and performance of redundant network security devices. An example security system for use between at least one upstream router and at least one downstream router is described. A group or pool of security devices can be used to provide stateful security to bidirectional packet flows between the upstream router and the downstream router. Packets of the bidirectional flows are forwarded to particular security devices based on a consistent hashing ring process. Bidirectional state information is synchronized between some, but not all, of the security devices for a given flow. The security devices that share such bidirectional flow state information are determined using the same consistent hashing ring process.