45results about How to "Avoid Single Points of Problem" patented technology

The invention discloses a big data storage and access system and method based on a distributed file system. The big data storage and access system based on the distributed file system comprises the distributed file system which comprises a metadata server, a storage server cluster, an access client and a log server, the metadata server manages and monitors metadata information in the distributed file system and the storage server cluster, the storage server cluster stores data contained in the metadata information in multiple data storage nodes of the storage server cluster, the log server backs up the metadata information in the metadata server and synchronizes operation logs of the metadata server in real time, and the access client is used for selecting standby metadata service processes in the metadata server or in the log server according to the connection state of the metadata server to store the server cluster.

The invention provides a timed task clustering method and a timed task clustering device. The method comprises the following steps: distributing nodes in a cluster at server nodes with odd number which is at least three, distributing timed task instances on the timed task nodes, obtaining timed task executing right from the cluster according to planned time point by each timed task instance, executing the timed tasks by the timed task instances with a write-in right, giving up executing the timed tasks by the timed task instances without the write-in right, till the next planned time point, and then circularly executing and obtaining the timed task executing right. After the adoption of the technical scheme, the method and the device can solve a single-point problem of the timed processing technology, and implement high availability of the timed cluster.

The invention provides an incremental data pushing system and method. The incremental data pushing method comprises the following treatments carried out under task allocation and scheduling of a distributed scheduling center: capturing treatment; increment analysis treatment; logic theme merge treatment; intermediate storage treatment; data field filtration treatment; export treatment.

The invention discloses a data server cluster system and a data synchronization method. The data synchronization method includes the steps that data sent by a client side are received by a master server and then numbered, the numbered data are broadcasted to a slave server, after the master server receives more than half of confirmation information of the numbered data received by the slave server, storage data commands are broadcasted to the slave server, and the master server and the slave server store the numbered data to local. By the adoption of the data server cluster system and the data synchronization method, the single-point problem of a traditional master server and a traditional slave server is solved, meanwhile, uniformity of cluster data is guaranteed, and reliability and availability of clusters are improved.

The invention discloses a session management method which comprises a step of obtaining a session token and a cache version number carried by a session request when the session request sent by a client is received, a step of searching the service end cache corresponding to the session token at a server local, a step of judging whether the service end cache is a latest version according to the cache version number if the service end cache corresponding to the session token is searched successfully, and a step of obtaining the service end cache corresponding to the session token from a distributed cache cluster if the service end cache corresponding to the session token is not searched or the service end cache is not the latest version, and carrying out session operation according to the session request. The invention also discloses a session management system. The session management in WEB application of multiple servers is realized.

The invention provides an orderly sequence number generating method of a Zookeeper-based distributed system. The method comprises the following steps of 1, creating an orderly sequence root node on Zookeeper of the distributed system; 2, creating a lasting sequence node under the orderly sequence root node; 3, obtaining the sequence number of the lasting sequence node; 4, deleting the sequence node corresponding to the front sequence number of the sequence numbers. The generation of the orderly sequence number under a high-subsequent request environment of a multi-service terminal in the distributed system is realized by using data high consistency and the node sequence property of the lasting sequence node of the Zookeeper.

The invention provides a method and system for processing workflow. The method and system for processing workflow can avoid a single-point problem in a distribution system, and improve reliability of task execution. The method for processing workflow comprises that a tree structure execution plan converted into by workflow which comprises a plurality of task nodes is saved, and in the execution plan, a task node with a plurality of father nodes appears repeatedly in a branch which the father node exists in, and for the task nodes which appear repeatedly, the task nodes with child nodes are used as execution nodes of the task nodes, and the other task nodes are used as non-execution nodes of the task nodes; the execution plan is traversed, and when a task node which the task is not executed at is traversed, the task node is occupied; all father nodes of the task node are confirmed to be executed, and then the execution node of the task node is executed.

The invention discloses a Redis cluster management method, which comprises the following steps of: when a data access request is monitored, determining a target data partition which is not downtime through a Redis cluster agent component and a data key value in the data access request; routing the data access request to the target data partition, and processing the data key value to obtain a hashslot number; and routing the data access request to a Redis node corresponding to the hash slot number in the target data partition. The invention also discloses a Redis cluster management device, equipment and a readable storage medium. According to the invention, the data integrity and high availability of the Redis cluster can be effectively ensured.

One or more embodiment of the present specification discloses a dynamic current limiting method, apparatus, and system for achieving globally accurate current limiting by dynamically adjusting a current limiting threshold. The method is applied to a current limiting device which establishes a long connection with a server cluster, includes obtaining a flow restriction state of each server in the server cluster and a total flow restriction threshold of the server cluster, wherein the flow restriction state comprises a current flow restriction state of each server in a current working cycle and/or a historical flow restriction state in a historical working cycle; determining a current limiting threshold of each server in a next duty cycle according to the current limiting state and the totalcurrent limiting threshold; performing a current limiting operation on each server according to the current limiting threshold.

The invention discloses a shared resource access method based on zookeeper, a client, a server and a system. The method comprises the following steps that when an access request for a preset resourceis sent to the server, and corresponding unique instant ordered nodes are created under a catalogue corresponding to the resource, wherein the corresponding unique instant ordered nodes comprise nodeserial numbers; the change of all the instant ordered nodes under the catalogue sent by the server is obtained; when it is judged that the node serial numbers of the own instant ordered nodes are minimum according to the change, a lock is obtained from the server so as to obtain the permission for having access to the resource. On the basis of the method, the single-point problem, the non-reentryproblem, the non-blocking problem and the problem that the lock cannot be released can be effectively solved, and the lock is high in safety and can be durable.

The invention provides a distributed dispatching system and method. The distributed dispatching system comprises a dispatching portion, an operation portion and a storage portion, wherein the dispatching portion comprises at least one dispatching unit, the operation portion comprises multiple operation units, the storage portion comprises at least one storage unit, and the dispatching units are used for extracting to-be-disposed tasks from a file system and dividing the to-be-disposed tasks into multiple subtasks, are also used for dispensing the multiple subtasks to the corresponding operation units and executing the corresponding subtasks by the operation units and are further used for recording record information in the subtask execution process and storing the record information in thestorage units. The single-point problem caused by the fact that a central device performs centralized task processing is solved, and when tasks fail or an execution device breaks down, other device nodes continue to perform execution, automatic multi-machine retrying of failed tasks is achieved, and timely and correct task operation is ensured. When other problems are caused in the task executionprocess, and an alarm notification can be also provided for a user through the system.

The invention provides a workflow management system and method, the workflow management system comprises an access interface, a background server, a client and a task machine, and the client at leastcomprises a workflow engine and a distributed file system. The background server receives a first workflow sent by the access interface and a second workflow sent by the task machine, converts the first workflow and the second workflow into a file in a target network format, and sends the file in the target network format to a preset directory of the distributed file system for storage. The workflow engine acquires the target network format file as a to-be-executed task. And the task machine executes the to-be-executed task according to a preset script, and generates a second workflow representing an execution result of the to-be-executed task to the background server. Therefore, according to the embodiment, the connection between the task machine and the background server is established,the background server can perform task allocation on the first workflow and the second workflow, and the single-point problem of the task machine is solved.

The invention provides a distributed data dispatching system and method. The system comprises at least one dispatching component. The dispatching component is adapted to obtain an off-line log to be processed from a file system and divide the off-line log to be processed into a plurality of sub-logs. The scheduling component is further adapted to distribute the plurality of sub-logs to a pluralityof data mining units, and the data mining unit mines log metadata from the sub-logs according to preset rules; the scheduling component is further adapted to store the log metadata and other processinformation into a storage component including a plurality of preset databases. The single point problem caused by the centralized processing of tasks by the central equipment is solved. When the taskfails or the executing equipment fails, the other equipment nodes continue to execute the task, thereby realizing the automatic multi-machine retry of the task failure, ensuring the timely and correct operation of the task. The system can also alert the user when other problems occur during the execution of the task.

The invention discloses a template distribution method and system. The method comprises the following steps: a provider uploads a template to a cloud storage module and acquires a storage path of the template; the provider registers the template information to a template registration center, wherein the template information at least comprises a template name, a version number and a storage path; a user subscribe the template to use at the template registration center wherein the subscription information at least comprises the template name and the version number; the template registration center acquires the storage path of the template according to the template name and the version number registered by the user; and the template is acquired in the cloud storage module according to the storage path of the template. The system comprises the cloud storage module, a template providing template, a template registration center and a template using module. The template centralized management is realized, so that a communication bridge is established between the provider and the user, the distribution flow of the template is simplified, and the automatic forced updating of the template is realized.

The invention discloses a data acquisition method, a server and a related system, and the method comprises the steps of judging whether a client task of the client task is a main process for acquiringdata from a third-party server or not after the client task of the client task is started, wherein the main process is a unique process capable of acquiring data from the third-party server at the current moment; when the judgment result is yes, connecting the third-party server and requesting to acquire data; when each data message arrives, judging whether a client task of the client is still the main process or not; and if so, acquiring and processing the data. According to the data acquisition method disclosed by the invention, the single-point problem existing in the prior art can be avoided by scheduling the client task in each server, and the timely acquisition of real-time data can be ensured when a machine crashes, a network fault or other problems influencing connection occur.

The embodiment of the invention provides a timed task execution method, device and system based on a time window. At least one worker periodically executes tasks in parallel at a specified scheduling interval, wherein the process that each worker schedules and executes the tasks each time comprises the steps: firstly, setting the corresponding time window, querying a task table according to the corresponding time window, and acquiring timing tasks from the task table to serve as respective corresponding preparation tasks; each worker executes a respective corresponding preparation task, and modifies the task state of the corresponding preparation task in the task table to execution completion. By means of the scheme, the single-point problem in a timed task system is solved, and the effects of achieving thermal disaster recovery and automatically recovering timed tasks without manual intervention are achieved.