366 results about "Database clustering" patented technology

The invention relates to the technical field of databases, and discloses a multi-type database performance optimization and operation and maintenance management method and system.The multi-type database performance optimization and operation and maintenance management method comprises the steps that real-time performance indexes of a heterogeneous database cluster are collected, and a noise reduction data set is obtained; generating a cross-library performance coupling degree matrix through the resource competition coupling degree and the data dependence coupling degree; constructing a coupling relation graph; identifying a bottleneck node set based on the coupling relation graph and the abnormal level; generating an optimization strategy; and performing strategy conflict identification according to the optimization strategy, generating a processing strategy, and executing the processing strategy optimization strategy. Through the improved weighting centrality algorithm, the core bottleneck node which has the greatest influence on the whole cluster is accurately identified, a differential optimization strategy is adopted according to the database type, a perfect strategy conflict detection and avoidance mechanism is established, mutual interference in the optimization process is effectively prevented, the optimization success rate is improved, and the optimization efficiency is improved. And the optimization time is shortened.

The invention belongs to the technical field of computer software, and particularly relates to a heterogeneous database unified deployment management method based on a cluster management platform, which comprises the following steps: predefining metadata of various database services; the method comprises the following steps: pre-constructing a multi-version program package warehouse, and establishing a structured warehouse directory system; obtaining a chip architecture and an operating system type of the target node, and generating environment fingerprint information; dynamically selecting a database installation package matched with the current environment and a dependency item set of the database installation package by mapping a warehouse directory system; through a unified database adaptation layer, a standardized operation interface is called, and installation, configuration and start-stop operation of the database are executed; a visual deployment and arrangement function based on component roles is provided by expanding a Web service interface of the management platform; and according to a deployment arrangement result and the environment fingerprint information, calling a deployment script to complete automatic deployment and configuration of the database cluster. According to the invention, unified adaptation, deployment and full-life-cycle management of various domestic databases are realized.

The embodiment of the invention discloses a capacity expansion and shrinkage method, system and device for a database cluster, a storage medium and a computer program product. The method comprises the steps that response time information of the database cluster with the capacity to be expanded and shrunk for different types of data access requests is obtained; based on the response time information and the type of the data access request, determining a capacity expansion and contraction strategy for the database cluster; and based on the capacity expansion and shrinkage strategy, carrying out capacity expansion and shrinkage processing on the database cluster. Therefore, the capacity expansion and shrinkage strategy of the database cluster is determined by the database cluster according to the response time information of different types of data access requests and the types of the data access requests, so that a mechanism for automatically triggering the capacity expansion and shrinkage of the database cluster based on the business requirements of the database cluster and the business processing requirements of the database is realized; the intelligence of triggering capacity expansion and shrinkage of the database cluster and the efficiency of capacity expansion and shrinkage of the database cluster are improved.

The invention relates to the technical field of distributed data storage, and discloses a database cluster partition operation method and device under a network partition scene and a medium, and the method comprises the steps that a multi-master distributed database cluster is constructed, and each node continuously captures a database native transaction log; obtaining change event data, and performing unified data storage; analyzing the original log into structured data, and executing uniform fragmentation and parallelization processing; processing the block log change record, and arranging change events of the primary key according to a rule; executing conflict processing and data synchronization, and synchronizing all nodes after the conflict is solved; through exception handling and fault tolerance, self-healing is realized and data consistency is maintained. According to the method, millisecond conflict identification and data consistency convergence after network disconnection can be realized, continuous and stable operation of a cluster in a partition state is ensured, and self-healing and continuous availability of a system are realized.

The invention relates to the technical field of computers, and discloses a multi-database deployment method based on a server cluster, which comprises the following steps of: distributing transaction state information for each database node, and realizing data consistency among the nodes in the database cluster by utilizing a consistency operator; a data synchronization path is optimized according to the topological structure between the nodes, and an optimal path is selected to reduce data transmission delay; the synchronization data volume is reduced through an increment synchronization and redundancy elimination method; designing and implementing a fault recovery strategy to ensure that the normal operation of the system is rapidly recovered when the database node has a fault; and dynamically adjusting a synchronization strategy and a consistency level of the database cluster according to a system load condition. By adopting the technical scheme based on transaction state vector allocation and consistency operator mapping, the technical effect of efficiently realizing data consistency in a distributed database environment is achieved. The problems that the consistency maintenance efficiency is low and data conflicts are prone to occurring in a high-concurrency scene are solved.

The invention discloses a database connection pool management method and device, equipment and a storage medium, and relates to the technical field of database management, the method comprises the following steps: creating a corresponding cluster connection pool for each cluster according to cluster parameter information and cluster configuration information, the cluster connection pool comprising a plurality of connection clusters, the connection cluster comprises a plurality of target connections, and the target connections carry first tags and timers; creating a database cluster pool according to the corresponding relationship between the cluster identifier and the cluster connection pool; and managing the target connection based on the database cluster pool, the first tag and the timer. The database connection resources are managed in a unified manner through the two-stage pool, so that the overhead of connection establishment and disconnection is reduced, and the processing efficiency is improved. And assigning a label to each connection, and upgrading the database connection pool into unequal stateful connections. And the timer is used for marking and tracking the use condition of each connection, so that the life cycle of the connection can be managed, and the management efficiency of the connection pool is improved.

The invention belongs to the technical field of intelligent control, and discloses an energy optimization management system and method for hotel guest rooms, and the system comprises a central data pool unit which constructs a time sequence database cluster, and carries out the uploading and interaction of data between all units through a data bus; the existence sensing unit is used for integrating multiple sensors by adopting a sensing layer architecture to form a sensing network, capturing existence sensing data and generating a room existence state map based on the existence sensing data; the existence sensing data comprises guest room reservation information data, guest room environment data and guest information data; the partition environment control unit is used for obtaining customer preference data by analyzing a room existence state map; based on the guest preference data and the guest room environment data, selectively activating environment control of different guest room areas by adopting an intelligent optimization algorithm; and the hotel energy utilization rate and customer satisfaction are improved.

The invention discloses a database cluster, a data processing method, a data processing system and related equipment, and relates to the technical field of databases. The database cluster comprises a first service instance, a second service instance and a storage engine instance, the storage engine instance is connected with the first service instance, the second service instance and the storage device, and metadata in the storage device is shared by the first service instance and the second service instance. The first service instance is used for sending an operation command to the storage engine instance according to the database statement; and the storage engine instance is used for updating the first data and the second metadata persistently stored in the storage device according to the operation command. Therefore, a plurality of service instances can share the same part of metadata in the storage device through the storage engine instance, and a part of metadata does not need to be stored individually and persistently in a local storage area, so that a metadata synchronization process does not need to be executed after the metadata in the storage device is updated, and the performance of a database cluster is improved.

The invention discloses an intelligent database switching method and device, computer equipment and a storage medium, belongs to the technical field of big data, and is applied to a financial database downtime processing scene. According to the method, the node state of the database is monitored in real time, multi-dimensional load evaluation is combined, the optimal main and standby database combination is automatically selected, second-level fault sensing and rapid switching are achieved, and a minute-level time window needed by traditional database recovery is remarkably shortened. The database identifier is embedded in the service main key, so that the request can accurately position the corresponding database node, complex cross-database query and routing overhead are avoided, and the system response efficiency is improved. In addition, through data seamless transmission and automatic abnormal switching between the main database and the standby database, the risk of service interruption caused by database faults is reduced. According to the scheme, the high availability and the fault recovery speed of the database are improved, repeated construction of a multi-product-line database cluster is reduced through resource integration, and a large amount of hardware and operation and maintenance cost are saved.

The invention relates to the technical field of data processing, in particular to a database pre-downsampling method and device, electronic equipment and a storage medium, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. The method comprises the steps that after time series data are written into all storage fragments of a storage node, a plurality of preset pre-downsampling tasks are obtained, and each pre-downsampling task is used for executing corresponding pre-downsampling processing on the time series data; merging a plurality of fragment tasks associated with the same storage fragment in the plurality of pre-downsampling tasks to obtain each merged task; and for each merging task, instructing the computing node to pull the written target data from the storage fragment associated with the merging task, executing corresponding pre-downsampling processing on the target data according to the merging task, and writing the obtained pre-downsampling data into the storage node. According to the invention, the memory of the storage node can be saved, and the availability of the time sequence database cluster is improved.

In some embodiments, a database cluster to cluster synchronization system may include multiple replicators coupled to a source database cluster and a destination database cluster, where the source and destination cluster may be shard clusters. Each of the multiple replicators may correspond to a respective subset of the source database cluster and configured to monitor changes of data on the respective subset of the source database cluster and translate the changes of data to one or more database operations to be performed on the destination cluster. The changes of data on the source database may be contained in respective change streams associated with each of the replicators.

The invention discloses a spatio-temporal data hybrid storage system and device based on a smart city and a storage medium, and relates to the technical field of data processing, the spatio-temporal data hybrid storage system based on the smart city comprises a processing module and a storage module, and the storage module comprises a file system and a database cluster. Data interaction is carried out between the file system and the database cluster, the file system comprises a local file system and a distributed file system, the database cluster comprises a relational database, a non-relational database and a search database, and data interaction is carried out between the relational database and the non-relational database. And data interaction is performed between the non-relational database and the search database. The problems that spatio-temporal data storage and management are disordered and are not easy to manage, and spatio-temporal data read-write analysis and calculation efficiency in the field of smart cities is low are solved, and the use performance and efficiency of the spatio-temporal data in the field of smart cities are improved.

The invention discloses a distributed file management system and method, and belongs to the technical field of computer software, the system adopts a tree hierarchical structure to organize files, and the system comprises an application layer used for receiving file operation requests from a plurality of independent clients, and each request carries a unique application identifier; the file system layer comprises a configuration module, an operation module, a metadata management module, a day module, a timed task module and a gateway module, and the request is routed to a corresponding logic storage partition in the data layer according to an application identifier; the data layer comprises a database cluster and an object storage cluster, and performs persistent storage and management of logic isolation on metadata and file data according to application identifiers; and the infrastructure layer provides an extensible server and network resources. Based on a distributed architecture, load balancing and efficient access are realized through data storage partitioning, a metadata management and index technology and a client load balancing technology, and the expandability, fault tolerance and performance of the system are improved.

The embodiment of the invention provides a database upgrading method, electronic equipment, a storage medium and a program product, and is applied to the technical field of computers. The method comprises the steps of obtaining configuration parameters of a target database in response to an upgrading operation for the target database; the target database is a database which is providing service; judging whether the parameter value of the configuration parameter is the same as the parameter value of a corresponding preset configuration parameter or not, and if not, judging whether an effective mode of the configuration parameter during upgrading is an effective mode after reloading or not; if yes, reloading the configuration parameters based on the parameter values of the corresponding preset configuration parameters, and upgrading the target database into a single-node database cluster based on a reloading result; according to the method, the current load state is determined, and when the current load state is the low load state, the single-node database cluster is upgraded to the multi-node database cluster, so that services are continuously provided for the user in the upgrading process, the service efficiency and quality are improved, and the user experience effect is improved.

The invention discloses a database bidirectional synchronization method and a computer program product, and relates to the technical field of databases, and the method comprises the steps that a source end database cluster receives externally inserted data change operation, updates a source data table and writes the source data table into a first archiving log to obtain a second archiving log, determining a data change record corresponding to the data change operation from the second archiving log and reporting the data change record to an acquisition processor system; the acquisition processor system receives the data change record, determines a target data table having a synchronization relationship with the source data table, and sends the data change record to a target database cluster where the target data table is located; and the target database cluster performs data synchronization operation on the target data table according to the data change record, and writes the data synchronization operation into a third archiving log. According to the method, the synchronization flow direction of the data change among the multiple clusters is controlled by taking the data table as a basic unit, so that the performance loss and redundant space occupation in the data synchronization process are reduced, and the data synchronization efficiency is improved.

The invention relates to the technical field of data synchronization, in particular to a distributed database system disaster recovery method and system based on cross-cluster data synchronization. The method comprises the steps that a target database cluster pulls source cluster metadata from a source database cluster based on a synchronization period, the target database cluster is any slave database cluster, and the source cluster metadata comprises total data information and total node information of the source database cluster; the target database cluster compares the target cluster metadata of the target database cluster with the source cluster metadata to determine difference data; the target database cluster determines a plurality of transactions based on the difference data and executes a data version synchronization task corresponding to each transaction, and the data version synchronization tasks are used for synchronizing data from the source database cluster. According to the invention, efficient and flexible cross-cluster data synchronization can be realized.

The invention discloses a database cluster log collection method, system and device and a medium, is applied to the technical field of databases, and is used for solving the problem of relatively low fault reason analysis efficiency of a database cluster in the prior art. The method specifically comprises the steps of determining to-be-investigated database node information corresponding to a database cluster and to-be-investigated log type information corresponding to the to-be-investigated database node information according to a to-be-investigated fault type based on a first configuration file; based on a second configuration file, determining target log collection component information according to the to-be-checked log type information; and calling the target log collection component corresponding to the target log collection component information, and collecting the log file of the to-be-checked database node represented by the to-be-checked database node information based on the to-be-checked log type information, thereby realizing automatic log collection when the database cluster fails, and improving the log collection efficiency. The log collection efficiency when the database cluster fails is improved, and then the fault reason analysis efficiency of the database cluster is improved.

The invention relates to an intelligent optimization method and device for a database cluster, equipment and a storage medium. The method comprises the following steps: generating a heterogeneous feature vector based on multi-modal data of the database cluster; inputting the heterogeneous feature vector and the constructed parameter dependent directed graph into a graph neural network model, outputting a high-priority parameter, inputting the heterogeneous feature vector into a parameter adjustment strategy network, generating a parameter adjustment action of the high-priority parameter, and updating the parameter adjustment strategy network on the basis of recording performance feedback data after parameter adjustment; uploading the encrypted encryption model gradient update vectors of the parameter adjustment strategy network to a federated learning server, performing weighted aggregation processing on the encryption model gradient update vectors of the plurality of database clusters to obtain a global model, and broadcasting the decrypted global model to the plurality of database clusters; according to the technical scheme provided by the invention, the precision, dynamic and knowledge sharing of database cluster parameter tuning can be realized, and the performance and operation and maintenance efficiency of the database cluster are remarkably improved.

A government affair decision support platform based on a big data technology comprises a data layer used for performing multi-dimensional fusion on electric power data and industry data, constructing a structured database cluster, an unstructured data storage system, a full-text retrieval data storage and a cache data storage; the data analysis layer is used for constructing a multi-dimensional comprehensive analysis theme achievement support model based on the multi-dimensional data fused by the data layer, and the multi-dimensional comprehensive analysis theme achievement support model comprises a decision data thematic database, a real estate analysis model, an enterprise operation analysis model and an industry analysis model; the service layer is used for adopting various micro-service architectures, providing government affair decision support channel services and continuously and dynamically monitoring risks; and the display layer is used for visually exporting and displaying the analysis result by adopting a plurality of interface display technologies. According to the invention, a service platform for supporting the unification of government policy research is constructed.

The invention discloses a database cluster management method, device and equipment and a computer program product, and relates to the technical field of basic setting and IT supporting, and the method comprises the steps: monitoring the operation state of a first main server node; when the operation state of the first master server node has an operation fault, acquiring a node state of a slave server node cluster; and based on the node state of the slave server node cluster, determining a second master server node from the slave server node cluster through a pushing algorithm. The new master server node is determined between the slave server nodes through the pushing algorithm, automatic switching between the master server node and the slave server nodes is achieved, and the technical problem that a database deployed in a single-point mode cannot be used when having a fault is solved.

The invention provides a database cluster deployment method and related products. The database cluster deployment method comprises the steps that node information of all databases is obtained, and the node information at least comprises a node data state, a database operation state and a database operation mode; identifying a database operation mode of each node; if the operation mode of the database is a stand-alone mode, executing backup operation on all the nodes; and if the database operation mode is a cluster mode, performing differentiated operation on each node according to the node data state and the database operation state. The method has the advantages that efficient multiplexing of existing data and rapid deployment of the database cluster are achieved.

The invention relates to the technical field of computers, and provides a distributed timed task scheduling method and system, a server and a program product, and the method comprises the steps: obtaining a timed task creation request scheduled by a second load balancer in a target task cluster in at least one task cluster; binding a timed task to a time wheel in the first server according to the timed task creation request; the target task cluster receives a timed task creation event and stores the timed task creation request into a storage database cluster of the target task cluster, the storage database cluster has notification and monitoring functions, and other servers of the target task cluster obtain the timed task creation request from the storage database cluster of the target task cluster after monitoring the timed task creation event; and the timed task is bound to the time wheel in other servers according to the timed task creation request, so that the distributed design and the high-availability redundancy design of the system can be increased, and the stability, the expandability and the reliability of timed task scheduling are improved.

The invention relates to a distributed high-availability code warehouse management method and system and a medium, and relates to the technical field of version control and code management.The method comprises the steps that a pre-configured external load balancer, a plurality of GitLab Web nodes, an internal load balancer, a PostgreSQL database cluster, a Redis cluster, a Gitaly cluster and object storage or file storage are obtained; carrying out load balancing on the plurality of GitLab Web nodes through the external load balancer; performing load balancing on the internal connection of the GitLab application program through the internal load balancer; the data of the GitLab Web and the data of the GitLab Pracect are stored through the PostgreSQL database cluster, and the data of the GitLab Web and the data of the GitLab Pracect are And providing access to a Git warehouse by using the Gitaly cluster. According to the invention, high availability, high performance and expandability of code warehouse management are realized through a distributed architecture and redundancy configuration.

The invention discloses a log synchronization method, device, system and equipment, a medium and a program product, and relates to the technical field of databases. The method comprises the following steps: receiving a to-be-synchronized log packet sent by a main library through a standby library in a database cluster; adding the to-be-synchronized log packet into the asynchronous analysis reconstruction task, and sending a response message to the main library, wherein the response message is used for indicating the main library to write the to-be-synchronized log packet into an online log file of the main library; and executing the asynchronous analysis reconstruction task to obtain a reconstructed log packet of the to-be-synchronized log packet, and performing log replay according to the reconstructed log packet. The standby library adds the received to-be-synchronized log packet into the asynchronous analysis reconstruction task, and the response message is returned to the main library without waiting for the completion of the execution of the asynchronous analysis reconstruction task, so that the response waiting time of the main library is effectively shortened, the transaction submission speed of the main library is accelerated, and the log synchronization efficiency between the main library and the standby library is improved.

The invention relates to the field of data transmission, and provides a time series data cross-gatekeeper two-way transmission method, device and equipment, and the method comprises the steps: configuring an Internet of Things database cluster and an isolation gatekeeper, and determining a configuration result; according to the configuration result, the sending end is started to package the time sequence data, a data packet containing redundancy check information is generated, and the redundancy check information comprises a double-length field and a check code; transmitting the packaged data packet to a receiving end through an isolation gatekeeper; after the receiving end receives the data packet, verifying the dual-length field and the check code, if the verification is passed, returning a success signal, otherwise, returning a failure signal; and if the sending end receives the failure signal, automatically retransmitting the data packet until a successful signal is received or a preset retry condition is met. The problem that in the prior art, cross-gatekeeper time sequence data transmission cannot meet the requirements for high safety and real-time performance at the same time is solved, and safe and reliable time sequence data two-way real-time transmission under the strict network isolation condition is achieved.

The invention provides a database cluster inspection control method and system and a server, and relates to the field of database operation and maintaining.According to the method, health examination, performance analysis and fault diagnosis can be conducted on a database cluster in real time through multi-modal data fusion analysis, automatic repair can be conducted on the basis of a dynamic strategy selection mechanism when problems are found, and the problem is solved. And the disposal progress can be synchronously pushed through the multi-channel alarm platform, so that the inspection control effect of the database cluster is remarkably improved.

A database clustering search system includes a processor; and a non-transitory, processor readable storage medium communicatively coupled to the processor, the non-transitory, processor readable storage medium including one or more instructions stored thereon that, when executed, cause the processor to: receive input comprising a search query; deploy a search based on the input; populate a list that refers to one or more content types; and output the list to a user device.

The invention discloses a data processing method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: receiving real-time data streams sent by a plurality of clients through a target server cluster; wherein the target server cluster comprises an asynchronous socket server based on a non-blocking network, and the target server cluster realizes event-driven network data processing based on a reactor mode; according to the real-time load of each partition in a target message queue, distributing the received real-time data flow to the corresponding partition in the target message queue; cleaning the real-time data flow of each partition in the target message queue by using a target rule engine with a hot loading function to obtain cleaned data; and storing the cleaned data to a distributed memory database cluster. Therefore, resource tension and response delay caused by connection congestion are reduced, and data consistency and service continuity are guaranteed.

The invention relates to the technical field of databases, and discloses a database pre-synchronization implementation method and device, equipment, a medium and a product. The method comprises the steps of obtaining synchronization task configuration after a pre-synchronization process is started, and obtaining metadata corresponding to each to-be-synchronized table according to the synchronization task configuration; generating a file path corresponding to each to-be-synchronized table according to the metadata and a preset directory structure rule, and performing directional reading according to the file path to obtain a global event data file corresponding to each to-be-synchronized table; and according to the global event data file, obtaining a system change number corresponding to each to-be-synchronized table to complete database pre-synchronization. The accurate path positioning based on the metadata is adopted to replace full disk traversal, so that the disk input / output performance bottleneck can be avoided, the pre-synchronization efficiency can be improved, the pre-synchronization time consumption can be remarkably reduced, the normal operation of a database cluster can be prevented from being interfered, and the system stability can be improved.

The invention discloses a resource allocation method, a computer program product and electronic equipment. The method comprises the following steps: receiving an application data operation request, wherein the application data operation request carries a namespace identifier matched with a first application for triggering operation; determining a node list of a node database occupied by the application data of the first application in the first storage database cluster based on the namespace identifier; obtaining a node demand resource load of a node database in the node list in a second processing period; and determining a first node database from the second storage database cluster and migrating the application data of the first application to the first node database under the condition that the first application is determined to meet the resource redistribution condition based on the node demand resource load. The technical problem of low resource utilization rate caused by neglecting the resource use demand change condition of the application is solved.