143 results about "Connection pool" patented technology

The invention provides a multi-node dynamic switching method and a multi-node dynamic switching system for a MongoDB (MongoDB). The method comprises the following steps: acquiring configuration information of a plurality of MongoDB nodes, and establishing a data access layer based on the configuration information; through a data access layer, acquiring index data marks of the MongoDB nodes about a plurality of operation indexes according to a hierarchical frequency acquisition task configured by each MongoDB node; in combination with a node weight preset value of the MongoDB node, converting the node performance index into a weight value of the MongoDB node; based on a connection pool management mechanism and the node available state data, a request distribution strategy is determined according to the weight value, and the request distribution strategy is used for indicating the distribution proportion of the MongoDB requests distributed to the MongoDB nodes; and performing state detection on the MongoDB node through a data access layer, marking the detected MongoDB node as an unavailable state when detecting that the MongoDB node meets an abnormal state condition, and updating available state data of the node. The problems that in the prior art, manual intervention is needed, the real-time load condition of the node cannot be sensed, and the code coupling degree is high are solved.

The invention belongs to the technical field of data processing, and discloses an active intelligent operation and maintenance monitoring method for a data center, which comprises the following steps: step 1, component modeling and topology configuration; 2, carrying out distributed health detection and data acquisition; 3, performing real-time health assessment and anomaly detection; 4, performing intelligent alarm and root cause analysis; 5, unified operation and maintenance and closed-loop control are carried out; and step 6, dynamically optimizing the intelligent operation and maintenance strategy. According to the method, a monitoring object and a dependency relationship are clarified through component modeling and topological configuration, and specific scenes, such as multi-mode acquisition, active detection, index pulling, log analysis, coverage message queue theme accumulation and database connection pool exhaustion, of distributed health detection are combined. The quantitative health score is calculated based on the preset scoring model, and by combining with the dynamic baseline learned by the ARIMA or LSTM algorithm, the module abnormity can be actively detected in different periods and weekly updating, and the problems of fault discovery lagging and incomplete monitoring coverage are solved.

The invention discloses a data transmission method, system and device and a storage medium, is applied to a host end in a non-uniform memory access architecture, and relates to the technical field of data transmission, and the method comprises the steps that a target path mapping table from a target end is received and analyzed, and a plurality of monitoring entrances are obtained; based on the target path mapping table, establishing a plurality of connection paths corresponding to the plurality of monitoring entrances in sequence, and generating a connection pool; in response to a data read-write request generated by the host end for a target end, determining a target processor domain corresponding to the target storage unit according to the target path mapping table, and selecting a connection path associated with the target processor domain from the connection pool as a current read-write path; and monitoring and judging whether the current read-write path is available or not, and if not, reselecting. The data transmission method can be combined with NUMA architecture perception, path state monitoring and a policy-driven selection mechanism to at least solve the problems that in the prior art, path selection lacks topology perception and multiple network cards are difficult to aggregate and use.

The invention relates to the technical field of data processing, and discloses a log data batch updating method and device, equipment and a medium, and the method comprises the steps: obtaining a full-amount historical log in a preset database in real time, converting the full-amount historical log into an event stream, and writing the event stream into preset message middleware, obtaining a partition ID of the total historical log in the message middleware, generating a global unique identifier bound with a partition according to the partition ID to obtain a distributed ID, generating a consumer ID associated with the user identity information according to the distributed ID by calling a pre-designed distributed algorithm, and sending the consumer ID to the message middleware. And carrying out lock-free parallel updating on the database according to the consumer ID based on connection pool preheating and asynchronous submission to obtain an updating result, carrying out consistency verification on the updating result to obtain a verification result, and confirming the updating result according to the verification result. And the batch updating efficiency of the log data is improved.

The invention provides a heterogeneous database query method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out security verification processing on a plurality of SQL execution requests sent by a user side, and carrying out SPI mechanism scanning processing on a plurality of database plug-ins in a heterogeneous database plug-in directory based on a high-performance database connection pool after the security verification processing is passed, determining a target database plug-in package corresponding to each SQL execution request; and performing metadata query processing on the SQL execution request in the target data source based on a database operation client interface and a metadata service interface under the target database plug-in package, and determining a structured tree result. According to the method, an extensible and dynamically integrated heterogeneous database plug-in mechanism is realized, unified access of various databases is supported, the platform maintenance cost is remarkably reduced, hierarchical metadata of any type of databases is obtained through a consistent metadata service interface, and the user experience and the cross-database query efficiency are improved.

The invention provides a method and system for accessing a database interface layer through an SAS language based on C + + middleware. The method comprises the following steps that the C + + middleware is called to obtain SAS statements of SAS language analysis software originating from the C + + middleware; performing word segmentation identification on database information in the SAS statement, and calling a database connection module of the C + + middleware based on the database information to establish a database connection pool; identifying the SAS statement by using the C + + middleware and converting the SAS statement into a database recognizable statement; the C + + middleware is used for transmitting the database recognizable statements to the corresponding database through the database connection pool, the problem that in the prior art, SAS language analysis software cannot be directly and efficiently connected with a universal database is solved, and high-performance, high-reliability and safe data access can be achieved.

The invention relates to the technical field of unmanned aerial vehicle control and management systems, in particular to an unmanned aerial vehicle parameter management method based on template management, and the method comprises the steps: building multi-device concurrent connection with an unmanned aerial vehicle flight control system through a communication abstraction layer, integrating a protocol adaptation module, a port automatic detection engine and a multi-device connection pool, and achieving a unified device access interface; parameters are structured and managed through the core processing module, a dynamic parameter tree, a batch operation engine and a template management system are constructed, verification operation is monitored through the safety control module, and an auditing system, a version control engine and a parameter verification module are integrated. According to the invention, by constructing a modular architecture of a communication abstraction layer, a core processing layer and a security control layer, adopting a dynamic parameter tree, a difference comparison algorithm and a template management system, and combining an operation auditing and atomic transaction writing technology, low resource occupation and high-concurrency batch operation are realized, the production line efficiency is improved, and the production cost is reduced. And parameter consistency, safety and traceability are guaranteed.

The invention relates to the technical field of Internet of Vehicles, and discloses an Internet of Vehicles communication stability optimization method and system based on multichannel connection multiplexing, and the system comprises a proxy service module, a terminal interaction module, a connection multiplexing module, a base station monitoring module, a load balancing decision module, and a strategy optimization module. The method comprises the following steps of: deploying a multi-thread proxy service at an edge node adjacent to a base station, constructing a surviving global shared connection pool, and realizing cross-thread security access by using a read-write lock; the terminal sends a request containing a target server IP and a TLS session identifier to the virtual IP entry through a TLS 3.1 + encryption protocol, so that the three-way handshake delay is avoided; a base station switching event is synchronously detected, a dynamic load balancing mechanism is triggered, health scores and weight factors are calculated in real time, and requests are distributed according to flow types. According to the method, handshake delay is shortened, communication continuity is improved, the problem of request packet loss is solved, and a high-reliability communication infrastructure is provided for an intelligent network connection vehicle.

The invention relates to the technical field of data testing and the field of smart medical treatment and financial science and technology, and provides a load balancing testing method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: generating a load balancing testing request based on a testing module of a client; forwarding the load balancing test request to a load balancing proxy node based on a switch; determining a target test script from the load balancing agent node according to the load balancing test request; performing data forwarding processing on the client and the communication test instrument based on the load balancing agent node according to the target test script, and detecting on the load balancing agent node to obtain a CPU utilization rate, a memory occupancy rate, connection pool state information and request processing delay information; and determining a load balancing test result according to the CPU utilization rate, the memory occupancy rate, the connection pool state information and the request processing delay information. Through the technical scheme, comprehensive and reliable test processing can be performed on the load balancing server simply, conveniently and quickly.

This invention relates to the field of experimental data storage technology for nuclear fusion devices, and discloses a method for large-scale EPICS data archiving based on a time-series database. The method involves acquiring process variables in an event-driven acquisition mode through an embedded channel access protocol data acquisition module and writing them into buffered memory channels. Through group aggregation and batch persistence, the process variables are grouped into fixed-size groups, each group corresponding to an aggregation coroutine. The latest event data is read non-blockingly from the buffered memory channels of each group at fixed intervals. Metadata mapping is used to convert the process variable names into device paths and measurement value names in the time-series database, assembling them into a sparse wide table. Null value columns in the sparse wide table are filtered, and the data is then batch-written into a distributed time-series database through a connection pool. This invention achieves stable high-throughput writing and low-latency retrieval of large-scale experimental data in the EPICS system, and provides an efficient data access interface for subsequent artificial intelligence training and inference platforms.

The invention discloses an efficient data transmission method and system based on Arrow Flight, and the method comprises the steps: achieving the direct interaction between Spark and an Arrow Flight Server based on a self-defined data source interface of a Spark DataSource API (Application Program Interface); the method comprises the following steps: defining a data reading process through a Scan interface of Spark, and acquiring distributed data in an Arrow Flight Server by adopting a parallel fragmentation mode; the method comprises the following steps of: customizing a batch processing reading process in Spark, and converting a data format obtained from an Arrow Flight Server into a column type storage format of the Spark; and designing a connection pool management and fault-tolerant mechanism. According to the method and the system, direct connection with an Arrow Flight Server is realized by creating a self-defined data source interface based on a Spark framework, and the performance and the stability of data interaction between systems in a big data environment are improved by utilizing efficient data fragmentation, batch transmission and column storage format conversion technologies, a connection pool and an automatic retry mechanism.

The application discloses a business information processing method and device, electronic equipment and a storage medium. The method comprises the following steps: a gateway module receives an HTTP request from a client, obtains a database name corresponding to an access object according to ID information of the access object in the HTTP request; the database identifier is put into the HTTP request, and the HTTP request is forwarded to a business module; the business module obtains the database identifier based on the HTTP request, and stores the database identifier into a thread local variable of a current thread; the database name is obtained from the thread local variable, and a corresponding database connection pool is selected according to a key-value pair; when other business modules are called, the corresponding database identifier is obtained from the thread local variable through an interceptor, and the database name is written into a new HTTP request header and transmitted to the called other business modules. The application ensures that the system can accurately match the corresponding data source when processing data, and meets the business secret protection requirement.

The application belongs to the technical field of computers and discloses a general metadata operation method and system based on a graph database, which comprises the following steps: S1, using a configuration file yml of SpringBoot to configure to create a connection pool SessionPool of a NebulaGraph Database graph database based on a NebulaPool custom implementation, containing definitions of maxCountSession, minCountSession, hostAndPort, userName and passWord parameters; S2, defining parameter models for accepting different interfaces, including a model for creating a graph space, a model for creating a point, a model for creating an edge, a model for creating an index and a general query model; S3, using an independently developed NebulaUtil tool to automatically arrange nGQL statements for the accepted parameter models according to the input models; S4, defining different Controller layers to provide an open and unified Api interface externally; S5, using an execution service of a GraphCommonService graph database to obtain an effective connection of the NebulaGraph Database graph database from the SessionPool; and S6, executing the nGQL statements automatically arranged by the NebulaUtil tool and analyzing and returning a result model, and uniformly returning the result.

The invention discloses a gateway-based script execution method, apparatus and device, and a readable storage medium, and relates to the technical field of network services. The gateway-based script execution method comprises the following steps: calling a preset script by a gateway, and performing access control on a received user request; obtaining an idle connection from a connection pool configured between the gateway process and the database server in response to access control passing of the user request; verifying the execution qualification of the user according to the user request; in response to the fact that the execution qualification of the user passes the verification, sending the preset script to a database server through the idle connection, and executing the service logic of the preset script to obtain an execution result; and storing the execution result in a non-volatile storage medium to realize persistence of the execution result. By implementing the technical scheme of the invention, the user request can be prevented from penetrating back-end service, and the request response speed is improved.

The invention provides a connection pool communication processing method, equipment and a medium, which are applied to the field of data transmission. The method comprises the following steps: acquiring length information and service type fields of data to be transmitted in a current process; classifying each piece of data to be transmitted into a first priority connection pool and a second priority connection pool according to the service type field; in the first priority connection pool, when the priorities of the to-be-transmitted data corresponding to the service type field are different, the to-be-transmitted data with the high priority are transmitted firstly; when the priorities of the to-be-transmitted data corresponding to the service type field are the same, the to-be-transmitted data with the high priority corresponding to the data length represented by the length information is transmitted first; in the second priority connection pool, the to-be-transmitted data with the high priority corresponding to the data length represented by the length information is transmitted first; therefore, according to the invention, the connection pool is dynamically divided according to the service type field, the problem of tail time delay caused by the transmission of the mixed message in the network environment is effectively reduced, and meanwhile, the resource occupation is reduced.

This invention provides a shipboard crew positioning network communication system and method, relating to the field of network communication technology. The system includes a data acquisition module, a system establishment module, a data evaluation module, a connection pool parameter configuration module, a testing module, and an optimization module. The method includes starting the data acquisition module, establishing a database management system, evaluating database performance, configuring connection pool parameters, conducting system testing, optimizing connection pool parameters, and realizing crew positioning and communication, as well as continuous monitoring and maintenance. The shipboard crew positioning network communication system and method provided by this invention employs an optimized connection pool configuration approach, gradually adjusting the minimum number of connections, the maximum number of connections, and the connection timeout time in the connection pool to progressively approach the optimal connection pool configuration parameters. This enables rapid response and processing of crew location information update requests under high-concurrency access scenarios.

The application discloses a multi-source data management method and system based on a cache, belongs to the technical field of data management, and aims to solve the technical problem of how to ensure that an application system is quickly connected with a data source under the premise of reducing system energy consumption.The method comprises the following steps: cache initialization: configuring basic parameters of the cache to form a configuration file, initializing the local cache through the configuration file, storing a data source connection pool in the cache, configuring a listening service for the cache, and listening to the connection and use state of the data source connection pool through the cache listening service; cache addition: during the process of processing data by a business system, a newly-built data source is added into the cache based on data source information; and cache removal: removing the data source connection pool in the cache.

Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for connection pool efficiency. The method may include determining a set of connections in a connection pool of a storage system, the set of connections corresponding to open connections established by the storage system to a set of end-points of an object storage service of a cloud provider, wherein an application of the storage system accesses the object storage service through one or more connections from the set of connections; monitoring data transfers over the set of connections; collecting performance metrics associated with the data transfers; identifying a first connection having performance metrics below a predetermined threshold; identifying a second connection being added to the set of connections in the connection pool, wherein the second connection is a new connection; and marking the first connection for removal from the connection pool.

The invention provides a method and system for improving system concurrency capability in a live broadcast scene, and the method comprises the steps: obtaining historical data corresponding to live broadcast according to the configuration information of the live broadcast, asynchronously generating a plurality of accounts in advance through intelligent prediction, converting the most time-consuming write operation in a real-time link into efficient memory allocation, and improving the system concurrency capability in a live broadcast scene. Therefore, write lock competition and connection pool exhaustion of the database are avoided. Meanwhile, through smooth supplement during real-time live broadcast and an automatic cleaning mechanism after the live broadcast is finished, on the premise of ensuring system elasticity, optimization of resource utilization is realized, the extreme experience and stability of the system under high concurrency are fundamentally ensured, and the user experience is improved. The problem that response delay soaring and even service avalanche are caused by dense database writing at the flow flood peak is well solved, meanwhile, user experience is greatly improved, and the registration and login time of a new user is greatly shortened.

The invention discloses a database connection pool management method and device based on DataX, and solves the problems of low resource utilization rate, concurrence disorder, thread safety and the like caused by lack of a built-in connection pool in the existing DataX. The method comprises the following steps: deploying an independent connection pool service (CPS), integrating a third-party connection pool and configuring a global maximum connection number; the DataX communicates with the CPS through a network, and each Channel independently applies for connection; the CPS dynamically adjusts the DataX parallelism degree (limitation or waiting when the DataX parallelism degree is insufficient) according to the available connection number, and allocates a unique connection identifier; the DataX sends an SQL request through the identifier, and the CPS calls the third-party connection pool for execution and returns a result; and after the task is completed, notifying the CPS to release the connection. Connection multiplexing is achieved through centralized management, the physical connection sharing risk is avoided, the parallelism degree is dynamically regulated and controlled to guarantee efficient utilization of resources, and DataX task stability and thread safety are improved.

The embodiment of the invention provides a dynamic connection pool management method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining the resource information of a connection pool in a current monitoring period, the resource information being information reflecting the performance index of the connection pool; when it is determined that a capacity adjustment condition is met according to the resource information, the capacity of the connection pool is adjusted, and the adjusted capacity is applied to operation of the connection pool in the next monitoring period. According to the embodiment of the invention, the dynamic adjustment speed of the connection pool can be improved, and resource waste caused by connection exhaustion or more idle connections is avoided.

The invention relates to a data query method and device, computer equipment and a storage medium. The method comprises the following steps: receiving a data query request sent by a client; analyzing the data query request to extract coding information corresponding to the target type from the data query request; carrying out validity verification on the coded information; if the coding information passes the validity verification, performing routing rule matching and data source selection processing on the coding information based on a preset rule engine to obtain a corresponding candidate data source; performing state evaluation on the candidate data sources to obtain a state evaluation result, and screening out a target data source from the candidate data sources according to the state evaluation result; obtaining a query statement corresponding to the data query request; sending the query statement into the target data source based on the connection pool so as to execute query processing on the query statement through the target data source to obtain a query result; and returning a query result to the client. The processing efficiency and the processing flexibility of the data query request are improved.

The application discloses a kind of methods for supporting the unified management and query of multi-cluster heterogeneous data source, belong to big data technical field, comprising: S1: start DataLinks Executor component service;S2: user uploads custom driver for specific data source, and uses the custom driver to configure data source, then click test connection;S3: after test connection passes, enter unified query menu page, select the data source of test connection pass, and write the SQL statement that needs to be executed;S4: when executing SQL statement, Druid connection pool records the SQL statement executed, and DataLinks Executor regularly collects the data source information in the Druid connection pool, and carries out summary statistics;S5: the load balancing algorithm of DataLinks Executor selects the node with lower load in the summarized data source information, and allocates query task to each node.Greatly simplify the configuration process, so that user can more conveniently and efficiently manage and operate multi-cluster data source, thereby solve the problem of multi-cluster data source connection and control.

The invention discloses a data center management system and management method fusing multiple databases, and the system comprises a data source layer which comprises a relational database, a continuous database and an artificial intelligence vector database, and collects the basic feature data of each database in real time, including the total data amount D, the daily average update record number U, the field dimension F and the concurrent access peak value C; the data access layer is internally provided with a self-adaptive connection pool module, establishes connection with the data source layer through JDBC and ODBC protocols and a database native interface, calculates a dynamic connection number N based on a concurrent access peak value C collected by the data source layer, and records response delay T of each connection at the same time; the data fusion layer comprises a semantic mapping engine and a heterogeneous conversion module, the semantic mapping engine constructs a cross-library ontology model based on the field dimension F of the data source layer, and semantic matching is achieved by calculating the cosine similarity Sim of entity attributes. According to the invention, data center management can be carried out more comprehensively.

This invention provides a method and system for accessing a database interface layer using SAS language based on C++ middleware. The method includes the following steps: calling C++ middleware to obtain SAS statements from SAS language analysis software; identifying database information in the SAS statements through word segmentation, and establishing a database connection pool based on the database information by calling the database connection module of the C++ middleware; using the C++ middleware to identify the SAS statements and convert them into database-recognizable statements; and using the C++ middleware to pass the database-recognizable statements to the corresponding database through the database connection pool. This solves the problem in the prior art that SAS language analysis software cannot directly and efficiently connect to general databases, and enables high-performance, high-reliability, and secure data access.

The invention discloses a containerized system start-stop control method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: configuring a process termination signal hierarchical response mechanism of a container platform, and controlling a process termination process through a preset maximum downtime parameter; triggering a service unavailable mark based on the process termination signal, closing a monitoring port according to the service unavailable mark to intercept a new request, and cancelling interface information of the service registration center; monitoring operation states of the application layer thread pool and the timed task cluster, and waiting for all active requests of the application layer thread pool and uncompleted tasks of the timed task cluster to be processed within a preset timeout period; and sequentially closing the log framework, the thread pool and the database connection pool according to a preset resource release sequence, completing safe release of resources, and recording a log state. The method has the technical effects of guaranteeing service continuity in a containerized deployment or updating process, ensuring data integrity and realizing safe and orderly release of resources.

The application provides a database access method and device. The method comprises the following steps: determining a virtual IP address pre-solidified by a micro-service application; sending a connection request to the virtual IP address, so that a database connection agent on a database server side returns database metadata according to the connection request; receiving the database metadata; determining a target driver type according to a database category, loading a database driver corresponding to the target driver type, and creating a database connection pool by using the database driver according to database connection information. In the method, the micro-service application can receive the database metadata returned by the database connection agent, which comprises a database category and database connection information, by sending a connection request to the virtual IP, and dynamically constructs a database connection pool according to the database metadata. The process does not require the application to perceive information of a real database, shields differences of heterogeneous databases, realizes zero modification of configuration, and significantly reduces the workload of application deployment and operation and maintenance.

The invention discloses an end-to-end connection system, which comprises a client, a cloud platform and an equipment end, and is characterized in that the cloud platform acquires an equipment serial number and a user identifier; the cloud platform obtains an external network address of the equipment end according to the equipment serial number; the cloud platform inquires whether unused connection resources exist in a connection pool or not according to the user identifier and the external network address of the equipment side; and under the condition that the connection resources exist, end-to-end connection between the client and the equipment end is established through the connection resources. Through the method and the device, the technical problem of resource waste caused by repeated NAT traversal in related technologies is solved.

Communication connections with databases, such as when deployed as a database layer of a system architecture, are resource intensive to establish, take down, and maintain. However, from the applications' perspective, deployed in a business logic layer, more database connections result in less delay interacting with the databases. In order to optimize system resources, systems and methods are disclosed that create an optimized number of database connections and / or application connections, that are then maintained and shared by the applications via a proxy intercepting the communications and forwarding the communications to the database via the optimized number of pooled database connections. The connections may be multiplex so that simultaneous communications may be hosted concurrently.

This invention discloses a method and system for remote access connection pooling of core network element devices, relating to the field of network connectivity technology; including: Step 1: Encapsulating key information in the network element device docking process, creating and configuring a connection pool for each network element device, and uniformly initializing the connection pool through a connection pool manager; Step 2: Designing the network communication model of the connection pool: utilizing device-related classes to inherit the relevant classes of SSH general connection and SSH general driver in the SSH communication client, and overriding client methods to implement personalized service customization; Step 3: Obtaining the network element name and command to be issued based on the request parameters, obtaining the corresponding connection pool of the network element device through the connection pool manager for connection, calling the command issuance method corresponding to the connection pool based on the connection, completing the network element device interaction, and returning the response result.