270 results about "Connection pool" patented technology

The invention provides a Dores front-end node-based service proxy method and system, and relates to the technical field of database access request processing, and the method comprises the steps: collecting front-end node performance indexes, extracting node performance characteristics through employing multi-head recursive attention and causal convolution solution, extracting inter-node correlation characteristics through combining graph attention convolution, and obtaining a database access request; the method comprises the following steps: constructing a node state vector, optimizing a multi-layer connection pool parameter by utilizing an online learning algorithm, establishing a node incidence matrix based on comparative learning, constructing a node performance probability graph evolution model through variational reasoning and Monte Carlo sampling, predicting a node performance change trend, and receiving a database access request; and generating a query plan by adopting an edge attention message passing mechanism, carrying out resource evaluation in combination with performance trend prediction, carrying out scheduling decision making by utilizing multi-agent reinforcement learning and Monte Carlo tree search, carrying out fault detection and migration according to node performance deviation, and updating a probability graph evolution model.

The embodiment of the invention provides a distributed server cluster log processing method and device, and the method comprises the steps: constructing a distributed server cluster log collection network, and dynamically distributing collection tasks through a load balancing scheduling center. An incremental data acquisition channel is designed, efficient transmission is realized by using a websocket long connection pool, and the transmission efficiency is optimized in combination with real-time compression coding and a repeated data detection mechanism. An intelligent log analysis model is constructed, the intelligent log analysis model comprises an anomaly detection sub-model, a mode recognition sub-model and an event association analysis sub-model, anomaly score calculation, log classification labeling and multi-dimensional association analysis are achieved based on a deep learning method, and visual display and data export functions are provided. According to the method, the defects of the traditional technology in the aspects of distributed acquisition, data transmission, intelligent analysis and the like are effectively overcome, and the performance and practicability of a log processing system are remarkably improved.

The invention provides an integrated metadata management method and system based on data source expandability, and relates to the technical field of data sources, and the method comprises the steps: executing real-time detection on a plurality of heterogeneous data sources through a PF4J plug-in framework, collecting connection information, and establishing a dynamic connection pool; obtaining a table structure, a field attribute and an association relationship list, and assembling the table structure, the field attribute and the association relationship list into JSON format data; performing analysis and standardization processing on the data, and constructing an association degree calculation matrix; generating a standardized data structure description file; establishing a three-layer index directory, and detecting structure change and updating; and calculating a data table reference dependency relationship, and generating a DAG data flow path. According to the method, efficient integration and management of heterogeneous data sources are realized, and the metadata processing efficiency is improved.

The invention provides a performance optimization method and system for cross-data-source paging query, and relates to the technical field of data processing, and the method comprises the steps: calculating a data distribution weight and a data source routing table, determining an optimal data access path, and splitting a paging query request into sub-query tasks; dynamically adjusting the number of parallel query threads based on the load state and distributing sub-query tasks; establishing a local index and adaptively adjusting an index structure; dynamically adjusting the connection number of the connection pool and acquiring data in batches; performing feature analysis according to a query result, determining the size of a data fragment, and selecting an optimal merging strategy to merge the data; constructing a cache access mode graph, and aggregating and storing related data based on the access association degree; and optimizing a performance strategy set through reinforcement learning, adjusting a cache updating interval, a data access path and a feature matrix, and finally obtaining a query result. According to the method, the cross-data-source paging query efficiency can be effectively improved, the query delay is reduced, and the overall performance of the system is improved.

The invention relates to the technical field of network security, and particularly discloses a front digital boundary security switching method and system based on an optical one-way transmission characteristic and a storage medium, and the method comprises the steps that a front digital boundary security gateway of a service extranet receives a request, analyzes a context parameter of the request, and stores the context parameter of the request; legality verification is carried out on the carried exchange account, the key and other request parameters, and request context parameters are signed and encrypted; after the legality verification is passed, the service extranet front digital boundary security gateway encapsulates the request context parameter into a request data packet, a unidirectional data transmission optical shutter connection pool from the extranet to the intranet is used, and the request data packet is sent to the intranet front digital boundary security gateway after the connection is obtained; and after receiving the request data packet, the intranet front digital boundary security gateway decrypts and verifies the signature of the request message in the request data packet, reconstructs the request message after the verification is successful, and sends the request message to a real target server to obtain a result.

The invention provides a streaming data processing method, system and device based on Flink and a medium, and belongs to the technical field of data processing.The method comprises the following steps that data streams are received from different data sources, a connection pool is established according to the types of the data sources, and data in the data streams are processed; dividing the data stream into a plurality of windows for processing based on event time, storing the processing state of the data stream in an increment form, and performing partition management; task faults in the data processing process are detected, task restarting and load balancing are automatically carried out, meanwhile, data stream processing delay is monitored, and computing resources are dynamically adjusted when the delay exceeds the standard; and monitoring the state of the data stream in real time, carrying out visual display, and giving an alarm when the data stream is abnormal. According to the method, mass real-time data streams from different data sources are received and preprocessed, so that efficient and stable processing is realized; and the stability and the real-time performance of the system are guaranteed by means of mechanisms such as fault detection and load balancing.

The invention provides a TCP long connection server automatic expansion method and system. The method comprises the following steps: acquiring and monitoring various load indexes of a server in real time; receiving and analyzing the real-time load data, and judging whether resources need to be expanded or reduced currently according to a preset threshold value; based on the current load trend and historical data, performing short-term load prediction by adopting an extended decision algorithm; when expansion or reduction conditions are met, expansion or reduction operation is started, and a capacity expansion and reduction decision instruction is output; configuring a plurality of load balancing strategies, and distributing the received TCP long connection request to each service instance; and dynamically adjusting the size of a connection pool in the TCP long connection service cluster according to the system load condition and the capacity expansion and contraction decision instruction, and executing TCP connection load balancing operation after resource expansion or reduction. According to the invention, dynamic adjustment of resources is realized through technical means of load monitoring, automatic expansion, load balancing, dynamic connection management and the like, so as to meet service requirements under different load conditions.

The invention provides a multi-version dynamically expandable data source method and device, a terminal and a storage medium, and relates to the technical field of database data source loading. The method comprises the steps that a JAVA SDK package is created, the JAVA SDK package comprises core dependence, a core entity class, a core interface class, a database connection pool management core class, thread isolation, a routing data source class and basic attributes and target data source mapping of the routing data source class, and the target data source mapping is used for storing a loaded database connection pool; initializing a system database connection pool in the application system based on a JAVA SDK packet; after a system database connection pool in an application system is initialized, a new database connection pool is dynamically expanded in the application system based on a JAVA SDK package. The system has the design characteristics of strong dynamic expansion capability and loose coupling, the flexibility and expansibility of the system can be greatly improved, and the development and maintenance cost is reduced.

The invention discloses a database connection pool management method and device, equipment and a storage medium, and relates to the technical field of database management, the method comprises the following steps: creating a corresponding cluster connection pool for each cluster according to cluster parameter information and cluster configuration information, the cluster connection pool comprising a plurality of connection clusters, the connection cluster comprises a plurality of target connections, and the target connections carry first tags and timers; creating a database cluster pool according to the corresponding relationship between the cluster identifier and the cluster connection pool; and managing the target connection based on the database cluster pool, the first tag and the timer. The database connection resources are managed in a unified manner through the two-stage pool, so that the overhead of connection establishment and disconnection is reduced, and the processing efficiency is improved. And assigning a label to each connection, and upgrading the database connection pool into unequal stateful connections. And the timer is used for marking and tracking the use condition of each connection, so that the life cycle of the connection can be managed, and the management efficiency of the connection pool is improved.

The invention provides a dynamic database connection pool adjustment method and system, and relates to the technical field of computers, and the method comprises the steps: collecting the operation state data of a target dynamic database connection pool in real time, and sending the data to a configuration center; the configuration center determines a connection number adjustment index based on the received operation state data of the connection pool in combination with a connection request of the current target dynamic database connection pool; and based on the connection number adjustment index, the configuration center dynamically updates the maximum connection number of the connection pool, and notifies the connection pool module to update. The operation state data of the target dynamic database connection pool acquired in real time is calculated by using the configuration center in combination with the connection request to obtain the connection number adjustment index, and then the maximum connection number of the connection pool is dynamically updated by using the connection number adjustment index, so that the external service capability of the connection pool is effectively improved, and stable operation of a service system is ensured.

The invention provides a data query method, system and device and a storage medium, and the method comprises the steps: defining access connection information of a new database, and adding the new database into a connection pool; defining corresponding relations between tables and fields among different databases in the connection pool, and storing the corresponding relations between the tables and the fields among the different databases in a centralized warehouse; according to a preset business requirement, a preset query statement corresponding to each table in each database is disassembled, and the preset query statement is packaged in a storage process corresponding to the table, and the storage process is a pre-compiled SQL statement; searching a data mapping rule corresponding to the query entity data in the warehouse according to the user query request, and matching a target table of at least one target database in the connection pool according to the data mapping rule; and returning a query result based on a preset query statement corresponding to the target table. Data in different databases can be integrated, and the data management efficiency is effectively improved.

The invention provides a multi-node dynamic switching method and a multi-node dynamic switching system for a MongoDB (MongoDB). The method comprises the following steps: acquiring configuration information of a plurality of MongoDB nodes, and establishing a data access layer based on the configuration information; through a data access layer, acquiring index data marks of the MongoDB nodes about a plurality of operation indexes according to a hierarchical frequency acquisition task configured by each MongoDB node; in combination with a node weight preset value of the MongoDB node, converting the node performance index into a weight value of the MongoDB node; based on a connection pool management mechanism and the node available state data, a request distribution strategy is determined according to the weight value, and the request distribution strategy is used for indicating the distribution proportion of the MongoDB requests distributed to the MongoDB nodes; and performing state detection on the MongoDB node through a data access layer, marking the detected MongoDB node as an unavailable state when detecting that the MongoDB node meets an abnormal state condition, and updating available state data of the node. The problems that in the prior art, manual intervention is needed, the real-time load condition of the node cannot be sensed, and the code coupling degree is high are solved.

The invention relates to the technical field of network communication, and discloses a multi-path routing proxy method, a storage medium and a system, and the method comprises the steps: identifying the session type, protocol type and session feature parameters of a data flow, querying corresponding performance index coefficients according to the session type of the data flow, and obtaining a multi-path routing proxy according to the corresponding performance index coefficients. And calculating the score of each potential path in the path connection pool according to the real-time performance indexes of the potential paths. And matching with a preset demand type according to the session type, the protocol type and the session feature parameter, and determining a data distribution strategy according to the matched demand type. And finally, according to the determined data distribution strategy, calculating the number of paths corresponding to the data flow, selecting the potential paths with the corresponding number with the top scores as the distribution paths of the data flow, and completing the distribution of the data flow along the corresponding distribution paths according to the corresponding data distribution strategy. According to the method, the transmission quality of the routing system for different types of application data can be improved.

The invention relates to a double-token authorization refreshing method, device and system based on dynamic expiration tolerance. The method comprises the following steps: modeling the dynamic expiration tolerance according to the number of requests per second qps of a key API, the response time Tresp of the key API and the utilization rate U of a database connection pool; for a first request of which the access token is expired and the refresh token is not expired, if the expiration duration of the access token corresponding to the request is within the corresponding dynamic expiration tolerance, setting a distributed lock for the request, triggering a token refresh mechanism at the same time, and after the token refresh mechanism is executed, releasing the distributed lock; and for a subsequent request of the first request, if the expiration duration of the access token corresponding to the request is within the corresponding dynamic expiration tolerance, the token refreshing mechanism is not triggered, and normal access is directly allowed. According to the method, modeling is performed on the dynamic expiration tolerance, and the pain points and the problems that in the prior art, expiration tolerance time parameters are fixed, and static configuration is not flexible are solved.

The invention provides a connection pool management method and device, equipment and a storage medium, and relates to the technical field of computers. The method is applied to configuration equipment and comprises the steps that operation data sent by a client side and aiming at a target connection pool are received, the target connection pool is a connection pool related to a target application, the operation data comprise the number of times that the target connection pool accesses a server side in a preset time interval, and the server side is used for providing background service for the target application; based on the operation data, connection pool parameters corresponding to the target connection pool are adjusted to obtain adjusted connection pool parameters, and the connection pool parameters are used for describing the target connection pool; and sending the adjusted connection pool parameters to the client, so that the client configures the target connection pool based on the adjusted connection pool parameters. The adjusted connection pool parameters are more fit with the real use condition of the target connection pool, dynamic adjustment of the connection pool parameters of the target connection pool is achieved, and then the connection pool management effect is improved.

The invention discloses an improved method for logging in a weak network by a mobile phone number through one key and electronic equipment, and belongs to the technical field of communication, and the method comprises the following steps: a main processor creates a pre-connection pool, records a pre-connection domain name and a port list, and executes pre-connection; meanwhile, various parameters required by number prefetching are prepared, and a number prefetching request is sent; the network processor creates a retry controller, and the retry controller intelligently determines a corresponding retry mode according to the failure reason and the network state; unnecessary repeated operation is avoided by means of implementing a breakpoint resume type retry mechanism, error type detection and classification processing, network availability judgment and automatic reconstruction, private network IP change detection and URL reconstruction, an exponential backoff algorithm to control a retry interval and the like, and a staged timeout mechanism is introduced, so that the retry efficiency is improved. The personalized reasonable timeout time of different stages is set, the problem that a certain stage consumes too long time is avoided, and the network request success rate of the SDK in the weak network environment is improved.

Methods for managing a connection pool for cloud-based applications accessing databases are provided. Aspects include establishing a plurality of connections to one or more databases, receiving connection requests from a plurality of cloud-based applications to access the one or more databases, and allocating the connection requests to among the plurality of connections, where the allocation is determined based at least in part on one or more connection pool parameters. Aspects also include monitoring a usage of the plurality of connections by the plurality of applications and updating the one or more connection pool parameters based on an analysis of the usage of the plurality of connections by the plurality of applications.

The invention discloses a data source switching method and device, a medium and electronic equipment, and the method comprises the steps: in response to monitoring that a data access method is called, determining a to-be-switched target data source in at least two candidate data sources based on an annotation associated with the data access method; based on a connection pool configured for the target data source in a memory, accessing the target data source to switch from an initial data source to the target data source; wherein the candidate data source is registered in advance by adopting a data source registration method, and the data source registration method is operation in data source management and routing classes. By executing the technical scheme provided by the invention, the dynamic switching of the data sources is realized, and the switching flexibility of the data sources and the switching efficiency of the data sources are improved.

The present application discloses a data transmission method, device and computer storage medium, and relates to the field of network communication technology. The method is applied to a data transmission proxy module on the sending side, and includes: receiving a data transmission request, determining the service priority corresponding to the data transmission request; obtaining the current utilization rate of the transmission link between the export route and the export route on the receiving side; if the current utilization rate is greater than or equal to a first threshold, when there is an idle TCP connection in the TCP connection pool, and there is a data transmission request with the service priority of the first priority, and there is no ongoing data transmission, the data packet corresponding to the data transmission request with the service priority of the first priority is compressed according to the first compression level; and the compressed data packet is transmitted using the idle TCP connection. The present application effectively improves the data transmission efficiency and reduces the packet loss caused by the competition for bandwidth resources.

The invention belongs to the technical field of data processing, and discloses an active intelligent operation and maintenance monitoring method for a data center, which comprises the following steps: step 1, component modeling and topology configuration; 2, carrying out distributed health detection and data acquisition; 3, performing real-time health assessment and anomaly detection; 4, performing intelligent alarm and root cause analysis; 5, unified operation and maintenance and closed-loop control are carried out; and step 6, dynamically optimizing the intelligent operation and maintenance strategy. According to the method, a monitoring object and a dependency relationship are clarified through component modeling and topological configuration, and specific scenes, such as multi-mode acquisition, active detection, index pulling, log analysis, coverage message queue theme accumulation and database connection pool exhaustion, of distributed health detection are combined. The quantitative health score is calculated based on the preset scoring model, and by combining with the dynamic baseline learned by the ARIMA or LSTM algorithm, the module abnormity can be actively detected in different periods and weekly updating, and the problems of fault discovery lagging and incomplete monitoring coverage are solved.

The invention discloses a data transmission method, system and device and a storage medium, is applied to a host end in a non-uniform memory access architecture, and relates to the technical field of data transmission, and the method comprises the steps that a target path mapping table from a target end is received and analyzed, and a plurality of monitoring entrances are obtained; based on the target path mapping table, establishing a plurality of connection paths corresponding to the plurality of monitoring entrances in sequence, and generating a connection pool; in response to a data read-write request generated by the host end for a target end, determining a target processor domain corresponding to the target storage unit according to the target path mapping table, and selecting a connection path associated with the target processor domain from the connection pool as a current read-write path; and monitoring and judging whether the current read-write path is available or not, and if not, reselecting. The data transmission method can be combined with NUMA architecture perception, path state monitoring and a policy-driven selection mechanism to at least solve the problems that in the prior art, path selection lacks topology perception and multiple network cards are difficult to aggregate and use.

The invention discloses a method for switching a data source when a springboot service interface is called based on an xxl-job timed task, and relates to the technical field of computers, and the method comprises the steps of S1, obtaining performance data collected once per second, generating a JSON structured log file, and obtaining a target data source identifier and connection information, S2, loading the target data source identifier through a preset scheduling engine interface, dynamically adjusting the size of the connection pool to 50 connections, and determining that a new data source connection is activated; according to the method for switching the data sources when the springboot service interface is called based on the xxl-job timed task, whether switching operation meets service requirements or not is analyzed according to performance indexes, the refresh frequency is dynamically adjusted and configured, and an overtime automatic rollback mechanism is set, so that efficient and stable switching in a multi-data-source scene is achieved, and the overall performance and reliability of a system are improved.

The invention discloses a microservice gateway performance optimization method and system based on real-time flow monitoring and analysis. The invention relates to the technical field of microservice gateways. The real-time flow monitoring module is deployed on a gateway service layer in a sidecar mounting mode to collect flow data of a gateway in real time, wherein the flow data comprise request response time, the number of requests, the number of error requests and system resource use conditions. By dynamically adjusting the key parameters of the gateway, such as the thread pool parameter, the connection pool parameter and the load balancing strategy, the resource allocation and request processing flow is optimized, so that the throughput of the gateway is remarkably improved, the request response time is shortened, and the error rate is reduced. The real-time monitoring and performance evaluation mechanism provided by the invention is helpful to timely discover and solve potential performance bottlenecks and security problems, thereby greatly enhancing the stability of the gateway. Meanwhile, configuration and strategies are dynamically adjusted, so that the gateway can better deal with sudden flow and abnormal conditions, and continuous and stable operation is ensured.

The invention relates to the technical field of data processing, and discloses a log data batch updating method and device, equipment and a medium, and the method comprises the steps: obtaining a full-amount historical log in a preset database in real time, converting the full-amount historical log into an event stream, and writing the event stream into preset message middleware, obtaining a partition ID of the total historical log in the message middleware, generating a global unique identifier bound with a partition according to the partition ID to obtain a distributed ID, generating a consumer ID associated with the user identity information according to the distributed ID by calling a pre-designed distributed algorithm, and sending the consumer ID to the message middleware. And carrying out lock-free parallel updating on the database according to the consumer ID based on connection pool preheating and asynchronous submission to obtain an updating result, carrying out consistency verification on the updating result to obtain a verification result, and confirming the updating result according to the verification result. And the batch updating efficiency of the log data is improved.

The invention provides a college entrance examination volunteer filling assisting method and system based on a large language model, and the method is characterized in that the method comprises the steps: presetting a question template and an SQL template according to the volunteer filling scene demands and a database schema; analyzing a natural language query Q by adopting a large language model and designing a cue word, and combining a question template sequence number and an analysis result to generate a structured query condition set CQ; generating a database SQL query statement according to the CQ in combination with a preset SQL template; and managing database connection by using a connection pool technology, executing SQL query, generating a result set RS, and outputting information to a user. Through the method, the query efficiency and the interaction friendliness of the volunteer filling auxiliary system are improved.

The invention provides a heterogeneous database query method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out security verification processing on a plurality of SQL execution requests sent by a user side, and carrying out SPI mechanism scanning processing on a plurality of database plug-ins in a heterogeneous database plug-in directory based on a high-performance database connection pool after the security verification processing is passed, determining a target database plug-in package corresponding to each SQL execution request; and performing metadata query processing on the SQL execution request in the target data source based on a database operation client interface and a metadata service interface under the target database plug-in package, and determining a structured tree result. According to the method, an extensible and dynamically integrated heterogeneous database plug-in mechanism is realized, unified access of various databases is supported, the platform maintenance cost is remarkably reduced, hierarchical metadata of any type of databases is obtained through a consistent metadata service interface, and the user experience and the cross-database query efficiency are improved.

The invention provides a data direct-connection writing method and system and electronic equipment, and belongs to the technical field of data processing, and the method comprises the following steps: writing to-be-written data into a buffer channel of a streaming acquisition framework; extracting the to-be-written data from the buffer channel according to a preset batch size to form a data batch, and determining a target database table according to information contained in the data batch; according to the target database table, constructing the data batch into a write-in request conforming to a target database batch write-in interface specification; and sending the write-in request to the target database table through the connection pool of the preset transmission protocol. Through data batch extraction based on the buffer channel and dynamic determination of the target database table, the batch writing requests are constructed and are directly connected and transmitted through the connection pool, so that the real-time performance and stability of data writing are improved in a large-scale real-time data writing scene.

The invention provides a method and system for accessing a database interface layer through an SAS language based on C + + middleware. The method comprises the following steps that the C + + middleware is called to obtain SAS statements of SAS language analysis software originating from the C + + middleware; performing word segmentation identification on database information in the SAS statement, and calling a database connection module of the C + + middleware based on the database information to establish a database connection pool; identifying the SAS statement by using the C + + middleware and converting the SAS statement into a database recognizable statement; the C + + middleware is used for transmitting the database recognizable statements to the corresponding database through the database connection pool, the problem that in the prior art, SAS language analysis software cannot be directly and efficiently connected with a universal database is solved, and high-performance, high-reliability and safe data access can be achieved.

The invention relates to the technical field of big data, in particular to a Java-based multi-data-source access big data method and system, comprising the following steps: constructing a system architecture comprising a data source management module, a connection pool management module, a data query module and a log and monitoring module; the Java-based multi-data source access big data method and system provided by the invention have the beneficial effects that unified access, management and operation of various types of data sources are realized through a standardized interface and a flexible framework design. According to the technology, an adapter mode and a factory mode are adopted, different data source drives can be dynamically loaded and managed, meanwhile, a unified data query interface is provided, developers do not need to care about implementation details of underlying data sources, the development work is greatly simplified, and the flexibility and expandability of the system are improved.

Methods for managing a connection pool for cloud-based applications accessing databases are provided. Aspects include establishing a plurality of connections to one or more databases, receiving connection requests from a plurality of cloud-based applications to access the one or more databases, and allocating the connection requests to among the plurality of connections, where the allocation is determined based at least in part on one or more connection pool parameters. Aspects also include monitoring a usage of the plurality of connections by the plurality of applications and updating the one or more connection pool parameters based on an analysis of the usage of the plurality of connections by the plurality of applications.