Method and system for dynamically switching data sources for pressure measurement scene

Abstract

The invention discloses a method and a system for dynamically switching data sources for a pressure measurement scene. The invention relates to the technical field of computers. Selecting a specific data source from a predefined data source pool according to the configuration parameters, and initializing a data source required by the pressure test; the method comprises the following steps of: updating a data context (DataSource ContextHolder), so that the DataSource ContextHolder can correctly reflect a current data source configuration state; registering the MyBatis interceptor into an execution chain to ensure that the pressure measurement identifier can be captured and processed when SQL query or update is executed; when a pressure measurement request arrives, the MyBatis interceptor firstly captures the request and checks whether a request parameter contains a pressure measurement identifier or not; through the dynamic data source configuration and MyBatis interceptor technology, the capability of adjusting the data source in real time according to requirements in the pressure measurement process is realized. The complexity and limitation of manual configuration or data source configuration through predefined scripts are avoided, and the flexibility and adaptability of testing are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method and system for dynamically switching data sources for stress testing scenarios. Background Art

[0002] In the contemporary complex software system R & D cycle, performing stress testing (also known as load testing) is a core part of verifying the stability and reliability of the system under high-intensity operating conditions. This process aims to simulate the behavior of the system when facing extreme load conditions, and the load level is often set to several times or even dozens of times the normal operating load.

[0003] This artificially imposed high-load environment is intended to reveal potential bottlenecks, weaknesses, and performance thresholds of the system, so as to ensure that it can smoothly handle peak traffic during actual deployment. To this end, many existing technologies provide different stress testing techniques, such as:

[0004] CN201811605048.X discloses a software stress testing method and device (publication date: May 31, 2019); CN201510688555.4 discloses a testing method and testing system applied in testing software (publication date: January 27, 2016); CN202110804535.4 discloses a software stress testing method and device (publication date: September 28, 2021); CN201710867801.1 discloses a software system stress testing method, device, equipment, and storage medium (publication date: January 26, 2018);

[0005] However, traditional stress testing methods often directly use the data source configuration of the production environment, which is unable to meet the requirements when facing large-scale stress testing needs. The reason is that a single data source is extremely likely to become a bottleneck of system performance under high-intensity access pressure, not only unable to meet the requirements of stress testing for high concurrency and large data volume processing capabilities, but also leading to resource exhaustion and then system crashes, seriously affecting the accuracy and effectiveness of testing. Regarding the above existing technologies, they can be classified as follows:

[0006] (1) Static data source switching: Manually or through predefined scripts, different data sources are configured before stress testing. Lack of flexibility and difficult to handle real-time changes.

[0007] (2) Proxy-based load balancing: The application layer is distributed to different data sources through proxy methods. There are performance bottlenecks and complex implementation.

[0008] (3) High-availability data source solutions: Such as using clusters or distributed databases to improve the bearing capacity of data sources. These solutions are costly and complex to implement.

[0009] These solutions are either inflexible or complex to implement, and cannot meet the real-time dynamic switching requirements in the stress testing scenario.

[0010] For this reason, the present invention proposes a method and system for dynamically switching data sources for the stress testing scenario. Summary of the Invention

[0011] In view of this, the present invention hopes to provide a method and system for dynamically switching data sources for the stress testing scenario to solve or alleviate the technical problems existing in the prior art, that is: to solve the problem of poor flexibility in manually or through predefined scripts to configure different data sources before stress testing, and also to ensure that the application layer is distributed to different data sources through a relatively simple proxy method, while improving the bearing capacity of the data source, and at least providing a beneficial choice for this; the technical solution of the present invention is realized as follows:

[0012] In the first aspect, a method for dynamically switching data sources for the stress testing scenario:

[0013] (1) Overview:

[0014] The present invention aims to build a flexible and efficient stress testing system to simulate and evaluate the performance of the system under high load conditions. By initializing and configuring specific data sources, the solution can dynamically switch data sources according to actual needs to ensure that the behavior of the system under different loads can be accurately reflected during the stress testing process. Using the MyBatis interceptor technology, the solution captures and processes the stress testing identifier before the SQL execution, and thus dynamically routes to the corresponding data source according to the identifier value. While executing the SQL operation, the solution also monitors the load situation of the data source in real time. Once it is found that the load exceeds the threshold or the response is unstable, the data source switching logic is automatically triggered to ensure the continuity and stability of the test. In addition, the solution also has a recording and reporting function, which can generate a detailed stress testing report to provide strong data support for subsequent system optimization.

[0015] (2) Technical solution:

[0016] To achieve the above technical objectives, when the stress testing start instruction and configuration parameters are input, after identifying the stress testing identifier, the expected load level, and the data source configuration, the following operation steps are started.

[0017] 2.1 Step S1, data source initialization and configuration:

[0018] According to the configuration parameters, select a specific data source from the predefined data source pool to initialize the data source required for stress testing.

[0019] Update the data context (DataSourceContextHolder) so that it can correctly reflect the current data source configuration status.

[0020] 2.1.1 Step S100, Read configuration parameters:

[0021] Read the configuration parameters related to the data source from a configuration file (such as a file in XML, JSON, or YAML format). These include the data source type (MySQL or Oracle), database connection information (URL, username, and / or password), and connection pool parameters (maximum connection number, minimum connection number, and / or connection timeout).

[0022] 2.1.2 Step S101, Select a specific data source from the predefined data source pool:

[0023] Read the data source pool that includes all available data source configuration information; these data sources can be different database instances, different types of databases, or different environments (development, test, or production environments) of the same database.

[0024] According to the configuration parameters read in S100, match the data source type or database URL, and select the corresponding data source configuration from the data source pool.

[0025] 2.1.3 Step S102, Initialize the data source:

[0026] Use the selected data source configuration information to create the corresponding data source instance; call the API of the database connection pool, such as HikariCP, Apache DBCP2, or C3P0, to initialize a connection pool.

[0027] 2.1.4 Step S103, Update the data context

[0028] The data context (such as DataSourceContextHolder) is a container for storing the current data source configuration status. It allows other components in the system to obtain the current data source information at runtime. Set the initialized data source instance into the data context (bind the data source instance to a globally accessible identifier or thread-local variable) so that it can correctly reflect the current data source configuration status.

[0029] 2.2 Step S2, Set up the MyBatis interceptor:

[0030] Register the MyBatis interceptor into the execution chain to ensure that the stress test identifier can be captured and processed when executing SQL queries or updates.

[0031] 2.2.1 Step S200, Define the interceptor class:

[0032] Create a new Java class to implement the Interceptor interface provided by MyBatis. This interface defines the methods that an interceptor must implement, including intercept(Invocation invocation), plugin(Object target), and setProperties(Properties properties).

[0033] The interception logic lies in receiving an Invocation object that encapsulates the information of the intercepted method and related objects as a parameter. Through the Invocation object, the intercepted method, target object, and method parameters can be accessed.

[0034] In the interception logic, check the parameters in the SQL query or update request to find if there is a performance testing flag; if the performance testing flag is found, perform corresponding processing according to the value of the flag, including modifying the data source and recording logs.

[0035] 2.2.2 Step S201, configure the interceptor signature:

[0036] Use the @Intercepts annotation on the interceptor class to indicate that this class is a MyBatis interceptor. The @Intercepts annotation can contain multiple @Signature annotations inside, which are used to specify the method signatures that the interceptor will intercept.

[0037] For each @Signature annotation, three key attributes need to be specified: type, method, and args. The type attribute specifies the target type of the interceptor, that is, which interface's method will be intercepted;

[0038] The method attribute specifies the name of the intercepted method; the args attribute specifies the list of parameter types of the intercepted method, which is used to distinguish overloaded methods.

[0039] For the capture and processing of the performance testing flag, it is necessary to intercept the methods of interfaces related to SQL execution such as StatementHandler, Executor, etc. For example, to intercept the prepare method of StatementHandler, the performance testing flag needs to be captured and processed before the SQL statement is prepared for execution.

[0040] 2.2.3 Step S202, register the interceptor;

[0041] S2020, register in the MyBatis configuration file: If the project uses an XML configuration file to configure MyBatis, it is necessary to add it in the mybatis-config.xml file <plugins>Add under the label

[0042] <plugin>Element, register the interceptor into the MyBatis plugin chain. <plugin>The interceptor attribute of the element specifies the fully qualified class name of the interceptor.

[0043] S2021, Automatically registered by the Spring container: If the project uses the Spring framework to manage MyBatis beans, use @Component or other Spring annotations on the interceptor class to let the Spring container automatically scan and register the interceptor bean. Then in the MyBatis configuration file through <plugins>of the label <plugin>The element references an interceptor bean in the Spring container.

[0044] S2022, configure the interceptor properties: If the interceptor requires additional configuration properties, in

[0045] <plugin>Used inside the element <property>Sub - tags are used to set these properties. These properties will be passed to the setProperties(Properties properties) method of the interceptor so that the interceptor can obtain these configuration information during initialization.

[0046] 2.3 Step S3, Stress - test request processing:

[0047] When the stress - test request arrives, the MyBatis interceptor first captures the request and checks whether the request parameters contain a stress - test identifier; according to the presence or absence of the stress - test identifier and its value, the interceptor calls the data context to set the current data source type (such as "pressureTestDataSource" or "defaultDataSource").

[0048] 2.3.1 Step S300, Judge whether the stress - test identifier exists:

[0049] The interceptor checks whether the extracted information contains a stress - test identifier. The stress - test identifier is a specific parameter value, a request header, a cookie, or a special marker embedded in the SQL statement.

[0050] 2.3.2 Step S301, Set the data source according to the stress - test identifier:

[0051] If the stress - test identifier exists, call the data context according to the value of the identifier to set the current data source type; by binding the data source instance to the current thread, call the API of the data context to update the current data source configuration.

[0052] 2.3.3 Step S302, Continue to execute the request:

[0053] After setting the data source, the interceptor allows the request to continue execution. That is, SQL query or update operations will be executed using the data source set by the interceptor.

[0054] 2.3.4 Step S303, Clean up the data source configuration:

[0055] After the request is processed, the interceptor unbinds the data source instance from the current thread and cleans up the data source configuration associated with the current request.

[0056] 2.4 Step S4, Dynamic data source switching:

[0057] Before or during the SQL execution, obtain the data source key that should be used currently; according to the data source key, the AbstractRoutingDataSource routes the request to the corresponding data source.

[0058] 2.4.1 Step S400, calling method:

[0059] Before or during the execution of SQL, call the determineCurrentLookupKey method of the DynamicDataSource class to determine the data source key to be used based on the current thread local variable, request parameter, and user information; this method parses the context information internally and finally returns a data source key of string type, which corresponds to the key in the data source configuration and is used for subsequent data source routing.

[0060] 2.4.2 Step S401, routing to the corresponding data source:

[0061] Configure an instance of the abstract routing data source for the data source used by MyBatis, which is also an abstract class that implements the logic of data source routing.

[0062] 2.4.3 Step S402, setting the current data source key:

[0063] Before calling the SQL execution method, set the data source key obtained in S400 to the current data source key property of the abstract routing data source. This can be achieved by calling the setCurrentLookupKey method of the abstract routing data source.

[0064] 2.4.4 Step S403, performing data source routing:

[0065] When MyBatis executes an SQL query or update, call the getConnection method of the abstract routing data source to obtain a database connection. The abstract routing data source will select the corresponding data source from the data source pool based on the currently set data source key and return the corresponding database connection.

[0066] 2.4.5 Step S404, executing SQL:

[0067] Use the obtained database connection to execute SQL query or update operations. Since the data source has been dynamically switched according to the current context, the SQL operation will be executed using the correct data source.

[0068] After the SQL execution is completed, in order to avoid affecting subsequent operations, clear the currently set data source key by calling the clearCurrentLookupKey method of the abstract routing data source.

[0069] 2.5 Step S5, performing SQL operations:

[0070] SQL operations are executed on the selected data source to complete data query, insertion, update, or deletion. The load of the data source is monitored in real time to evaluate the performance status of the current data source. If it is found that the load of the data source exceeds the threshold or the response stability exceeds the threshold, the data source switching logic is triggered, and an alarm signal is sent.

[0071] 2.5.1 Step S500, execute SQL on the selected data source:

[0072] According to the data source dynamically switched in step S4, obtain the corresponding database connection through the data source pool (such as HikariCP or Druid, etc.); use the obtained database connection to execute SQL query, insertion, update, or deletion operations; process the returned results according to the type of SQL operation; for query operations, the result set needs to be mapped to a Java object.

[0073] 2.5.2 Step S501, monitor the data source load:

[0074] During the execution of SQL operations, the load of the data source is monitored in real time through the monitoring interface provided by the data source or the monitoring tool of the database management system.

[0075] 2.5.3 Step S502, evaluate the performance status:

[0076] Based on the monitoring data, evaluate the performance status of the current data source based on the metrics of CPU usage, memory occupancy, disk I / O, network bandwidth, and the query response time of the database and the preset thresholds.

[0077] 2.6 Step S6, data source switching and recovery:

[0078] According to the monitoring results and the preset switching strategy, automatically switch to the standby data source to ensure the continuity of the stress test.

[0079] After each SQL operation or periodically check whether the load and response stability of the data source exceed the set thresholds.

[0080] If it is found that the load of the data source exceeds the threshold or the response stability exceeds the threshold, trigger the data source switching logic, call the data source dynamic switching method in step S4, and switch the current data source to another alternative data source.

[0081] (III) Mechanism for solving technical problems:

[0082] 3.1 MyBatis interceptor combined with data context:

[0083] Using MyBatis interceptor technology, capture the stress test identifier before SQL execution, and dynamically set the current data source type according to the identifier value through the data context (DataSourceContextHolder). This approach simplifies the interaction between the application layer and the data source, enabling the application layer to be unaware of the specific data source configuration details.

[0084] Implement the dynamic data source routing logic by inheriting the AbstractRoutingDataSource abstract class. This class provides an abstract method determineCurrentLookupKey for determining the data source key to be used for the current request. By overriding this method, the data source key can be dynamically returned based on the current context (such as the stress test identifier).

[0085] 3.2 Improvement in flexibility:

[0086] By parameterizing the data source configuration and dynamically initializing the data source based on these parameters at runtime, flexible configuration of the data source is achieved. At the same time, using interceptor technology to decouple the data source switching logic from the application layer enables the application layer to be unaware of the specific data source configuration details.

[0087] 3.3 Proxy distribution and transparency:

[0088] Using the MyBatis interceptor as a proxy, intercept the request before SQL execution and dynamically set the data source type according to the context information. This proxy method is transparent to the application layer, allowing the application layer to achieve dynamic switching of the data source without modifying the code.

[0089] By monitoring the load status of the data source in real time and automatically switching to the standby data source when necessary, ensure the continuity and stability of the stress test. In addition, the dynamic data source configuration and switching mechanism also helps to improve the bearing capacity of the data source and avoid performance bottlenecks caused by overloading of a single data source.

[0090] In the second aspect, a system for dynamically switching data sources for stress test scenarios:

[0091] This system is used to implement the method for dynamically switching data sources for stress test scenarios described above, and it includes:

[0092] (1) A data source management module responsible for the initialization, configuration, and management of the data source:

[0093] Data source initialization: Select a specific data source from the predefined data source pool according to the configuration parameters and initialize the data source required for the stress test.

[0094] Data context update: Update the DataSourceContextHolder to reflect the current data source configuration status.

[0095] Interaction: Interact with the configuration management module to obtain configuration parameters, and interact with the data context module to update the current data source status.

[0096] (2) MyBatis interceptor module that registers interceptors into the execution chain and captures and processes stress test identifiers:

[0097] Interceptor registration: Register the interceptor into the MyBatis execution chain.

[0098] Stress test identifier processing: Capture requests before SQL execution, and check and process stress test identifiers.

[0099] Interaction: Interact with the MyBatis execution chain to register interceptors, and interact with the data source management module and the data context module to set the data source according to the stress test identifier.

[0100] (3) Stress test request processing module that processes stress test requests and sets the data source type according to the stress test identifier: Process stress test requests and set the data source type according to the stress test identifier. Interact with the MyBatis interceptor module to capture requests and process stress test identifiers, and interact with the data context module to set the current data source type.

[0101] (4) Data source dynamic switching module that dynamically switches the data source before or during SQL execution. Data source key acquisition: Call the determineCurrentLookupKey method of DynamicDataSource to obtain the data source key that should be used currently.

[0102] Data source routing: Route requests to the corresponding data source using the abstract routing data source according to the data source key.

[0103] Interaction: Interact with the data context module to obtain the current data source status, and interact with the data source management module to switch to the corresponding data source.

[0104] (5) SQL execution and monitoring module that executes SQL operations on the selected data source and monitors the load and performance of the data source:

[0105] SQL execution: Execute data query, insertion, update, or deletion operations.

[0106] Load monitoring: Real-time monitor the load and performance status of the data source.

[0107] Switching logic and alarm: If it is found that the data source load exceeds the threshold or the response stability exceeds the threshold, trigger the data source switching logic and send an alarm signal.

[0108] Interaction: Interact with the data source management module to perform SQL operations, and interact with the monitoring and alarm module to monitor the load and issue alarms.

[0109] (6) The data source switching and recovery module that automatically switches the data source according to the monitoring results and the preset switching strategy, and restores the default data source after the stress test:

[0110] Interact with the monitoring and alarm module to obtain the monitoring results, and interact with the data source management module to switch and recover the data source.

[0111] Compared with the prior art, the beneficial effects of the present invention are:

[0112] First, improve flexibility: Through dynamic data source configuration and MyBatis interceptor technology, the present invention realizes the ability to adjust the data source in real time according to requirements during the stress test. This avoids the cumbersome and limitations of manually configuring or configuring the data source through predefined scripts, and improves the flexibility and adaptability of the test.

[0113] Second, simplify the application layer code: The present invention encapsulates the data source switching logic in the interceptor and the abstract routing data source, so that the application layer code does not need to care about the specific data source configuration and switching details. This simplifies the code logic of the application layer, reduces the maintenance cost, and improves the readability and maintainability of the code.

[0114] Third, improve performance and stability: By real-time monitoring the load of the data source and automatically switching to the standby data source when necessary, the present invention ensures the continuity and stability of the stress test. This helps to discover potential performance bottlenecks and stability problems, and provides strong support for the optimization and upgrade of the system.

[0115] Fourth, enhance scalability: Based on the design of the abstract routing data source and configuration parameters, the present invention enables the system to easily add new data source types or adjust the existing data source configuration. This enhances the scalability of the system and enables the system to better adapt to the needs of future business development and technological changes.

[0116] Fifth, improve test efficiency: Through automated and dynamic data source configuration and switching, the present invention reduces the manual operations and time costs of testers. At the same time, the real-time monitoring and alarm mechanism also helps to timely discover and handle problems in the test, improving the test efficiency and accuracy. Description of the Drawings

[0117] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0118] Figure 1 Schematic diagram of the method flow of the present invention;

[0119] Figure 2 Schematic diagram of the MyBatis interceptor in step S2 of the present invention;

[0120] Figure 3 Schematic diagram of the execution method in step S3 of the present invention;

[0121] Figure 4 Schematic diagram of step S4 of the present invention. Detailed implementation manners

[0122] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific implementation manners of the present invention with reference to the drawings. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein. Those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below;

[0123] It should be noted that the embodiments in this specification are described in a progressive manner, and the key points of each embodiment are the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0124] Explanation of related terms:

[0125] (1) Configuration parameters: Values or options used to initialize the data source and set options related to the stress test.

[0126] (2) Data source: A physical or logical database instance that stores data and provides a data access interface.

[0127] (3) Dynamic switching: The process of automatically changing the data source at runtime according to conditions or rules.

[0128] (4) MyBatis interceptor: Intercepts requests before and after MyBatis executes SQL to add additional processing logic (such as processing stress test identifiers).

[0129] (5) Load balancing: Distribute requests to multiple data sources to optimize performance and resource utilization, and avoid overloading a single point.

[0130] (6) Data source context (DataSourceContextHolder): An information container that stores the current data source status, ensuring consistency when switching data sources.

[0131] (7) Execution chain: In MyBatis, the process in which a series of interceptors and core logic are executed in sequence.

[0132] (8) Data source type (such as "pressureTestDataSource" or "defaultDataSource"): A string that identifies different data source types, used to distinguish between the pressure test data source and the default data source.

[0133] (9) The determineCurrentLookupKey method of DynamicDataSource: The method that determines the data source key (i.e., the data source identifier) to be used currently during the dynamic data source switching process.

[0134] (10) Data source key: A string or identifier that uniquely identifies a certain data source, used to locate a specific data source in the data source collection.

[0135] (11) AbstractRoutingDataSource (duplicate item, already explained in point 4): An abstract class that implements the dynamic data source routing logic, routing requests to specific data sources based on the data source key.

[0136] Example 1: As Figures 1 to 4 shown, this example provides a method for dynamically switching data sources in the pressure test environment of the data stream of the online car-hailing platform, aiming to verify the stability and performance of the online car-hailing platform under high concurrency, and achieve dynamic switching of data sources and load monitoring. It includes the following steps S1 to S6.

[0137] In this example, regarding step S1: Data source initialization and configuration:

[0138] Read configuration parameters (step S100): Read the data source configuration from a YAML configuration file, including the data source type (such as MySQL), database connection information (URL, username, password), and connection pool parameters (maximum connection number, minimum connection number, connection timeout).

[0139] Select data source (step S101): The data source pool contains MySQL database instances in the development environment, test environment, and production environment. According to the configuration parameters, select the MySQL database in the production environment as the data source for the stress test.

[0140] Initialize data source (step S102): Use the HikariCP connection pool API to initialize the MySQL database connection pool in the production environment.

[0141] Update data context (step S103): Bind the initialized data source instance to a globally accessible thread-local variable so that the current data source configuration status can be correctly reflected in subsequent operations.

[0142] In this embodiment, regarding step S2: Set MyBatis interceptor:

[0143] Define interceptor class (step S200): Create a Java class named PressureTestInterceptor that implements the Interceptor interface of MyBatis. In the interception logic, check whether the parameters in the SQL request contain a stress test flag named pressureTestFlag.

[0144] Configure interceptor signature (step S201): Use the @Intercepts annotation on the PressureTestInterceptor class to specify that the method signature to be intercepted is the prepare method of StatementHandler.

[0145] Register interceptor (step S202): Since the project uses the Spring framework, we use the @Component annotation on the PressureTestInterceptor class to let the Spring container automatically scan and register the interceptor Bean. Then in the MyBatis configuration file through <plugins>The tag references an interceptor bean in the Spring container.

[0146] In this embodiment, regarding step S3: stress test request processing:

[0147] Determine whether the stress test flag exists (step S300): When a stress test request arrives, the PressureTestInterceptor interceptor first checks whether the parameters in the SQL request contain the pressureTestFlag.

[0148] Set the data source according to the stress test flag (step S301): If the pressureTestFlag is found and its value is true, the interceptor calls the data context to switch the current data source to a data source dedicated to stress testing (such as pressureTestDataSource).

[0149] Continue to execute the request (step S302): After setting the data source, the interceptor allows the SQL request to continue execution.

[0150] Clean up the data source configuration (step S303): After the SQL request is executed, the interceptor unbinds the stress test data source from the current thread and restores the default data source configuration.

[0151] In this embodiment, regarding step S4: dynamic data source switching:

[0152] Call the method (step S400): Before the SQL execution, call the determineCurrentLookupKey method of the DynamicDataSource class to determine the data source key (such as pressureTestDataSourceKey or defaultDataSourceKey) that should be used according to the stress test flag in the current thread local variable.

[0153] Route to the corresponding data source (steps S401 to S404): Configure an

[0154] instance of AbstractRoutingDataSource as the data source of MyBatis. Before calling the SQL execution method, set the determined data source key to the current data source key property of AbstractRoutingDataSource. When MyBatis executes the SQL, AbstractRoutingDataSource will route to the corresponding data source according to the current data source key and return a database connection. The SQL operation is executed using the obtained database connection. After execution, clean up the currently set data source key.

[0155] In this embodiment, regarding step S5: Execute SQL operations and monitoring:

[0156] Execute SQL on the selected data source (step S500): According to the dynamically switched data source, obtain the corresponding database connection through the HikariCP connection pool, and execute SQL query, insert, update, or delete operations.

[0157] Monitor the data source load (step S501): Use the monitoring interface provided by the MySQL database or a third-party monitoring tool (such as Prometheus, Grafana) to perform real-time monitoring of the data source load.

[0158] Evaluate the performance status (step S502): According to the monitoring data, evaluate the performance status of the current data source, including indicators such as CPU usage, memory occupancy, disk I / O, network bandwidth, and the query response time of the database. If any indicator exceeds the preset threshold, the data source switching logic will be triggered.

[0159] In this embodiment, regarding step S6: Data source switching and recovery: Check the load and response stability of the data source after each SQL operation or periodically. If it is found that the data source load or response stability exceeds the threshold, trigger the data source switching logic, call the data source dynamic switching method in step S4, and switch the current data source to the standby data source (such as the database instance in the development environment or test environment).

[0160] At the same time, send an alarm signal to notify relevant personnel to handle the performance problem. After the performance problem is solved, the data source can be switched back to the data source in the production environment manually or automatically.

[0161] It can be understood that through the configuration file and the dynamic switching mechanism, we can flexibly configure and switch different data sources without manually modifying the code or predefined scripts. This improves the flexibility and maintainability of the system. By monitoring the load and performance status of the data source and automatically switching to the standby data source when necessary, we can ensure the stability and availability of the system under high concurrency. At the same time, using a connection pool to manage database connections improves the connection efficiency and bearing capacity of the data source.

[0162] In summary, this solution uses a MyBatis interceptor to capture the stress test identifier in the SQL request and dynamically switches the data source according to the identifier. The routing logic of the data source is implemented through AbstractRoutingDataSource, and the corresponding data source is selected according to the current data source key. At the same time, by monitoring and evaluating the load and performance status of the data source, the data source switching logic is triggered to ensure the stability and availability of the system.

[0163] Example 2: This example will further provide an execution program for the method of dynamically switching data sources in the stress testing environment of the online car-hailing platform data stream. It uses Java as the programming language and utilizes the Spring framework, MyBatis ORM framework, and data source connection pool (such as HikariCP):

[0164] (1) Data source initialization and configuration: Define a data source configuration class and initialize the data source instance. Use Spring's @Configuration and @Bean annotations to complete:

[0165]

[0166]

[0167]

[0168] Among them, DynamicDataSource is a custom class that inherits from AbstractRoutingDataSource and is used to implement the dynamic switching of data sources.

[0169] (2) Set up MyBatis interceptors to capture and process stress testing identifiers:

[0170]

[0171]

[0172] (3) Stress testing request processing and dynamic data source switching: When a stress testing request arrives, the interceptor will first capture the request and set the current data source according to the stress testing identifier.

[0173] DynamicDataSourceContextHolder is a class used to store the current data source key. It uses ThreadLocal to ensure thread safety. The DynamicDataSource class will dynamically switch the data source according to the data source key stored in DynamicDataSourceContextHolder;

[0174] During the SQL execution process, use the monitoring interface provided by the data source or the monitoring tool of the database management system to monitor the load of the data source in real time. If it is found that the load exceeds the threshold, the data source switching logic can be triggered. According to the monitoring results and the preset switching strategy, we can automatically switch to the standby data source. After each SQL operation or periodically check the load and response stability of the data source. If an abnormality is found, the data source is switched.

[0175] (4) It is necessary to define the relevant configurations of the data source and the interceptor in the configuration file. Taking application.yml as an example:

[0176]

[0177]

[0178] It can be understood that the above solution improves the flexibility and maintainability of the system, enables quick configuration and switching of different data sources, ensures the stability and availability of the system under high concurrency, and improves the user experience. By monitoring and evaluating the load and performance status of the data source, performance problems can be detected and handled in a timely manner, ensuring the healthy operation of the system.

[0179] All of the above embodiments only express the implementation manners of the relevant actual applications of the present invention. The descriptions are relatively specific and detailed, but should not be construed as limitations on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

[0180] For those skilled in the art, it can be further realized that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0181] Meanwhile, those skilled in the art can understand that all or part of the processes in the methods of all the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to the memory, storage, database or other media provided in this application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.< / plugins> < / property> < / plugin> < / plugin> < / plugins> < / plugin> < / plugin> < / plugins>

Claims

1. A method for dynamically switching data sources for stress testing scenarios, characterized in that: After entering the stress test startup command and configuration parameters, identifying the stress test identifier, expected load level, and data source configuration, perform the following steps: S1, according to the configuration parameters, select a specific data source from the predefined data source pool, initialize the data source required for the stress test, and update the data context; S2, register the MyBatis interceptor to the execution chain; S3, when the stress test request arrives, the MyBatis interceptor first captures the request and checks whether the request parameter contains the stress test identifier; according to the existence and value of the stress test identifier, the interceptor calls the data context to set the current data source type; S4, before or during SQL execution, obtain the data source key that should be used currently; Based on the data source key, the abstract routing data source routes the request to the corresponding data source; S5, SQL operations are executed on the selected data source, and data query, insertion, update or deletion are completed, and the load of the data source is monitored in real time; S6, automatically switching to the backup data source according to the monitoring results and the preset switching strategy.

2. The method for dynamically switching data sources according to claim 1, characterized in that: The implementation method of S1 includes: S100, reading configuration parameters related to the data source from the configuration file; S101, reading a data source pool including all available data source configuration information; S102, using the selected data source configuration information, creating a corresponding data source instance; calling the database connection pool API to initialize a connection pool.

3. The method for dynamically switching data sources according to claim 2, characterized in that: In S1, the method for updating the data context is: binding the data source instance to a globally accessible identifier or a thread local variable.

4. The method for dynamically switching data sources according to claim 1, characterized in that: In S2, the implementation method of the MyBatis interceptor includes: S200, create a new Java class to implement the Interceptor interface provided by MyBatis, and receive an Invocation object that encapsulates the intercepted method information and related objects as a parameter; S201, use @Intercepts annotation on interceptor class; S202, automatically register the MyBatis interceptor through the Spring container and configure the interceptor properties.

5. The method for dynamically switching data sources according to claim 1, characterized in that: The implementation method of S3 includes: S300, checking whether the extracted information contains a stress test identifier; S301, if the stress test identifier exists, call the data context to set the current data source type according to the value of the identifier; update the current data source configuration by binding the data source instance to the current thread and calling the data context API; S302, after setting the data source, the interceptor allows the request to continue executing; S303, after the request is processed, the interceptor unbinds the data source instance from the current thread and cleans up the data source configuration associated with the current request.

6. The method for dynamically switching data sources according to claim 1, 2, 4 or 5, characterized in that: The implementation method of S4 includes: S400, determining the data source key to be used according to the current thread local variables, request parameters and user information; parsing the context information, and finally returning a data source key of string type, which corresponds to the key in the data source configuration; S401, configuring an instance of an abstract routing data source of a data source; S402, before calling the SQL execution method, setting the data source key obtained in S400 to the current data source key attribute of the abstract routing data source; S403, when executing SQL query or update, call the getConnection method of the abstract routing data source to obtain the database connection; S404: Use the acquired database connection to execute SQL query or update operation.

7. The method for dynamically switching data sources according to claim 6, characterized in that: In S5, the process of SQL operation includes: S500, according to the data source dynamically switched in S4, obtain the corresponding database connection through the data source pool; use the obtained database connection to execute SQL query, insert, update or delete operation; according to the type of SQL operation, process the returned result; for the query operation, the result set needs to be mapped to the Java object; S501, during the execution of the SQL operation, the load of the data source is monitored in real time through the monitoring interface provided by the data source or the monitoring tool of the database management system.

8. The method for dynamically switching data sources according to claim 6, characterized in that: In S6, after each SQL operation or periodically checking whether the load and response stability of the data source exceed a set threshold; If it is found that the data source load exceeds the threshold or the response stability exceeds the threshold, the data source switching logic is triggered, and the data source dynamic switching method in S4 is called to switch the current data source to another alternative data source.

9. A system for implementing the method for dynamically switching data sources as claimed in any one of claims 1 to 8, characterized in that: The system comprises: The data source management module is responsible for the initialization, configuration and management of the data source; Register the interceptor to the execution chain, capture and process the MyBatis interceptor module of the stress test mark; Process stress test requests and set the data source type stress test request processing module according to the stress test identifier; A data source dynamic switching module that dynamically switches data sources before or during SQL execution; SQL execution and monitoring module that executes SQL operations on selected data sources and monitors the load and performance of data sources; The data source switching and recovery module automatically switches the data source according to the monitoring results and the preset switching strategy, and restores the default data source after the stress test is completed.

10. The system according to claim 9, characterized in that: In the data source dynamic switching module, the request is routed to the corresponding data source using the abstract routing data source according to the data source key.

Images