A data isolation method, device, equipment and storage medium for stress testing scenarios

Abstract

The present invention discloses a data isolation method, device, equipment and storage medium for a stress testing scenario. The method includes: dividing and routing all traffic flowing into a system gateway to obtain processed traffic; identifying the processed traffic through middleware, wherein traffic with a stress testing identifier is stress testing traffic, and traffic without a stress testing identifier is real traffic; inputting the identified real traffic into the main database of the business, and inputting the data generated by the identified stress testing traffic into the main database in the in-memory database. The present invention adopts the method to achieve complete data isolation, will not generate dirty data, will not affect normal online users, can be used directly in a real online environment, obtain real performance indicator data, and provide an accurate and reliable reference for system expansion and operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data isolation method, apparatus, device, and storage medium for a stress testing scenario. Background Art

[0002] To more accurately assess the true performance of the entire system, stress testing the entire system is necessary, based on a live online environment and actual business scenarios, by simulating massive user requests. Since stress testing is conducted in a live online environment, it is crucial to prevent stress test data and traffic from contaminating or interfering with the production environment. Specifically, to ensure that stress testing is safe and controllable, without impacting normal user use or causing any data contamination in the online environment, the data and status generated by stress test traffic must not be mixed with normal user data and status. Storage isolation is essential to prevent the generation of corrupted data.

[0003] In common performance stress testing scenarios, if storage is involved, the code logic related to storage implementation needs to be processed in the business logic. The cost of storage isolation modification is very high, especially when it comes to database isolation. The industry usually adopts the isolation solution of building a shadow database. The DB budget will be more than twice the normal budget. In addition, continuous manpower is required for shadow database maintenance and follow-up data cleanup and recovery during stress testing.

[0004] Currently, common performance stress testing solutions in the industry focus on data isolation, with the following main methods:

[0005] 1. In the test environment, interface performance tools such as AB and Jemter are used to simulate high-concurrency and high-traffic performance stress for testing. New databases and data tables are not created separately for the data generated by the stress test traffic. In other words, normal user traffic data and traffic data generated by the stress test tool are stored in the same database and data tables, and the data are mixed together. Usually, the data table structure is modified in the existing database table to add relevant label fields. The label fields are used to distinguish between data generated by the stress test tool and data generated by normal user behavior. This stress testing method cannot achieve complete data isolation. All data is mixed together. The traffic data generated by the stress test tool will cause data pollution in the database, forming dirty data, which may affect normal user behavior and statistical reports, and bring high hidden dangers and risks to the business. Therefore, this performance stress testing method has great limitations and cannot be used in real online environments and real business scenarios. It can only be used by testers in a test environment.

[0006] 2. Internet companies are implementing full-link stress testing solutions that use traffic tagging to identify and customize stress testing traffic across the entire business chain. By building a shadow database, the data generated by stress testing traffic is stored separately in the shadow database. This avoids the generation of dirty data, provides transparency to online businesses, and does not affect normal production business logic, approaching real-world scenarios. However, this approach is costly and requires significant manpower for business transformation and ongoing maintenance. It requires customization across the entire business process, including the overall architecture, middleware, storage, and environment deployment topology. It also carries the risk of impacting existing businesses.

[0007] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0008] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a data isolation method, device, equipment and storage medium for a stress testing scenario, aiming to solve the problems that the existing data isolation methods cannot achieve complete data isolation, have limitations in use, have many modification links, are costly, and pose hidden dangers to existing businesses.

[0009] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0010] In a first aspect, the present invention provides a data isolation method for a stress testing scenario, comprising:

[0011] Split and route all traffic flowing into the system gateway to obtain processed traffic;

[0012] The processed traffic is identified by the middleware, wherein the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic;

[0013] The identified real traffic is input into the main database of the business, and the data generated by the identified stress testing traffic is input into the in-memory database.

[0014] The method, wherein the segmenting and routing of all traffic flowing into the system gateway includes:

[0015] Use the Nginx reverse proxy server as the system gateway and set up the Nginx reverse proxy server on the business system end;

[0016] All traffic flowing into the Nginx reverse proxy server is segmented and routed by the Nginx reverse proxy server;

[0017] The total traffic includes the stress test traffic generated by the traffic generator and the real traffic generated by real online users.

[0018] The method, wherein the middleware is used to identify the processed traffic, wherein the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic, includes:

[0019] Using the database component as middleware to provide traffic identification and processing capabilities, and detecting whether the processed traffic contains stress testing indicators through the database component;

[0020] If the processed traffic contains a stress test identifier, the processed traffic type is stress test traffic;

[0021] If the processed traffic does not contain a stress test identifier, the processed traffic type is real traffic;

[0022] The stress test identifier is used to identify the stress test traffic generated when the traffic generator generates the stress test traffic.

[0023] The method, wherein, after inputting the identified real traffic into the main database of the business and inputting the data generated by the identified stress test traffic into the in-memory database, further includes:

[0024] Adjusting the parameter information of the memory database according to the parameter information of the main database;

[0025] Initialize the in-memory database's table structure by referring to the new table structure of the main database.

[0026] Synchronize the main database table data to the in-memory database to complete the loading of existing data;

[0027] Set basic parameters according to the main database basic parameters, including: IP port, storage capacity and character encoding;

[0028] Performance parameters are set according to the main database performance parameters, including connection queue and response delay.

[0029] The method, after inputting the identified real traffic into the main database of the service, further includes:

[0030] The identified real traffic data is copied to the slave database, the data in the slave database is kept updated synchronously with the data in the master database, and the slave database and the master database are switched seamlessly.

[0031] The method, wherein, after adjusting the parameter information of the memory database according to the parameter information of the main database, further comprises:

[0032] When the data storage structure in the main database changes, the change in the data storage structure is synchronized to the in-memory database.

[0033] The method, wherein, after adjusting the parameter information master database of the memory database according to the parameter information of the master database, further comprises:

[0034] When a stress test completion signal is detected and a stress test data deletion instruction is received, the response data in the memory database is deleted.

[0035] In a second aspect, an embodiment of the present invention further provides a data isolation device for a stress testing scenario, comprising:

[0036] Traffic distribution module is used to split and route all traffic flowing into the system gateway to obtain processed traffic;

[0037] A traffic identification module identifies the processed traffic through the middleware, wherein the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic;

[0038] The traffic isolation module is used to input the identified real traffic into the business's main database and input the data generated by the identified stress testing traffic into the in-memory database;

[0039] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the data isolation method for the stress testing scenario as described in any one of the above solutions are implemented.

[0040] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data isolation method for the stress testing scenario as described in any one of the above solutions are implemented.

[0041] Beneficial effects: The present invention discloses a data isolation method, device, equipment and storage medium for a stress testing scenario, the method comprising: dividing and routing all traffic flowing into the system gateway to obtain processed traffic; identifying the processed traffic through a middleware, wherein traffic with a stress testing identifier is stress testing traffic, and traffic without a stress testing identifier is real traffic; inputting the identified real traffic into the main database of the business, and inputting the data generated by the identified stress testing traffic into the memory database. By adopting the method of the present invention, the data generated by the stress test traffic can be stored in the in-memory database, which is easy to clean up and will not affect the production DB library. Complete data isolation can be achieved, dirty data will not be generated, and normal online users will not be affected. Since no dirty data is generated, the method of the present invention can be used directly in a real online environment. Compared with stress testing in a test environment, since the stress test is performed in a real online scenario, the environment used by real online users is the same, and there is no problem of performance indicators being affected by environmental differences. Very real performance indicator data can be obtained, providing an accurate and reliable reference for system expansion and operation and maintenance. In addition, the method of the present invention does not require in-depth transformation of the business architecture, has high autonomy and controllability, can be completed in a closed loop within the team, and does not require a complicated and lengthy DB application and construction process. The test complexity and investment cost are very low, which facilitates the business to quickly build an environment for performance testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of a specific implementation of the data isolation method for the stress testing scenario provided by an embodiment of the present invention.

[0043] Figure 2 This is a flowchart of a specific implementation plan of a data isolation method for a stress testing scenario provided by an embodiment of the present invention.

[0044] Figure 3 A schematic diagram of the functional modules of a data isolation device in a stress testing scenario provided by an embodiment of the present invention;

[0045] Figure 4 A schematic structural diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0048] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0049] Currently, common performance stress testing solutions in the industry focus on data isolation, with the following main methods:

[0050] 1. In the test environment, interface performance tools such as AB and Jemter are used to simulate high-concurrency and high-traffic performance stress for testing. New databases and data tables are not created separately for the data generated by the stress test traffic. In other words, normal user traffic data and traffic data generated by the stress test tool are stored in the same database and data tables, and the data are mixed together. Usually, the data table structure is modified in the existing database table to add relevant label fields. The label fields are used to distinguish between data generated by the stress test tool and data generated by normal user behavior. This stress testing method cannot achieve complete data isolation. All data is mixed together. The traffic data generated by the stress test tool will cause data pollution in the database, forming dirty data, which may affect normal user behavior and statistical reports, and bring high hidden dangers and risks to the business. Therefore, this performance stress testing method has great limitations and cannot be used in real online environments and real business scenarios. It can only be used by testers in a test environment.

[0051] 2. Internet companies are implementing full-link stress testing solutions that use traffic tagging to identify and customize stress testing traffic across the entire business chain. By building a shadow database, the data generated by stress testing traffic is stored separately in the shadow database. This avoids the generation of dirty data, provides transparency to online businesses, and does not affect normal production business logic, approaching real-world scenarios. However, this approach is costly and requires significant manpower for business transformation and ongoing maintenance. It requires customization across the entire business process, including the overall architecture, middleware, storage, and environment deployment topology. It also carries the risk of impacting existing businesses.

[0052] In order to solve the problems that existing data isolation methods cannot achieve complete data isolation, have limitations in use, involve many transformation steps, are costly, and pose hidden dangers to existing businesses, this embodiment provides a data isolation method for stress testing scenarios. Specifically, Figure 1 and Figure 2 FIG. 1 is a data isolation method for a stress testing scenario according to a preferred embodiment of the present invention, which includes the following steps:

[0053] S10, splitting and routing all traffic flowing into the system gateway to obtain processed traffic;

[0054] S20. Identify the processed traffic through the middleware, where the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic;

[0055] S30: Input the identified real traffic into the main database of the business (DB main database), and input the data generated by the identified stress test traffic into the in-memory database.

[0056] Specifically, by adopting the method of the present invention, the data generated by stress testing traffic can be stored in the in-memory database, which is easy to clean up and will not affect the production DB library. Complete data isolation can be achieved, dirty data will not be generated, and normal online users will not be affected. Since no dirty data is generated, the method of the present invention can be used directly in a real online environment. Compared with stress testing in a test environment, since the stress testing is performed in a real online scenario, the environment used by real online users is the same, and there is no problem of performance indicators being affected by environmental differences. Very real performance indicator data can be obtained, providing an accurate and reliable reference for system expansion and operation and maintenance. In addition, the method of the present invention does not require in-depth transformation of the business architecture, has high autonomy and controllability, can be completed in a closed loop within the team, and does not require a complicated and lengthy DB application and construction process. The test complexity and investment cost are very low, which facilitates the business to quickly build an environment for performance testing.

[0057] In some embodiments, the real traffic may include but is not limited to the traffic generated by users. When the real traffic can be further refined into two or more types, a corresponding storage area can be allocated for each refined real traffic, and the response data of this type of service request can be routed to the corresponding storage area to improve the management efficiency of the real traffic data.

[0058] In some embodiments, the segmenting and routing of all traffic flowing into the system gateway includes:

[0059] Use the Nginx reverse proxy server as the system gateway and set up the Nginx reverse proxy server on the business system end;

[0060] All traffic flowing into the Nginx reverse proxy server is segmented and routed by the Nginx reverse proxy server;

[0061] The total traffic includes the stress test traffic generated by the traffic generator and the real traffic generated by real online users.

[0062] Generally, proxy servers are categorized as forward proxies and reverse proxies. Simply put, a forward proxy is like this: I'm a user who can't access a website directly, but I can access a proxy server that can access the website I can't. So, I first connect to the proxy server, tell it I need the content from the inaccessible website, and the proxy server retrieves it and returns it to me. From the website's perspective, there's only a single record of the proxy server fetching the content. In short, a forward proxy is a server that sits between the client and the origin server (origin server). To retrieve content from the origin server, the client sends a request to the proxy, specifying the target (origin server). The proxy then forwards the request to the origin server and returns the content to the client. Using a forward proxy requires special client configuration. For example, if I want to access the page localhost:8080 / views / test1, the server corresponding to the view doesn't have the resource test1; it's accessed from another server. In this case, the server corresponding to the view is using a reverse proxy. This means that the user only needs to send requests to a specific reverse proxy server. The user doesn't need to know (or, in fact, doesn't know) who is handling the request; the proxy server handles it all. The conclusion is that the reverse proxy is just the opposite. To the client, it is like the original server, and the client does not need to make any special settings. The client sends a normal request to the content in the namespace of the reverse proxy, and then the reverse proxy will determine where to forward the request (the original server) and return the obtained content to the client, just as if the content was originally its own. The typical use of a forward proxy is to provide a way for LAN clients inside a firewall to access the Internet. The forward proxy can also use the buffering feature to reduce network usage. The typical use of a reverse proxy is to provide servers behind the firewall for Internet users to access. The reverse proxy can also provide load balancing for multiple servers in the back end, or provide buffering services for slower servers in the back end.

[0063] Specifically, by setting up a proxy server, the access speed can be improved, unsafe information can be filtered, and some inaccessible target sites can be accessed. The present invention selects an Nginx reverse proxy server and sets it up on the business system end. All traffic passing through the Nginx reverse proxy server can be evenly split and routed to reduce the load of the business system. The processed traffic is then distributed to various business services of the business system. Only simple configuration is required to split all traffic for processing, which greatly reduces the pressure on the business system. In addition, by routing and splitting all traffic, stress testing traffic and real traffic can be dispersed to various specific business services, so that a business service is not allocated to stress testing traffic and real traffic at the same time, avoiding data confusion. In addition to the Nginx reverse proxy server, other similar reverse proxy software can also achieve the above effects.

[0064] In this embodiment, the Nginx reverse proxy server is selected because it has a load balancing function. Load balancing means that the load is distributed to multiple operating units for execution, such as Web servers, FTP servers, enterprise key application servers, and other mission-critical servers, so as to complete the work tasks together. Simply put, when there are two or more servers, requests are randomly distributed to the designated servers for processing according to the rules. Load balancing configuration generally requires the configuration of a reverse proxy at the same time, and the reverse proxy jumps to the load balancing. Nginx currently supports three built-in load balancing strategies and two commonly used third-party strategies.

[0065] In some embodiments, step S20 of identifying the processed traffic through the middleware, wherein the traffic with a stress test identifier is stress test traffic and the traffic without a stress test identifier is real traffic, specifically includes:

[0066] The database component is used as middleware to provide traffic identification and processing capabilities, and the database component is used to detect whether the processed traffic contains a stress testing identifier; if the processed traffic contains a stress testing identifier, the type of the processed traffic is stress testing traffic; if the processed traffic does not contain a stress testing identifier, the type of the processed traffic is real traffic; wherein, the stress testing identifier is used to identify the stress testing traffic generated when the traffic generator generates the stress testing traffic.

[0067] Specifically, after allocating traffic to specific business services, since business services are all based on middleware, in order to reduce the impact on business services and the cost of transformation, the traffic identification and processing logic is placed in the middleware. The database component is used as the middleware, and the database component is used to provide traffic identification and processing capabilities. The processed traffic is identified to determine whether the traffic contains stress testing identifiers. Traffic with stress testing identifiers is stress testing traffic, and traffic without stress testing identifiers is real traffic.

[0068] In this embodiment, after the step S300 of inputting the identified real traffic into the main database of the business and inputting the data generated by the identified stress testing traffic into the in-memory database, the following steps are included:

[0069] Adjusting the parameter information of the memory database according to the parameter information of the main database;

[0070] Initialize the in-memory database's table structure by referring to the new table structure of the main database.

[0071] Synchronize the main database table data to the in-memory database to complete the loading of existing data;

[0072] Set basic parameters according to the master database basic parameters, including: IP port, storage capacity and character encoding;

[0073] Performance parameters are set according to the main database performance parameters, including connection queue and response delay.

[0074] Specifically, the present invention synchronizes the data of the memory database with that of the main database, ensuring that the data in the memory database is the sum of all traffic data. Since it includes the traffic normally generated by users, the amount of data in the memory database table is the most complete, and it also ensures that the indicators obtained from the stress test will not be inaccurate due to too little data in the memory table, resulting in inaccurate performance results. In order to keep the memory database synchronized with the data of the main database, it is necessary to adjust the library table structure, inventory data, basic parameters, performance parameters and other information of the memory database according to some parameter information of the main database to ensure that the data of the memory database is synchronized with the data of the main database.

[0075] Furthermore, in a preferred embodiment, the memory database also calculates and displays performance status information in real time, and the performance status information includes, but is not limited to, concurrency, throughput, latency, CPU, memory and other information.

[0076] In some implementations, after the step of landing the identified real traffic in the main database of the service, the following step is further included:

[0077] The identified real traffic data is copied to the slave database, the data in the slave database is kept updated synchronously with the data in the master database, and the slave database and the master database are switched seamlessly.

[0078] Specifically, in order to prevent data loss due to downtime of the main database, the present invention also establishes a slave database, copies all traffic data in the main database to the slave database, keeps the data in the slave database synchronized with the data in the main database, and enables seamless switching between the slave database and the main database to ensure the validity of the data.

[0079] The way to keep the data in the slave database synchronized with the data in the master database is as follows: the master database writes the updated data information to the binary log file, obtains the modified content of the binary file from the slave database, writes it to the relay log, and then reads the information in the relay log to achieve data synchronization.

[0080] In some implementations, after adjusting the parameter information of the memory database according to the parameter information of the main database in step S40, the method further includes:

[0081] When the data storage structure in the main database changes, the change in the data storage structure is synchronized to the in-memory database.

[0082] Specifically, except for not participating in the switching between the master database and the slave database, the in-memory database must be synchronized and updated with the master database at all other times. Therefore, when the data storage structure of the master database changes, the in-memory database should also change its data storage structure accordingly to achieve data synchronization with the master database.

[0083] In some implementations, after the step of adjusting the parameter information of the memory database according to the parameter information of the main database, the step further includes:

[0084] After the stress test is completed, a stress test is performed on other application systems (the application system in this application is defined as the first application system, and other application systems may be the second application system, the third application system, etc.), and response data stored in the memory database is obtained. The response data is analyzed with the response data stored in the memory database obtained by the first application system to determine whether the first application system and / or other application systems need to be expanded.

[0085] If so, the application system that needs to be expanded is expanded, and after the expansion is completed, stress testing and analysis are performed again until it is determined that the first application system and other application systems do not need to be expanded.

[0086] In order to isolate stress testing data from real business data, the prior art often establishes a test environment corresponding to the production environment. In the test environment, stress testing traffic requests are generated by simulating production traffic requests, and these stress testing traffic requests are continuously applied to the business systems in the test environment. The traffic carrying capacity of the business systems is determined based on the stress testing data in the test environment. In this case, the test environment is isolated from the production environment, and the corresponding stress testing data and real business data are also stored in the system of the corresponding environment. Therefore, there is no interference between the stress testing data and the real business data. However, due to cost issues, the configuration of the test environment is often lower than that of the production environment. If the entire process of a business chain in the production environment involves two or more application systems, a test environment is often deployed for only one application system, resulting in the stress testing results not accurately reflecting the traffic carrying capacity of the real production environment when multiple application systems call each other. If the application system in the production environment is expanded and adjusted based on the stress testing results, the traffic carrying capacity of the application system will not match the production traffic. If the traffic carrying capacity of the application system is too large, it will waste server resources. If the traffic carrying capacity of the system is too small, it is easy for the system to crash due to high traffic.

[0087] In response to the above-mentioned defects, this embodiment, based on the premise of using a real system environment for stress testing proposed in the previous embodiment, if it involves the linkage of two or more application systems, the stress testing management device will analyze the response data in the stress testing service data storage area corresponding to the linked application systems after detecting that the stress testing is completed, and determine whether each linked application system needs to be expanded. If one or several application systems need to be expanded, these application systems will be expanded according to demand (for example, according to the required concurrency of the business chain), and after the expansion is completed, the application system will be stress tested and analyzed again to determine whether there are still application systems that need to be expanded. If so, the systems that need to be expanded will continue to be expanded until the demand is met.

[0088] In this way, it is possible to use the real system environment to stress test the application systems linked in the business chain, and perform targeted expansion based on the stress test results, thereby improving the matching degree between the traffic carrying capacity of the expanded application system and the production traffic, and improving the accuracy of expansion.

[0089] Of course, if the business chain involves only one application system and there is no application system that is called in conjunction with it, stress testing analysis and capacity expansion can also be performed by referring to this embodiment.

[0090] In some implementations, after adjusting the parameter information of the memory database according to the parameter information of the main database, the method further includes:

[0091] When a stress test completion signal is detected and a stress test data deletion instruction is received, the response data in the memory database is deleted.

[0092] In order to avoid the corresponding data generated by massive stress testing traffic occupying a large amount of storage resources for a long time and affecting the actual business operation of the application system, the present invention will delete the response data stored in the memory database when it detects the stress testing completion signal and receives the stress testing data deletion instruction to avoid it continuing to occupy storage resources after the stress testing is completed.

[0093] The present invention also provides a data isolation device for a stress test scenario, such as Figure 3 As shown, the device includes: a flow distribution module 10, a flow identification module 20, and a flow landing module 30, wherein:

[0094] Traffic distribution module 10, used to split and route all traffic flowing into the system gateway to obtain processed traffic;

[0095] The traffic identification module 20 is used to identify the processed traffic through the middleware, wherein the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic;

[0096] The traffic isolation module 30 is used to input the identified real traffic into the main database of the business and input the data generated by the identified stress testing traffic into the in-memory database.

[0097] The data isolation device in the stress testing scenario also includes:

[0098] The parameter adjustment module 40 is used to adjust the parameter information of the memory database according to the parameter information of the main database.

[0099] Furthermore, the flow distribution module 10 is also used to

[0100] The Nginx reverse proxy server is used as the system gateway and is set up on the business system end; all traffic flowing into the Nginx reverse proxy server is segmented and routed by the Nginx reverse proxy server.

[0101] The traffic identification module 20 is also used to provide traffic identification and processing capabilities using the database component as middleware, and detects whether the processed traffic contains a stress testing identifier through the database component. If the processed traffic contains a stress testing identifier, the processed traffic type is stress testing traffic; if the processed traffic does not contain a stress testing identifier, the processed traffic type is real traffic.

[0102] The parameter adjustment module includes:

[0103] The library table structure initialization unit is used to initialize the library table structure of the memory database by referring to the newly created library table structure of the main database;

[0104] The stock data loading unit is used to synchronize the main database table data to the in-memory database to complete the stock data loading;

[0105] A basic parameter setting unit, configured to set basic parameters according to the basic parameters of the main database, wherein the basic parameters include: IP port, storage capacity, and character encoding;

[0106] The performance parameter setting unit is used to set performance parameters according to the main database performance parameters, and the performance parameters include: connection queue and response delay.

[0107] The data isolation device in the stress testing scenario also includes:

[0108] A data backup module is used to copy the identified real traffic data to a slave database, keep the data in the slave database synchronized with the data in the master database, and seamlessly switch between the slave database and the master database;

[0109] A data update module is used to synchronize the changes of the data storage structure to the memory database when the data storage structure in the main database changes;

[0110] The data deletion module is used to delete the response data in the memory database when a stress test completion signal is detected and a stress test data deletion instruction is received.

[0111] The present invention also provides a computer device, such as Figure 4 As shown, the computing device includes a processor, memory, and a network interface connected via a system bus. The processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a data isolation method for a stress testing scenario.

[0112] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0113] In summary, the present invention discloses a data isolation method, device, equipment and storage medium for a stress testing scenario. The method includes: splitting and routing all traffic flowing into the system gateway to obtain processed traffic; identifying the processed traffic through the middleware, wherein the traffic with a stress testing identifier is the stress testing traffic, and the traffic without a stress testing identifier is the real traffic; inputting the identified real traffic into the main database of the business, and inputting the data generated by the identified stress testing traffic into the memory database. The effects achieved by the present invention through the above method are: 1. The data generated by the stress test traffic can be stored in the memory database, which is easy to clean up and will not affect the production DB library. Complete data isolation can be achieved, dirty data will not be generated, and normal online users will not be affected; 2. Since no dirty data will be generated, the method of the present invention can be used directly in a real online environment. Compared with stress testing in a test environment, since the stress testing is performed in a real online scenario, the environment used by real online users is the same, and there is no problem of performance indicators being affected by environmental differences. Very real performance indicator data can be obtained, providing an accurate and reliable reference for system expansion and operation and maintenance; 3. The method of the present invention does not require in-depth transformation of the business architecture, has high autonomy and controllability, can be completed in a closed loop within the team, and does not require a complicated and lengthy DB application and construction process. The test complexity and investment cost are very low, which facilitates the business to quickly build an environment for performance testing.

[0114] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0115] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data isolation method for a stress testing scenario, characterized in that: include: Split and route all traffic flowing into the system gateway to obtain processed traffic; The processed traffic is identified by the middleware, wherein the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic; The identified real traffic is entered into the business's main database, and the data generated by the identified stress testing traffic is entered into the in-memory database. After inputting the identified real traffic into the main database of the business and inputting the data generated by the identified stress testing traffic into the in-memory database, the following steps are also included: Adjusting the parameter information of the memory database according to the parameter information of the main database; Initialize the in-memory database's table structure by referring to the new table structure of the main database. Synchronize the main database table data to the in-memory database to complete the loading of existing data; Set the basic parameters of the memory database according to the basic parameters of the main database, including: IP port, storage capacity and character encoding; Setting the performance parameters of the memory database according to the performance parameters of the main database, wherein the performance parameters include: connection queue and response delay; After adjusting the parameter information of the memory database according to the parameter information of the main database, the method further includes: When a stress test completion signal is detected and a stress test data deletion instruction is received, the response data in the memory database is deleted.

2. The method according to claim 1, characterized in that The segmentation and routing of all traffic flowing into the system gateway includes: Use the Nginx reverse proxy server as the system gateway and set up the Nginx reverse proxy server on the business system end; All traffic flowing into the Nginx reverse proxy server is segmented and routed by the Nginx reverse proxy server; The total traffic includes the stress test traffic generated by the traffic generator and the real traffic generated by real online users.

3. The method according to claim 1, characterized in that The middleware is used to identify the processed traffic, wherein the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic, including: Using the database component as middleware to provide traffic identification and processing capabilities, and detecting whether the processed traffic contains stress testing indicators through the database component; If the processed traffic contains a stress test identifier, the processed traffic type is stress test traffic; If the processed traffic does not contain a stress test identifier, the processed traffic type is real traffic; The stress test identifier is used to identify the stress test traffic generated when the traffic generator generates the stress test traffic.

4. The method according to claim 1, wherein After inputting the identified real traffic into the main database of the service, the method further includes: The identified real traffic data is copied to the slave database, the data in the slave database is kept updated synchronously with the data in the master database, and the slave database and the master database are switched seamlessly.

5. The method according to claim 1, wherein After adjusting the parameter information of the memory database according to the parameter information of the main database, the method further includes: When the data storage structure in the main database changes, the change in the data storage structure is synchronized to the in-memory database.

6. A data isolation device for a stress testing scenario, characterized in that: include: Traffic distribution module is used to split and route all traffic flowing into the system gateway to obtain processed traffic; A traffic identification module is used to identify the processed traffic through the middleware, wherein the traffic with a stress test mark is the stress test traffic, and the traffic without a stress test mark is the real traffic; The traffic isolation module is used to input the identified real traffic into the business's main database and input the data generated by the identified stress testing traffic into the in-memory database; A parameter adjustment module is configured to adjust the parameter information of the memory database according to the parameter information of the main database; the parameter adjustment module includes: a library table structure initialization unit, configured to initialize the library table structure of the memory database by referring to the newly created library table structure of the main database; a stock data loading unit, configured to synchronize the table data of the main database to the memory database to complete the stock data loading; a basic parameter setting unit, configured to set basic parameters according to the basic parameters of the main database, the basic parameters including: IP port, storage capacity and character encoding; a performance parameter setting unit, configured to set performance parameters according to the performance parameters of the main database, the performance parameters including: connection queue and response delay; The data deletion module is used to delete the response data in the memory database when a stress test completion signal is detected and a stress test data deletion instruction is received.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the data isolation method for the stress testing scenario according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data isolation method for the stress testing scenario according to any one of claims 1 to 5 are implemented.

Images