Automatic testing method based on traffic replication and playback strategy and code interception technology

Abstract

The invention provides an automatic testing method based on a traffic copy playback strategy and a code interception technology. The method is characterized by comprising the following steps: S1, establishing a production environment snapshot; s2, copying the traffic; s3, performing simulation reconstruction on the playback environment context; s4, performing comparison verification on write operations of the database, and printing inconsistent places into a report for analysis; s5, optimizing a traffic replication and playback strategy; the method comprises the following steps: establishing a production environment snapshot, and recovering a test environment by using the snapshot in the test environment; and meanwhile, instrumentation does not need to be carried out in a production environment, only write request traffic of a service interface needs to be intercepted and recorded through a network level, and consistency of context and production of program operation is ensured through a code interception technology, so that consistency of an operation process and production is ensured, and a consistent result is generated.

Description

Technical Field

[0001] The present invention relates to the technical field of write interface automation testing technology, and particularly to an automation testing method based on a traffic replication and playback strategy and code interception technology. Background Art

[0002] The traffic replication and playback strategy is a popular automation testing method in the prior art. It does not require testers to write test cases or test code related to business logic.

[0003] The currently common traffic replication and playback strategy is to perform instrumentation in the production environment and record traffic and process data, and then integrate the traffic and data into the automation testing framework. When replaying the traffic, the data obtained by instrumentation is used for corresponding restoration at the program locations where data needs to be obtained or processed.

[0004] This strategy still faces two problems when in use: One is that it is necessary to perform instrumentation in the production environment, which has very high requirements for system stability and hardware resources, and the maintenance cost is very high and the risk is great. If there is a slight mistake, it will have a serious impact on production. The second is that the traffic playback cannot cover the end-to-end path. Its instrumentation method will intercept calls to middleware such as associated services and databases, and the testing is not comprehensive enough.

[0005] Therefore, there is an urgent need for a new solution to solve the defects and deficiencies existing in the above prior art. Summary of the Invention

[0006] In order to solve the defects and deficiencies existing in the prior art, the present invention provides an automation testing method based on a traffic replication and playback strategy and code interception technology.

[0007] The specific solution provided by the present invention is as follows: An automation testing method based on a traffic replication and playback strategy and code interception technology, characterized in that it includes the following steps: S1: Establish a snapshot of the production environment; S2: Traffic replication; S3: Simulate and reconstruct the playback environment context; including the following steps: S3.1: Select corresponding interception code to intercept the status data, and store, transfer and call the context information; S3.2: Reconstruct the user session context, and keep the primary key written into the database consistent with the production result, and at the same time generate the serial number field in the data; S3.3: Obtain the consistency data when reconstructing the context through business logic and request time; S3.4: Make nested calls to the program code; S4: Compare and verify the write operations to the database, and print the inconsistent parts in the report for analysis; S5: Optimize the traffic replication and playback strategy.

[0008] As a further preferred embodiment of the present invention, in step S1, Set the starting point of the production environment snapshot at the low peak period of the traffic; The production environment snapshot includes at least a database snapshot, a program package image, and a system configuration; among them, the database snapshot uses the dump technology and the binary log synchronization technology. If data changes are found during the database dump, the binary log record is used to apply the changes to the dump result; the program package image and the system configuration are obtained through the operation and maintenance system; And when loading the data snapshot into the test environment, set the test environment as an environment with access restrictions, and at the same time perform desensitization operations on user privacy data.

[0009] As a further preferred embodiment of the present invention, in step S2, keep the start time of traffic replication consistent with the start time of creating the production environment snapshot.

[0010] As a further preferred embodiment of the present invention, in step S3.1, when selecting the corresponding interception code to intercept the status data: Inject the interception code into the running process by means of dynamic injection; And code injection functions are respectively set at three points of preprocessing, end of processing, and exception on the interception framework; At the preprocessing point, modify the parameter input of the program method, establish a thread context, or intercept the continued execution of the code and directly return the required result; at the end of processing point, modify the return result; if a thread context is established at the preprocessing point, clean up the corresponding thread context at the end of processing and exception points.

[0011] As a further preferred embodiment of the present invention, in step S3.1, when storing, transmitting, and calling the context information: Store the context information in the space of the execution thread, and make it only visible within the thread, and set that the context information of the thread can be obtained through the code running method, and content can be added to or queried from it; When creating a child thread, pass the request information to the child thread, and the child thread creates its own context object according to the passed request information; At the same time, write the corresponding request information into the header of the call message during service call; In addition, when synchronizing data through middleware, intercept the write method, write the corresponding request message into the extended information, and read the original request information in the extended information and reconstruct the context by intercepting the consumption method in the consumption code.

[0012] As a further preferred embodiment of the present invention, in step S3.2, when reconstructing the user session context and keeping the primary key written into the database consistent with the production result, and generating the sequence number field in the data: When reconstructing the user session context, in the user login request, intercept the login verification to successfully create the user session, and simulate a successful result and return it to the caller; When keeping the primary key written into the database consistent with the production result, for the auto-increment primary key, intercept and inject the primary key field and primary key value consistent with the production during writing to make the write result consistent with the production result; When generating the sequence number field in the data, generate the sequence number field in the data based on the context information, and the context information at least includes the system time and the current increment sequence number.

[0013] As a further preferred embodiment of the present invention, in step S3.3, when obtaining the consistency data during context reconstruction through business logic and request time: Use business logic to help determine the uniqueness of business data. In the case of changes in unique fields, combined with the request time, further determine the specific data; Find the unique data write statement in the binary log through the table name, business data unique field, and time point, obtain its primary key or sequence number field value from it, and return it to the interception point program for database writing, and inject it into the data write statement to keep the write result in the test environment consistent with the production result.

[0014] As a further preferred embodiment of the present invention, in step S3.4, when making nested calls to program code, the called program code needs to keep isolation when intercepting and processing the context, and manage the context of nested calls in a stack manner.

[0015] As a further preferred embodiment of the present invention, in step S4, when performing comparison and verification on the write operation of the database, perform it in the order of time sequence, table, and primary key in turn.

[0016] As a further preferred embodiment of the present invention, in step S5, when optimizing the traffic replication and playback strategy, use the traffic playback result as the baseline result, and compare the running result of the test code with this baseline result: When the test result is consistent with the baseline result, it is determined that no optimization is required, and the interception optimization process ends; When the test results are inconsistent with the baseline results, first determine the type of inconsistent data: When the inconsistent data is non-status data, determine the code that needs to be analyzed; When the inconsistent data is status data, determine that interception optimization needs to be performed; The interception optimization method at least includes adding new interception points and adding new interception logics.

[0017] Compared with the prior art, the technical effects that the present invention can achieve include: 1) The present invention provides an automated testing method based on a traffic replication and playback strategy and code interception technology, which can complete the playback of production environment traffic in a test environment by establishing a production environment snapshot and restoring the test environment using the snapshot in the test environment; at the same time, no instrumentation is required in the production environment, and only the write request traffic of the service interface needs to be intercepted and recorded at the network layer, and then the code interception technology is used to ensure the consistency of the program running context and production, ensure that the running process is consistent with the production, and produce consistent results.

[0018] 2) The present invention provides an automated testing method based on a traffic replication and playback strategy and code interception technology, which will not perform any processing and intervention on the production environment during implementation and can be safely performed in the test environment.

[0019] 3) The present invention provides an automated testing method based on a traffic replication and playback strategy and code interception technology, which does not affect production, can achieve a safe and repeatable automated testing process, can perform end-to-end testing coverage on the system under test, and can handle test scenarios such as service refactoring, technology framework upgrade, and middleware replacement, with a wide range of applications; and does not require changes to business code and operation and maintenance operations, and can be independently deployed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The following shows the step flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0022] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "front end", "back end", "both ends", "one end", "the other end", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0023] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0024] [First Embodiment] As Figure 1 shown in the first embodiment provided by the present invention, this embodiment provides an automated testing method based on a traffic replication and playback strategy and code interception technology, including the following steps: S1: Establish a production environment snapshot; in this step, before implementing the traffic replication and playback strategy, it is first necessary to determine a starting point. Usually, the starting point of the production environment snapshot is set during the low peak period of traffic, such as at night, when there are very few user requests in production, which is suitable for establishing a snapshot at this time; The production environment snapshot includes at least a database snapshot, a program package image, and a system configuration; among them, the database snapshot uses the dump technology and the binary log binlog synchronization technology. If data changes are found during the database dump, the binary log binlog record is used to apply the changes to the dump result; while the program package image and the system configuration are obtained through the operation and maintenance system. And when loading the data snapshot into the test environment, it should be noted that the test environment needs to be set as an access-restricted environment, and at the same time, desensitization operations are performed on user privacy data to further prevent the leakage of production data.

[0025] S2: Traffic replication; Since traffic replication technology, as well as its open-source, commercial, or self-developed aspects, all belong to the existing technologies in this field, the implementation solutions of the technology itself are not within the scope of discussion in this embodiment. In the method provided in this embodiment, it is necessary to keep the start time of traffic replication consistent with the start time of creating a snapshot of the production environment to ensure that the data and status of the database during the playback of the replicated traffic and the generation of database operations in the test environment are the same as those in the production environment at that time, so as to achieve consistency between the playback write process and the production write process. That is, the start time of traffic replication should be the time when the database dump is completed. The database dump has a duration, and the database state corresponding to the end of the database dump is the database state that needs to be reproduced during traffic playback. Traffic replication starts after this time, and traffic playback starts from this state.

[0026] S3: Simulate and reconstruct the playback environment context; The context information of the process operation changes dynamically. It includes both its own state, such as memory and cache, and external states, such as the state of the database and the response data of third-party services.

[0027] The method to achieve traffic replication of the above state context is through simulation to avoid actual context replication. Specifically, it includes: after identifying the main state data, at the points where state data is obtained during traffic playback, return specified data for it through code interception, and this state data is consistent with the data in production. The simulation method refers to the process of intercepting calls, obtaining production data status, and returning data.

[0028] Specifically, it includes the following steps: S3.1: Select the corresponding interception code to intercept the state data, and store, transfer, and call the context information; In this embodiment, the interception means mainly targets calls involving context state consistency, as well as interception and corresponding processing at the call points during thread and service calls.

[0029] In this embodiment, when selecting the corresponding interception code to intercept the state data: Select the corresponding interception technology to establish interception according to the characteristics and support of the language. For example, for the Java language, interception technologies such as ASM, Byte Buddy, or jvm sandbox can be selected for corresponding interception. When intercepting the code, inject the interception code into the running process through dynamic injection to avoid changes to the business code and configuration changes in operation and maintenance.

[0030] Code injection functions are respectively set at three points on the interception framework: before preprocessing, after returning after processing, and after throwing exceptions. At the before preprocessing point, modify the parameter input of the program method, establish a thread context, or intercept the continued execution of the code and directly return the required result; at the after returning after processing point, modify the return result; if a thread context is established at the before preprocessing point, clean up the corresponding thread context at the after returning after processing and after throwing exceptions points.

[0031] For example: An interception point can be set at the business request entry position. This interception point is mainly responsible for determining whether the request is a traffic replay request. If the determination result is yes, a ThreadContext thread context object is established. This object will run throughout the life cycle of the request and pass the request information to each running link, and call the tryRemove method to request to remove the context object at the end of the operation. Interception code can also be automatically generated based on the system time.

[0032] When storing, transmitting, and calling context information: Store the context information in the space of the execution thread, and make it only visible within the thread. Set that the context information of the thread can be obtained through code running, and content can be added to or queried from it. In this way, the creation and use of the context are separated, so that the context information related to the main method and the overall logic can also be perceived and obtained in sub-methods or local logics. When creating a child thread, the child thread does not directly obtain the context of the parent thread. By passing the request information to the child thread, the child thread creates its own context object based on the passed request information; when intercepting the creation and destruction points of the child thread, find a suitable point according to the implementation of the program. For example, for those using the mapped diagnostic context MDC, the setContextMap method can be intercepted to inject the request information, so that the child thread can also obtain the request information and establish its own context object; for threads managed in a thread pool, the afterExecute point can be intercepted to clean up the context object.

[0033] At the same time, when making a service call, write the corresponding request information into the header of the call message. In addition, when synchronizing data through middleware, intercept the write method, write the corresponding request message into the extended information, and read the original request information in the extended information and reconstruct the context by intercepting the consumption method in the consumption code. Among them, the write method and the consumption method respectively correspond to both sides of the producer-consumer pattern. The write method is used to write message data into the middleware when the producer generates data, and the consumption method is used to obtain message data from the middleware when the consumer consumes data.

[0034] S3.2: Reconstruct the user session context, and keep the primary key written into the database consistent with the production result, and at the same time generate the serial number field in the data; in this step, When reconstructing the user session context, in the user login request, intercept the login verification to successfully create the user session. For those using third-party authentication such as mobile phone verification codes and WeChat platforms, intercept them all, prevent any possible calls to the third party, and simulate a successful result and return it to the calling party; When keeping the primary key written into the database consistent with the production result, for the auto-increment primary key, since the context of the primary key is determined by the database, intercept during writing and inject the primary key field and primary key value consistent with the production in the Structured Query Language (SQL) to make the write result consistent with the production result; The strategies for generating the serial number field in the business data are diverse and are based on the context information at the time of generation. Therefore, when generating the serial number field in the data, the serial number field in the data can be generated based on the context information. The context information at least includes system time and the current increment serial number, etc. Obtain the actual serial number generated during production according to the business primary key, and then directly return the result serial number to the calling method by intercepting the code when generating the serial number.

[0035] S3.3: Obtain the consistent data when reconstructing the context through business logic and request time; when obtaining the consistent data when reconstructing the context through business logic and request time: Use business logic to help determine the uniqueness of business data, that is, which fields in the business data can determine the uniqueness of this piece of data. For the case where the uniqueness fields change, combined with the request time, further determine the specific data; Find the unique data write statement SQL in the binary log (binlog) through the table name, business data uniqueness fields, and time point, obtain its primary key or serial number field value from it, and return it to the interception point program for database writing, and inject it into the data write statement SQL, so as to keep the write result in the test environment consistent with the production result. The primary key or serial number field value is the status data that needs to be kept consistent.

[0036] An example of the content of a typical binary log (binlog) is as follows: # datetime=2024-12-24_08:15:20 INSERT INTO `funds` (`id`, `bill_str`, `cid`, `biz_id`, `biz_str`, `biz_type`, `created_at`, `updated_at`, `approved_by`) VALUES (1044823, 'THJS241224000001', 10000181, 80944, 'TD241224000001', 20, '2024-12-24 08:15:21', '2024-12-24 08:15:20', 1); S3.4: Make nested calls to the program code; when making nested calls to the program code, the intercepted processing context of the called program code needs to be isolated to prevent contamination of other calls; one business process corresponds to one call, and there is a main method. The main method will call other sub-methods to form nested calls. The context needs to be isolated between nested calls and between nested calls and their callers. Because the context used by each call may overwrite the context of other calls, if mutual isolation is not performed, the context between them will be confused, so isolation is required. Of course, isolation is also required between this transfer process and other call processes.

[0037] And manage the context of nested calls in a stack manner, so that the local context of this call is guaranteed to be accessible only by this call, and its lifecycle is consistent with this call, that is, when the call ends, the local context is cleared at the same time.

[0038] S4: Perform comparison and verification on the write operations of the database in the order of time sequence, table, and primary key, and print the inconsistent parts to the report for analysis; S5: Optimize the traffic replication and playback strategy. When optimizing the traffic replication and playback strategy: Take the traffic playback result as the baseline result, and compare the running result of the test code with this baseline result. The comparison process is to compare the data writing process generated by playback with the data writing process in production. Specifically, the data writing operations of the test and production are compared in the order of time sequence, table, and primary key, and the comparison objects are all data fields that need to be concerned in all actual business scenarios.

[0039] When the test result is consistent with the baseline result, it is determined that no optimization is required, and the optimization process is intercepted and ends; When the test results are inconsistent with the baseline results, first determine the type of inconsistent data: When the inconsistent data is non-status data, determine that the code needs to be analyzed; non-status data should be stable and unchanging values regardless of when it runs. The inconsistent data being non-status data means that there may be a problem with the business logic, and the code needs to be analyzed rather than optimizing the interception logic; When the inconsistent data is status data, determine that interception optimization is required; The interception optimization method at least includes adding new interception points and adding new interception logic to optimize the interception to make the context consistency more accurate until the status data is consistent with production during playback.

[0040] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

Claims

1. An automated testing method based on traffic replication and playback strategy and code interception technology, characterized by: Includes the following step: S1: Create a snapshot of the production environment; S2: traffic replication; S3: Simulate and reconstruct the playback environment context; including the following steps: S3.1: Select the corresponding interception code to intercept the state data, and store, transfer and call the context information; S3.2: Rebuild the user session context and keep the primary key written into the database consistent with the production result, while generating the sequence number field in the data; S3.3: Obtain consistent data when rebuilding the context through business logic and request time; S3.4: nested calls to program code; S4: Compare and verify the write operations on the database, and print the inconsistencies in the report for analysis; S5: Optimize the traffic replication playback strategy.

2. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In the step S1, Set the starting point of the production environment snapshot to the low-peak period of traffic; The production environment snapshot at least includes a database snapshot, a program package image and a system configuration; wherein the database snapshot uses a dump technology and a binary log synchronization technology, and if data changes are found during the database dump, the binary log records are used to apply the changes to the dump result; the program package image and the system configuration are obtained through the operation and maintenance system; When loading the data snapshot into the test environment, the test environment is set as an access-restricted environment, and user privacy data is desensitized.

3. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In step S2, the start time of traffic replication is kept consistent with the start time of establishing the production environment snapshot.

4. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In step S3.1, when the corresponding interception code is selected to intercept the status data: Inject the interception code into the running process through dynamic injection; The interception framework is equipped with code injection functions at three points: pre-processing, end of processing, and exception. At the pre-processing point, modify the parameter input of the program method, establish the thread context, or intercept the continued running of the code and directly return the required results; modify the return result at the end of the processing; If a thread context is established at the preprocessing point, the corresponding thread context will be cleared at the end of processing and at the exception point.

5. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 4 is characterized in that: In step S3.1, when the context information is stored, transferred and called: The context information is stored in the space of the execution thread and is visible only in the thread. The context information of the thread can be obtained by running the code and content can be added or queried therein. When creating a child thread, the request information is passed to the child thread, and the child thread creates its own context object based on the passed request information; At the same time, when the service is called, the corresponding request information is written into the header of the call message; In addition, when synchronizing data through the middleware, the write method is intercepted, the corresponding request message is written into the extended information, and the original request information in the extended information is read and the context is reconstructed by intercepting the consumption method in the consumer code.

6. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In step S3.2, the user session context is rebuilt, and the primary key written into the database is kept consistent with the production result, and when the sequence number field in the data is generated: When rebuilding the user session context, in the user login request, the user session is successfully created by intercepting the login verification, and a simulated success result is returned to the caller; When keeping the primary key written into the database consistent with the production result, for the auto-increment primary key, the writing result is kept consistent with the production result by intercepting it during writing and injecting the primary key field and primary key value that are consistent with the production result; When generating the sequence number field in the data, the sequence number field in the data is generated based on the context information, and the context information at least includes the system time and the current incrementing sequence number.

7. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In step S3.3, when obtaining the consistency data when reconstructing the context through business logic and request time: Use business logic to help determine the uniqueness of business data. If there is a change in the unique field, combine it with the request time to further determine the specific data. Find the unique data write statement in the binary log by table name, business data uniqueness field and time point, obtain its primary key or sequence number field value, and return it to the interception point program written to the database, and inject it into the data write statement to keep the results written in the test environment consistent with the production results.

8. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In step S3.4, when nested calls are made to program codes, the called program codes need to be kept isolated when intercepting the processing context, and the context of the nested calls needs to be managed in a stack manner.

9. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In step S4, when comparing and verifying the write operation of the database, it is performed in the order of time sequence, table, and primary key.

10. The automated testing method based on traffic replication and playback strategy and code interception technology according to claim 1 is characterized in that: In step S5, when optimizing the traffic replication playback strategy, the traffic playback result is used as the baseline result, and the running result of the test code is compared with the baseline result: When the test results are consistent with the baseline results, it is determined that no optimization is needed and the interception optimization process ends; When the test results are inconsistent with the baseline results, first determine the type of inconsistent data: When the inconsistent data is non-state data, it is determined that code analysis is required; When the inconsistent data is status data, it is determined that interception optimization is required; The interception optimization method at least includes adding a new interception point and adding a new interception logic.

Images