Auxiliary test method and system for variable deployment strategy

Abstract

The invention discloses an auxiliary test method and system for a variable deployment strategy, aims to solve the problems of difficulty in parallel test and low regression test frequency caused by host resource contention and queuing during development and test of a financial transaction system, and aims to solve the problem that the deployment and operation and maintenance process cannot be tested, and provides an auxiliary test method and system for general development and test personnel. A deployment scheme which is easy to change or convenient to cut is provided, limited host resources can support more projects, interfaces capable of testing deployment and operation and maintenance links are provided at the same time, developers and testers can conveniently cover more test scenes, and the delivery quality of the whole system is improved.

Description

Technical Field

[0001] The present invention relates to the field of automated testing and automated deployment, and in particular to an auxiliary testing method and system for a variable deployment strategy. Background Art

[0002] Most of the automated testing and automated deployment tools on the market are based on the CI / CD pipeline or playbook script operation mode. The configuration and logic are written together without sufficient abstraction and separation, which makes reading, modification and maintenance very burdensome. At the same time, the execution process does not incorporate deployment and operation and maintenance related content into the test scenario, does not directly support test case design, and requires manual verification of some deployment and operation and maintenance related test cases.

[0003] Financial transaction systems are characterized by large scale, multiple modules, and high quality requirements. Due to the complexity of system architecture and dependencies, and the diversity of configuration and operation, financial transaction systems cannot be deployed and operated as simply as web applications. The configuration and operation of each module are very different. Deployment, start and stop operations may have some preconditions and post-events, and there is no concise and standardized deployment plan.

[0004] Usually, the server hardware cost and operation and maintenance cost of running a system of this scale are very high, so the allocated resources are usually very limited, and even several project teams may share a set of environments. According to past experience, developers and testers are often unwilling to change the operating environment or do not have the ability to change the operating environment. They prefer to use a stable, relatively complete, and exclusive operating environment so that they can concentrate on development and testing tasks. However, if the possible reduction of subsystems and submodules is not changed, too many resources will be occupied, resulting in a waste of host resources and serious contention and queuing, making it inconvenient for parallel testing, difficult for regression testing, and inefficient. Summary of the invention

[0005] The existing technology has problems such as difficulty in parallel testing and low frequency of regression testing caused by host resource contention and queuing during the development and testing of financial transaction systems, and the problem that the deployment and operation and maintenance processes cannot be tested.

[0006] In order to solve the above problems, the present invention provides an auxiliary testing method of a variable deployment strategy, comprising:

[0007] Define the application to be deployed in the form of a software package according to the specification or template, and expose parameters including configuration items and software interfaces.

[0008] Set the application to the corresponding relationship and the simplest configuration based on the host address and software package name. Provide default values ​​and default implementations for the parameters and logic required by other applications. Change these default values ​​through command parameters or configuration.

[0009] List and organize the application modules that need to be operated in the deployment plan, define groups, perform parallel operations within groups, and perform serial operations between groups. Define the deployment plan in a scripting manner, declare the correspondence between the host address and the application module in the deployment plan, provide other configuration items in the deployment plan, and overwrite the default parameter values ​​of the application modules.

[0010] Store and manage the configuration or script of the deployment plan in combination with the test case sets of each project;

[0011] Use different deployment plans or scripts for development, debugging or test case design, and tailor deployment plans;

[0012] Use the interfaces provided by the software package to start, stop, and force exit deployment operations in the test cases, call operation and maintenance related interfaces to view CPU and memory usage parameters, and check whether the operation log or business log meets expectations;

[0013] The manually executed deployment plan can be triggered through the CI / CD system to perform automated testing and automatically schedule execution.

[0014] An auxiliary test system for variable deployment strategy, comprising:

[0015] The remote command execution module uses the SSH protocol to connect to the host server. It does not need to pre-install the agent program on the server where the tested system is located. It supports the execution of remote commands and the operation of remote files, and supports the operation of modifying preset ini, xml, and json configuration files.

[0016] The configuration management module standardizes the definition of deployment plans and scripts, and has the functions of parsing deployment information, dependencies, and parsing execution logic according to the script;

[0017] The application definition module standardizes and defines common application operation instructions, including deployment operations such as downloading, installing, starting, stopping, forcibly stopping, status checking, and reloading configurations, as well as operation and maintenance operations such as CPU usage, memory usage, disk usage, and performance monitoring, and operations such as reading remote application logs, remote data files, and remote configuration files;

[0018] Instruction batch processing module, which supports the deployment of applications in the solution through instruction operations;

[0019] The script execution module supports the interpretation and execution of deployment-related logic in the form of scripts. The scripts include custom logic and required check items.

[0020] The automated testing module is used to execute test cases in batches. During the testing process, the test environment can be deployed according to the deployment plan or script provided by each use case set.

[0021] In a preferred manner, the application definition module stores the logical definition of the application in the form of a software package, which is provided to test case designers in the form of a software package, and the relevant interfaces are called in the use case for operation and verification; it supports the direct use of general templates to define new applications, including no need to add logic code separately, and applications with special parameters or dependencies can be controlled by adding logic code.

[0022] In a preferred manner, the instruction batch processing module transparently transmits the command to all or part of the modules defined in the deployment plan.

[0023] In a preferred manner, if the instruction batch processing module carries non-predefined instructions or instruction parameters, it will be transparently passed to all or part of the modules in the deployment solution and can be executed as long as the application supports it.

[0024] Preferably, the script execution module uses scripts when the deployment scheme needs to be frequently switched in an automated testing scenario.

[0025] In the preferred mode, the automated testing module has the functions of detecting test cases, detecting source code, and batch execution; the test case warehouse usually contains multiple test case sets, different test case sets use different deployment plans, which are deployed before executing each test case set. During the test, test results are collected according to the deployment plan, and a test report and recovery environment are generated after the test of the test case set is completed.

[0026] Beneficial effects of the present invention: In the present method and system, when developers and testers change the operating environment including the deployment scheme, in the most simplified case, they only need to modify the corresponding relationship between the host and the module, or drag and drop through a graphical interface. For general developers and testers, there is no need to learn the system architecture and dependencies, and they only need to focus on business function testing, which greatly improves efficiency. The host resources required by a system can be split for use by multiple projects, and the host resource allocation can be temporarily adjusted, and the deployment scheme can be modified at will, which generally achieves the goal of facilitating deployment changes and tailoring deployment. In terms of automated test scheduling and operation, it is no longer necessary to deploy and regress separately, but only to associate different deployment schemes with use case sets. The test system automatically adjusts the operating environment according to the deployment scheme, and can test multiple scenarios in parallel, which greatly improves the test efficiency. Because the resource utilization rate is improved and the deployment scheme can be automatically switched, each project can perform more regression tests according to actual needs. In terms of test scenario and test case design, test cases for operation and maintenance such as startup timeout, excessive resource usage, and performance jitter can be added, and the scenario coverage rate is significantly improved, thereby effectively improving the delivery quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flow chart of the auxiliary testing method of the variable deployment strategy of the present invention;

[0028] Figure 2 The architecture diagram of the auxiliary test system of the variable deployment strategy of the present invention. DETAILED DESCRIPTION

[0029] Embodiment 1:

[0030] In conjunction with the attached drawings, the main function of this system is to support batch deployment of applications and remote execution of instructions.

[0031] The main purpose is to fully abstract and simplify the deployment configuration, improve the convenience of using deployment tools, and at the same time directly serve the entire software development and testing process to meet the deployment flexibility requirements of different test scenarios.

[0032] Step S1, in view of the shortcomings described in the background technology, in order to improve the convenience of reading, modifying and maintaining the deployment plan, the system mainly uses the method of configuration and logic separation to support solving these problems, so as to achieve the goal of easy modification and convenient tailoring. The method includes:

[0033] The characteristic of this method is that it takes the application as the main perspective, abstracts and separates the logic related to application deployment and operation and maintenance operations, and finally defines an application module by two parts: configuration items (also called parameters) and specific implementation logic (called software package). This is the key point. Whether it is a simple scenario or a complex scenario, its application deployment and operation and maintenance operations can be abstracted into two parts: configuration parameters and operation logic. Among them, the host address and the application module software package name are the main deployment configurations. By changing the corresponding relationship of this configuration, the deployment plan can be modified, so that the host resources can be reasonably allocated conveniently.

[0034] Based on the usage scenarios and test scenarios, users can be divided into two categories. One category is called ordinary users, who are directly faced with simplified configuration items and are responsible for compiling specific deployment plans or scripts. The other category of users is called application logic developers, who are responsible for summarizing the configuration items required by the application module and implementing specific operation logic, such as uploading installation packages to remote hosts, starting remote processes, etc.

[0035] Host resource definition centrally manages the host resources that need to be used. The host information is listed flatly in the host resource configuration. The host information includes host alias, IP, port, SSH user name, SSH user password, SSH user key string or file address. The host alias is used in the deployment plan to distinguish different hosts. This method is easy to store and manage and can be reused in similar projects.

[0036] The application module configuration items are abstracted and separated according to the specific needs of the application module. The configuration items that must be configured are the software package name and the host address alias. The optional general configuration items are the group name, installation directory, application name, application path, application instruction routing, and whether to disable or skip the configuration items required for temporary use of the deployment plan. For special configurations of the application module, such as the configuration that needs to connect to the database, the application logic developer should implement it based on the basic functions provided by this system, associate it with the database application, or provide a default processing method. The user can determine whether to not configure (use the default processing method) or configure (override the default processing method of this configuration item) in the deployment plan according to needs.

[0037] The specific implementation logic described is, first of all, general module operations. The deployment operations that need to be implemented include installation, uninstallation, start, stop, forced exit, checking running status, waiting for the service status to be available, etc. The operation and maintenance operations that need to be implemented may include checking CPU occupancy, checking memory occupancy, checking network monitoring status, monitoring memory jitter, monitoring freezes, etc. The associated operations of the application module that need to be implemented include obtaining the directory where the application is located, obtaining the path where the application is located, obtaining the application log, obtaining the application running time and exit time, etc. The specific operation logic of the interface is implemented using popular programming languages, such as Python. This system provides some basic functions for remote operations, such as uploading, downloading, executing remote commands, etc. The logic developer implements specific operation logic based on these interfaces, and abstracts and separates configuration items (also called parameters), exposes related operation interfaces to the outside, and supports command line parameters as needed.

[0038] Definition of software package. The logic of each application is implemented in a script file. The system will automatically correspond to a unique application package name according to the directory structure. The format is "package directory name. application name", which is called the logical package name. Usually, the specific logic of the application only needs to be developed once by the developer. After the development is completed, it will be managed by a unified product warehouse (dependency package). The configuration of the application is maintained by ordinary users and managed by each project in the source code or use case warehouse. The configuration of the application will change with the changes in project requirements, which is called a variable deployment strategy.

[0039] In view of the large differences in configuration and operation of different application modules (each application has customized parameters and configurations), as well as the ease of use (complex dependencies and preconditions), the configuration of this method and system design has a most simplified standard (normalization), that is, the initial configuration only needs to reflect the correspondence between the application and the host, and can indicate which host the application is deployed on. This method and system mainly solve the problem of large differences in configuration and operation between applications by differentiating configuration parameters. When formulating deployment plans, ordinary users can use the default behavior defined by the application, or list parameters in the configuration for overwriting, or use temporary parameters for overwriting in manual execution or use case design. Therefore, the differences between different applications can be ignored, and the same instructions can be used for control. The method and rules include:

[0040] During the logic development phase, when separating configuration and logic, the separated configuration parameters must have logical default values;

[0041] Parameters not filled in the configuration file will use the application logic default values;

[0042] The parameters filled in the configuration file will replace the default values ​​of the application logic and serve as the parameters actually used in the application logic;

[0043] Named configuration items (parameters) obtained through command line, script or test case function call will overwrite the configuration parameters or logical default values ​​of the same name.

[0044] The above rules can be explained with a priority, that is, command line or function parameters are greater than configuration parameters, and configuration parameters are greater than logical default values.

[0045] When ordinary users need to modify the deployment plan, they only need to modify the correspondence between the application and the host, that is, only modify one configuration item (host name).

[0046] When there are differences in the test scenarios, you only need to overwrite the different configuration parameters. If you do not want to change the configuration file and only want to temporarily change the parameters once during operation, you can use command line parameters to overwrite. This method increases the flexibility of automated testing and manual use of this tool, which is called a variable deployment strategy.

[0047] Step S2, the configuration of the deployment scheme is used to manage the application modules that need to be operated. The method and system are also designed with a flexible packet routing positioning strategy method, which includes:

[0048] Each application will define a group name. If the application's configuration item list contains a group name, it will be assigned to the corresponding group. If no group name is defined, it will be assigned to the nameless group. The design of grouping is a virtual concept designed to increase the flexibility of operation.

[0049] The order in which the group names in each application configuration list appear in the configuration file will directly determine the execution order of the groups in the operation. The group that appears first will be executed first. At the same time, the rules for parallel operations within a group and serial operations between groups are agreed upon.

[0050] When operating on the command line or calling a function, you need to specify the object to be operated. In order to simplify the input, a generalized expression is designed here to represent a group or an application instance, which we call a route. The complete route expression format is "group name / application name#serial number". When the "group name" is given, all applications and instances under this group will be operated. When the "group name / application name" is given, all instances of the application under the group will be operated. When the "group name / application name#serial number" is given, a specific application instance will be operated. "all" is a reserved word or special case, representing all applications to be operated.

[0051] Unnamed groups do not appear in the overall operation sequence and need to be operated separately using the route " / application name#serial number". They are mainly used in one-time tool usage scenarios (such as temporary use in the command line).

[0052] The method and system are also designed with unified and clear batch processing rules, which include:

[0053] For common deployment and O&M operations, such as connectivity check, installation, uninstallation, upgrade, start, stop, status check, upload and download, and monitoring, the framework implements them in-built or requires application logic developers to implement them. These are called built-in instructions.

[0054] When executing a command, the command batch processing module will check whether the specific logic package (corresponding to "package directory name. application name") is implemented. If it is not implemented, an error will be prompted and the execution will be interrupted. If the "skip" configuration item (parameter) is specified during execution, it will not be interrupted and other applications will continue to operate.

[0055] For some special scenarios, special instructions defined by application logic developers, because there is no direct general command line and function entry, this method and system will pass the instructions to the application in the deployment plan, automatically perform dynamic instruction binding, and check whether the instruction is implemented in the logic of the software package. If the instruction is implemented in the application logic, it will be executed, otherwise it will be prompted that the undefined instruction has been skipped.

[0056] The execution of instructions needs to return results or throw exceptions. The results include exit codes, standard output content, standard error content, instruction statements and other information. The exception information is the error information or stack information thrown when the logic cannot continue processing.

[0057] For complex deployments and deployment solutions with more operation logic, you can also use playbook scripts to organize and orchestrate the above instructions and add some special processes and check items.

[0058] Steps S3 and S4, deployment plans or scripts, can be managed in combination with the project source code repository or the use case set repository. It is more recommended to use the use case set repository for management. The use case set repository usually has different operating environments for different test scenarios. In this case, deployment plans or scripts can be set for use cases in different operating environments. In this way, testers can maintain deployment plans for different scenarios as needed, without having to manually modify or use large and comprehensive deployment plans when switching use case sets. Storing deployment plans and use case sets together is conducive to project management and automated execution.

[0059] Step S5: To assist the test, based on step S1 and step S2, a software package provided by a logic developer may be applied to design a test case. The method includes:

[0060] The tester imports the software package in the use case script code. The method and system will automatically search and load the software package in the specified directory according to the application configuration.

[0061] In the use case script, the tester starts the application, connects and logs in to related services, reads deployment and operation and maintenance related data, and sets inspection items according to the needs of the test scenario to determine whether the operation is normal and whether the business processing meets expectations. In the test case, the instruction routing shown in step S2 is combined to operate the specific application, and the interface is directly called to complete the relevant operations. All deployment information is automatically parsed and loaded by the automated testing framework. In the use case, the function interface and group routing expression provided in the software package are combined to control the application according to the needs of the test scenario, and assert the intermediate results.

[0062] Step S6: For some projects or development and testing personnel, the deployment task can be temporarily performed manually, or the deployment task can be automatically triggered by the automated testing framework. The method includes:

[0063] This method and system will load and parse the deployment configuration when the application starts, and perform necessary checks based on the definition of the software package.

[0064] When executed manually, the deployment solution configuration file or script is searched in the current directory by default, and can also be specified through parameters.

[0065] Automated testing triggers execution, which is similar to manual execution. The automated testing framework will search for deployment plans or scripts in the directory where the use case set is located, and then automatically load and parse them.

[0066] When switching the use case set, the automated testing framework determines whether to redeploy and run the system under test.

[0067] Embodiment 2:

[0068] An auxiliary testing method for a variable deployment strategy, comprising:

[0069] Define the application to be deployed in the form of a software package according to the specification or template, and expose parameters including configuration items and software interfaces.

[0070] Set the application to the corresponding relationship and the simplest configuration based on the host address and software package name. Provide default values ​​and default implementations for the parameters and logic required by other applications. Change these default values ​​through command parameters or configuration.

[0071] List and organize the application modules that need to be operated in the deployment plan, define groups, perform parallel operations within groups, and perform serial operations between groups. Define the deployment plan in a scripting manner, declare the correspondence between the host address and the application module in the deployment plan, provide other configuration items in the deployment plan, and overwrite the default parameter values ​​of the application modules.

[0072] Store and manage the configuration or script of the deployment plan in combination with the test case sets of each project;

[0073] Use different deployment plans or scripts for development, debugging or test case design, and tailor deployment plans;

[0074] Use the interfaces provided by the software package to start, stop, and force exit deployment operations in the test cases, call operation and maintenance related interfaces to view CPU and memory usage parameters, and check whether the operation log or business log meets expectations;

[0075] The manually executed deployment plan can be triggered through the CI / CD system to perform automated testing and automatically schedule execution.

[0076] An auxiliary test system for variable deployment strategy, comprising:

[0077] The remote command execution module uses the SSH protocol to connect to the host server. It does not need to pre-install the agent program on the server where the tested system is located. It supports the execution of remote commands and the operation of remote files, and supports the operation of modifying preset ini, xml, and json configuration files.

[0078] The configuration management module standardizes the definition of deployment plans and scripts, and has the functions of parsing deployment information, dependencies, and parsing execution logic according to the script;

[0079] The application definition module standardizes and defines common application operation instructions, including deployment operations such as downloading, installing, starting, stopping, forcibly stopping, status checking, and reloading configurations, as well as operation and maintenance operations such as CPU usage, memory usage, disk usage, and performance monitoring, and operations such as reading remote application logs, remote data files, and remote configuration files;

[0080] Instruction batch processing module, which supports the deployment of applications in the solution through instruction operations;

[0081] The script execution module supports the interpretation and execution of deployment-related logic in the form of scripts. The scripts include custom logic and required check items.

[0082] The automated testing module is used to execute test cases in batches. During the testing process, the test environment can be deployed according to the deployment plan or script provided by each use case set.

[0083] The application definition module stores the logic definition of the application in the form of a software package, which is provided to test case designers for use. Related interfaces are called in the use case for operation and verification. It supports the direct use of general templates to define new applications, including the need to add logic code separately. Applications with special parameters or dependencies can be controlled by adding logic code.

[0084] The command batch processing module passes commands to all or some modules defined in the deployment plan.

[0085] If the instruction batch processing module contains non-predefined instructions or instruction parameters, it will be transparently passed to all or part of the modules in the deployment plan and can be executed as long as the application supports it.

[0086] The script execution module uses scripts when the deployment scheme needs to be switched frequently in automated testing scenarios.

[0087] The automated testing module has the functions of detecting test cases, detecting source code, and batch execution; the test case warehouse usually contains multiple test case sets. Different test case sets use different deployment plans. Deployment is performed before executing each test case set. During the test, test results are collected according to the deployment plan. After the test of the test case set is completed, a test report and a recovery environment are generated.

[0088] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An auxiliary testing method for a variable deployment strategy, characterized in that: include: Define the application to be deployed in the form of a software package according to the specification or template, and expose parameters including configuration items and software interfaces. Set the application to the corresponding relationship and the simplest configuration based on the host address and software package name. Provide default values ​​and default implementations for the parameters and logic required by other applications. Change these default values ​​through command parameters or configuration. List and organize the application modules that need to be operated in the deployment plan, define groups, perform parallel operations within groups, and perform serial operations between groups. Define the deployment plan in a scripting manner, declare the correspondence between the host address and the application module in the deployment plan, provide other configuration items in the deployment plan, and overwrite the default parameter values ​​of the application modules. Store and manage the configuration or script of the deployment plan in combination with the test case sets of each project; Use different deployment plans or scripts for development, debugging or test case design, and tailor deployment plans; Use the interfaces provided by the software package to start, stop, and force exit deployment operations in the test cases, call operation and maintenance related interfaces to view CPU and memory usage parameters, and check whether the operation log or business log meets expectations; The manually executed deployment plan can be triggered through the CI / CD system to perform automated testing and automatically schedule execution.

2. An auxiliary test system with a variable deployment strategy, characterized in that: include: The remote command execution module uses the SSH protocol to connect to the host server. It does not need to pre-install the agent program on the server where the tested system is located. It supports the execution of remote commands and the operation of remote files, and supports the operation of modifying preset ini, xml, and json configuration files. The configuration management module standardizes the definition of deployment plans and scripts, and has the functions of parsing deployment information, dependencies, and parsing execution logic according to the script; The application definition module standardizes and defines common application operation instructions, including deployment operations such as downloading, installing, starting, stopping, forcibly stopping, status checking, and reloading configurations, as well as operation and maintenance operations such as CPU usage, memory usage, disk usage, and performance monitoring, and operations such as reading remote application logs, remote data files, and remote configuration files; Instruction batch processing module, which supports the deployment of applications in the solution through instruction operations; The script execution module supports the interpretation and execution of deployment-related logic in the form of scripts. The scripts include custom logic and required check items. The automated testing module is used to execute test cases in batches. During the testing process, the test environment can be deployed according to the deployment plan or script provided by each use case set.

3. The auxiliary testing method and system of the variable deployment strategy according to claim 2, characterized in that: The application definition module stores the logic definition of the application in the form of a software package, which is provided to the test case designer for use. The relevant interfaces are called in the test case for operation and verification. Supports directly using common templates to define new applications, including no need to add logic code separately, and can control applications with special parameters or dependencies by adding logic code.

4. The auxiliary testing method and system of the variable deployment strategy according to claim 2, characterized in that: The command batch processing module passes commands to all or some modules defined in the deployment plan.

5. The auxiliary testing method and system of the variable deployment strategy according to claim 2 or 4, characterized in that: If the instruction batch processing module contains non-predefined instructions or instruction parameters, it will be transparently passed to all or part of the modules in the deployment plan and can be executed as long as the application supports it.

6. The auxiliary testing method and system of the variable deployment strategy according to claim 2, characterized in that: The script execution module uses scripts when the deployment scheme needs to be switched frequently in automated testing scenarios.

7. The auxiliary testing method and system of the variable deployment strategy according to claim 2, characterized in that: The automated testing module has the functions of detecting test cases, detecting source code, and batch execution; the test case warehouse usually contains multiple test case sets. Different test case sets use different deployment plans. Deployment is performed before executing each test case set. During the test, test results are collected according to the deployment plan. After the test of the test case set is completed, a test report and a recovery environment are generated.

Images