A method for implementing regression testing of background programs in railway CTC system
By using automated regression testing methods, logging modules, and automated testing tools, the problems of time-consuming, labor-intensive, and incomplete coverage testing of the CTC system's backend program were solved, enabling fast and comprehensive testing and ensuring the reliability of software releases.
Patent Information
- Application Number
- CN202111253650.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-27
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-10-27
AI Technical Summary
Existing CTC system backend program testing methods are time-consuming and labor-intensive, and there is a risk of equipment failure due to incomplete test coverage. Especially in complex CTC systems, manual testing is prone to errors, leading to software release problems.
An automated regression testing approach is adopted, which uses a logging module to record test case data and create automated test scripts. Full coverage testing is then performed using automated testing tools, reducing manpower and material resources and improving testing efficiency.
It enables rapid and comprehensive testing of the CTC system's backend programs, avoiding errors caused by equipment failures and manual testing, and ensuring timely software release.
Smart Images

Figure CN114020608B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of railway system testing, and in particular to a method for implementing a railway CTC system background program regression test. Background Art
[0002] The CTC system (railway dispatching centralized system) is widely deployed in the dispatching centers and stations of various railway bureaus for train dispatching and command work. The CTC system software needs to be upgraded and changed according to new requirements, new problems and technical standards of the project site. The CTC system consists of multiple subsystems, which can be divided into front-end operation terminals and back-end programs according to whether manual monitoring is required. Software testing is an important part of the development and upgrade process of the (CTC system). Every time the software of each system is changed and upgraded, in addition to ensuring the completion of new requirements or repairing software problems, it must not affect other functions of the existing program and must not generate new software problems. Therefore, for the test of the CTC system upgrade, the tester cannot only test the newly added functions or the repaired problems, but also test other existing functions of the software to prevent new problems from appearing in the functions that had no problems before due to the mistakes of the R&D personnel during this change and upgrade.
[0003] The background program of the CTC system does not have a complete software human-machine interface. It is connected to other subsystems through the network and communicates according to the communication protocol. Its input is the data packets sent by other specific subsystems through the network channel, and its output is the data packets sent to other specific subsystems based on the input of other subsystems and its own internal logic. Therefore, the test of the background program requires the construction of a complete set of CTC system test environments that include other subsystems. By operating other operation terminal subsystems and verifying the output results on the display interface of other operation terminal subsystems, it is tested whether the internal logic of the background program meets expectations.
[0004] Currently, the following two testing schemes are mainly used:
[0005] Solution 1: The main process is:
[0006] 1) All functions of any CTC system background program must be written by testers and tested before they can be released. For all functions of each released CTC system background program, there are corresponding existing test cases.
[0007] 2) For each software test when the background program is changed or upgraded, it is necessary to build a CTC system environment related to the background program containing multiple subsystems, and write new test cases for the functions related to this software change.
[0008] 3) Through human-machine interface operation on other terminal programs and verification of output results, the functions involved in this program change are fully tested corresponding to the new test cases.
[0009] 4) The R&D personnel who submitted the test application for this software change and upgrade confirm that in this software change and upgrade, in addition to the new functions and software issues fixed by this change, the scope of change impact that needs to be tested on existing functions. Based on the scope of change impact provided by the R&D personnel, the tester selects the test cases that need regression testing from the existing test cases and builds a test environment for regression testing.
[0010] The drawback of Solution 1 is that for each test of the CTC system background program, the R&D personnel must give the scope of impact of the change and select some existing test cases for regression testing based on the scope of impact of the change. On the one hand, regressing the existing test cases is time-consuming and labor-intensive; on the other hand, some CTC system background programs have complex software structures, variable inheritance, function calls, etc. The scope of impact of the change given by the R&D personnel sometimes cannot fully cover the scope to be tested. Sometimes, new problems will occur in program functions outside the scope of impact of the change given by the R&D personnel, resulting in equipment failures on the project site. The equipment on-site failure and subsequent accountability handling process are relatively complicated, increasing maintenance costs and the workload of on-site personnel.
[0011] Option 2 is adopted when the CTC system background program makes major or more important changes. In Option 2, first refer to the first three steps of Option 1, and then perform the following steps: the tester further builds a complete CTC system test environment, performs a complete regression test on all existing test cases of the background program, verifies all scenarios in all existing test cases through human-computer interface operations on other terminal programs and verifies output results, so as to traverse all functions of the program to prevent new problems.
[0012] The drawback of Solution 2 is that the CTC system backend program has undergone many years of development, and its logic is relatively complex and has many functions. A backend program often involves more than 1,000 test case scenarios. Building a complete CTC system environment for complete regression testing requires a large amount of computer equipment and a lot of testing time. Figure 1 Taking a specific application scenario as an example, a complete CTC system test environment is demonstrated. Figure 1 Each CTC subsystem shown needs to occupy a computer or virtual machine. A large number of testers work overtime, which is very time-consuming and may not be able to complete the test tasks on time, and sometimes delay the on-site software upgrade time. Summary of the invention
[0013] The purpose of the present invention is to provide a railway CTC system background program regression test implementation method, which can completely traverse all functions of the background program, does not occupy a large amount of manpower and material resources, and can quickly complete the test task.
[0014] The objective of the present invention is achieved through the following technical solutions:
[0015] A railway CTC system background program regression test implementation method, comprising:
[0016] Use each test case to test the background program of the CTC system to be tested, and use the log recording module of the background program of the CTC system to be tested to record the program configuration file, the original sending and receiving data log file and the related parsing log file corresponding to the test case, and store the recorded data in the folder with the specified number as the automated test log script file;
[0017] Initialize the relevant settings of the automated testing tool, including: setting the test case number range and consistency comparison rules; extracting the corresponding test case according to the set test case number range, loading the relevant automated test log script file for the current test case, changing the program startup time and configuring it according to the program configuration file, inputting the current test case into the background program of the CTC system to be tested for testing, and comparing the data recorded by the log recording module of the background program of the CTC system to be tested with the relevant automated test log script file according to the consistency comparison rules, and judging the test result of the current test case according to the comparison result;
[0018] When the current CTC system background program is changed, a new automated test log script file is re-created for the new function, and an automated testing tool is used to perform a complete automated regression test on all existing functions of the current CTC system background program.
[0019] It can be seen from the technical solution provided by the present invention that, by adopting the automated regression testing method, on the one hand, the test cases of the background program to be tested can be distributed on multiple computers and automatically tested at the same time. The automatic test can be started before leaving work on the first day, and the result can be checked on the next morning, which greatly reduces the manpower and material resources for testing, saves testing time, greatly improves the testing efficiency, and ensures that the software is released on schedule; on the other hand, it avoids the problem of problematic programs being released to the site through testing due to possible errors and omissions in manual testing; in addition, the test cases can be completely traversed each time the background program is changed, and a complete regression test can be performed on the CTC background program to avoid the release of problematic software due to incomplete test coverage. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0021] Figure 1 A schematic diagram of a complete CTC system test environment provided as background technology of the present invention;
[0022] Figure 2 A flowchart of a method for implementing a regression test of a background program of a railway CTC system provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.
[0024] First, the terms that may be used in this article are explained as follows:
[0025] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.
[0026] The following is a detailed description of a railway CTC system background program regression test implementation method provided by the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field. If no specific conditions are specified in the embodiments of the present invention, the conventional conditions in the field or the conditions recommended by the manufacturer are followed. The instruments used in the embodiments of the present invention without indicating the manufacturer are all conventional products that can be purchased commercially.
[0027] A railway CTC system background program regression test implementation method provided by an embodiment of the present invention mainly includes the following steps:
[0028] Use each test case to manually test the background program of the CTC system to be tested, and use the log recording module of the background program of the CTC system to be tested to record the original sending and receiving data log file and the related parsing log file, and store the recorded data in the folder with the specified number as the automated test log script file;
[0029] Initialize the relevant settings of the automated test, including: setting the test case number range and consistency comparison rules; extracting the corresponding test case according to the set test case number range, loading the relevant automated test log script file for the current test case, inputting the current test case into the background program of the CTC system to be tested for testing, and comparing the data recorded by the log recording module of the background program of the CTC system to be tested with the relevant automated test log script file according to the consistency comparison rules, and judging the test result of the current test case according to the comparison result;
[0030] When the current CTC system background program is changed, a new test script file is re-created, and a complete automated regression test is performed on all existing functions of the current CTC system background program.
[0031] The automatic regression testing solution provided by the embodiment of the present invention can completely traverse all functions of the background program without taking up a lot of manpower and material resources, and can quickly complete the testing task, which can solve the following three technical problems:
[0032] 1) If the test cases cannot be completely traversed when the background program is changed, it may result in the software test not covering all the functions involved in the change. The program change and upgrade will generate new software problems, leading to on-site equipment failures. Handling the failures and subsequent accountability will be time-consuming and laborious.
[0033] 2) If the existing test cases are completely traversed when the background program is changed, it will take a lot of manpower, material resources and time to build a complete CTC test environment and completely traverse the test, and the test task may not be completed on time, which may sometimes delay the on-site software upgrade time.
[0034] 3) In the testing of background programs, the test results are mainly observed manually by testers on other terminal subsystems, which may lead to errors and omissions, resulting in problematic programs passing the test and being sent to the project site, which may cause the risk of on-site equipment failure.
[0035] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, a railway CTC system background program regression test implementation method provided by an embodiment of the present invention is described in detail below with a specific embodiment.
[0036] 1. Added the functions of receiving log and sending log.
[0037] For the background programs of the CTC system (e.g., autonomous machines, train tracking servers, communication servers, etc.), output is generated based on the communication input of other subsystems and its own logic. Modify the logging module of all CTC system background programs to enable them to record the original sending and receiving data log files and related parsing log files.
[0038] In the embodiment of the present invention, the original receiving and sending data log file and the related parsing log file include: an original receiving data log file (sdat-rcv log), an original sending data log file (sdat-snd log), a receiving data parsing log file (base-rcv log), and a sending data parsing log file (base-snd log); specifically:
[0039] 1) The logging module of the CTC system background program records the program startup time and each packet of network raw data received from each external subsystem one by one, and corresponds it to the system time (receiving time) when it is received. All received raw data and the time when the data is received (accurate to milliseconds) are recorded in the raw receiving data log file (sdat-rcv log).
[0040] 2) While recording the original received data log file, the data content of each packet of received original data is parsed according to the CTC communication protocol and recorded in the received data parsing log file (base-rcv log). The following information is obtained by parsing: receiving time, information source device, information type and information content.
[0041] 3) The logging module of the CTC system background program records each packet of network raw data sent to each external subsystem, and corresponds it to the system time when it is sent. It records all the raw data sent to the outside and the time of sending the data (accurate to milliseconds) in the raw sending data log file (sdat-snd log).
[0042] 4) While recording the original sending data log file, the data content of each packet of original data sent is parsed according to the CTC communication protocol and recorded in the sending data parsing log file (base-snd log). The following information is obtained by parsing: sending time, information destination device, information type and information content.
[0043] 2. Create automated test log scripts for existing test cases.
[0044] Based on the work in the first part above, the log recording module of the CTC system background program has the function of recording the original sending and receiving data log file and the related parsing log file. Therefore, during the free time of the project, the tester can record the automated test log script file for each existing test case of the CTC system background program. The automated test log script file corresponds to the test case one by one, and each test case has its corresponding automated test log script file. The steps include:
[0045] 1) Build the test environment, correctly configure the program according to the test case to be tested, and start the background program of the CTC system to be tested.
[0046] 2) For each test case, use the existing method to test.
[0047] Exemplarily, the test case is tested by operating on other terminal programs in a traditional manner and verifying output results on other terminal programs, and the program is closed after the test is completed.
[0048] 3) After the test is completed, obtain the original receiving and sending data log file corresponding to each test case (the file records the program startup time) and the related parsing log file, as well as the program configuration file corresponding to the test case, and put these files into a folder named with the corresponding test case number as an automated test log script file.
[0049] In the embodiment of the present invention, the automated test log script file of each test case includes a separate program configuration file corresponding to the test case, and the program configuration information mainly includes the configuration of program functionality and the configuration of external communications.
[0050] Combined with the description in the first part of the previous article, it can be seen that the original receiving and sending data log file and the related parsing log file obtained at this time contain four parts of log files, namely the original receiving data log file (sdat-rcv log), the original sending data log file (sdat-snd log), the receiving data parsing log file (base-rcv log), and the sending data parsing log file (base-snd log).
[0051] The tester performs the above script recording operation for each test case during the project's free time, and saves the automated test log script for each test case in a folder with a specific name (named after the test case number) as the automated test script of the CTC system background program for R&D and testers to debug automated test tools and conduct tests.
[0052] 3. Automated testing tool software functions and processes.
[0053] In the embodiment of the present invention, the R&D personnel, in cooperation with the test personnel, develop the CTC system background program automated testing tool software to implement automated testing. The automated testing process is roughly as follows: for the current test case, load the relevant automated test log script file, input the current test case into the CTC system background program to be tested for testing, and compare the data recorded by the CTC system background program log recording module to be tested with the relevant automated test log script file according to the consistency comparison rules, and judge the test result of the current test case according to the comparison result. Figure 2 As shown in the figure, the main functions and workflow of the automated testing tool software are shown, which mainly include:
[0054] 1) Initialization, including: being able to set consistency comparison rules, the rules contain thresholds corresponding to specific types of information, specifically time difference thresholds and content difference thresholds, and also needing to set the test case number range.
[0055] 2) Read all logs in the automated test log script file in the folder named with the test case number, load the relevant automated test log script file, and configure the system time and IP address according to the configuration information in the automated test log script file (from the program configuration file).
[0056] Specifically: change the system time to the program startup time recorded in the automated test log script file to simulate the test time at that time for automated testing, and modify the IP addresses of all other subsystems connected to the background program in the background program configuration file to the local address. Both of the above configurations need to be performed under the system administrator's authority.
[0057] 3) Start (here the automated test tool automatically starts the CTC background program) the CTC system background program to be tested, and send each packet of raw received data recorded in the raw received data log file in the automated test log script file to the CTC system background program to be tested according to the receiving time recorded in the corresponding raw received data analysis log file as the input of the automated test. This stage also needs to be performed under the system administrator's authority.
[0058] 4) The CTC system background program to be tested generates corresponding output for the original received data, including the sent data and the corresponding parsing log file (check-snd log), extracts the parsing log file in the output and compares it with the corresponding sent data parsing log file (base-snd log) in the automated test log script file for consistency.
[0059] 5) If all the information in the two data sending analysis log files is exactly the same, or the difference in the sending time recorded in the two analysis log files does not exceed the time difference threshold in the consistency comparison rule, and the difference in the content of the two analysis log files does not exceed the content difference threshold in the consistency comparison rule, then the current test case is deemed to have passed the test; otherwise, the current test case is deemed to have failed the automatic test and requires manual confirmation by the R&D personnel.
[0060] The above is an introduction to the test functions and processes of the automated testing tool using the current test case as an example. After testing all test cases based on the above principles, proceed to step 6).
[0061] 6) After all the test cases to be tested are tested, a test report is generated, including: the total number of test cases in the test, the total number of test cases that passed the test, and the total number of test cases that failed the test; for the test cases that failed the test, the base-snd log and check-snd log related data packet entries and specific contents that did not pass the consistency comparison rules are listed for R&D and testing personnel to review and analyze.
[0062] 4. Automated regression testing.
[0063] The previously recorded automated test log script file is called the automated test log script file of the existing function. When the background program needs to be upgraded, after the tester has completed the testing of the new functions related to this change and recorded the relevant test script (called the new automated test log script file, which is used for the next and subsequent automated tests), the tester can perform a complete automated regression test on all the existing functions of the background program based on the logging function of the background program itself, the previously recorded automated test script file of the existing function and the developed automated test tool. The steps are as follows:
[0064] 1) Store the CTC system background program and automated testing tools to be tested in a test computer, put the recorded automated test case script files (various logs and program configurations) of existing functions into a specific folder, configure the test case number range and comparison rules, and clarify the time difference threshold and content difference threshold corresponding to the specific information type.
[0065] 2) Set the computer to keep the screen on, cut off the local Internet connection to prevent the system from automatically calibrating the time, run the automated testing tool as an administrator, and start the automated testing, which is the automated testing tool workflow in Part 3.
[0066] 3) Observe that the system time is changed to the program startup time in the test case. The test tool successfully establishes a network connection with the background program to be tested, and prompts you to enter the test process. Since the process takes a long time, the tester can leave and do other work.
[0067] 4) After the test is completed, the results are output and the final brief test report is reviewed. If all the results are passed, the test is completed. If some of the results are not passed, contact the R&D personnel for analysis to find the cause of the problem.
[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0069] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A railway CTC system background program regression test implementation method, characterized in that: include: Use each test case to test the background program of the CTC system to be tested, and use the log recording module of the background program of the CTC system to be tested to record the background program configuration file, original sending and receiving data log file and related parsing log file corresponding to the test case, and store the recorded data in a folder with a specified number as an automated test log script file; Initialize the relevant settings of the automated testing tool, including: setting the test case number range and consistency comparison rules; extracting the corresponding test case according to the set test case number range, loading the relevant automated test log script file for the current test case, changing the background program startup time and configuring it according to the background program configuration file, inputting the current test case into the background program of the CTC system to be tested for testing, and comparing the data recorded by the log recording module of the background program of the CTC system to be tested with the relevant automated test log script file according to the consistency comparison rules, and judging the test result of the current test case according to the comparison result; When the current CTC system background program is changed, a new automated test log script file is re-created for the new function, and an automated testing tool is used to perform a complete automated regression test on all existing functions of the current CTC system background program; For the current test case, the relevant automated test log script file is loaded, the current test case is input into the CTC system background program to be tested for testing, and the data recorded by the CTC system background program log recording module to be tested is compared with the relevant automated test log script file according to the consistency comparison rule, and the test result of the current test case is judged according to the comparison result, including: Load the relevant automated test log script file, and configure the system time and IP address according to the program configuration file in the automated test log script file; Start the background program of the CTC system to be tested, and send each packet of original received data recorded in the original received data log file in the automated test log script file to the background program of the CTC system to be tested according to the receiving time recorded in the corresponding original received data parsing log file; The CTC system background program to be tested generates corresponding output for the original received data, including the sent data and the corresponding parsing log file, extracts the parsing log file in the output and compares it with the parsing log file corresponding to the sent data in the automated test log script file for consistency; If all the information in the two parsing log files is exactly the same, or the difference in the sending time recorded in the two parsing log files does not exceed the time difference threshold in the consistency comparison rule, and the difference in the content of the two parsing log files does not exceed the content difference threshold in the consistency comparison rule, then the current test case is deemed to have passed the test.
2. A railway CTC system background program regression test implementation method according to claim 1, characterized in that: The original receiving and sending data log files and the related parsing log files include: the original receiving data log file, the original sending data log file, the receiving data parsing log file and the sending data parsing log file; wherein: The log recording module of the CTC system background program records each packet of network raw data received from each external subsystem one by one, and records it in the raw received data log file in correspondence with the system time when it is received; at the same time, the data content of each packet of received raw data is parsed according to the CTC communication protocol and recorded in the received data parsing log file, and the following information is obtained by parsing: receiving time, information source device, information type and information content; The logging module of the CTC system background program records each packet of network raw data sent to each external subsystem, and corresponds it to the system time when it is sent, and records it in the raw sending data log file; at the same time, the data content of each packet of raw data sent is parsed according to the CTC communication protocol, and recorded in the sending data parsing log file. The following information is obtained through analysis: sending time, information destination device, information type and information content.
3. A railway CTC system background program regression test implementation method according to claim 1 or 2, characterized in that: The automated test log script file corresponds to the test case one by one, and each test case has its corresponding automated test log script file. The automated test log script file preparation steps include: Set up the test environment, correctly configure the background program according to the test cases to be tested, and start the background program of the CTC system to be tested; For each test case, use the existing method to test; After the test is completed, the original sending and receiving data log file and the related parsing log file corresponding to each test case, as well as the background program configuration file corresponding to the test case are obtained, and the obtained files are placed in a folder named after the corresponding test case number as an automated test log script file; among them, the original sending and receiving data log file records the background program startup time.
4. The method for implementing a regression test of a railway CTC system background program according to claim 1, characterized in that: After all test cases are tested, a test report is generated, including: the total number of test cases in the test, the total number of test cases that passed the test, and the total number of test cases that failed the test; for test cases that failed the test, the relevant data packet entries and specific contents that failed the consistency comparison rules are listed.
