Multi-tool integrated automatic test platform implementation method and system

Abstract

The invention relates to a multi-tool integrated automatic test platform implementation method and system, and the method comprises the steps: receiving a test case ID by a dispatching center, loading a test step sequence, an expected result and configuration parameters from a test case library and a station yard configuration library of a data and configuration management center, deploying a test environment through the tool integration adaptation layer and confirming that the test environment is ready; the dispatching center executes a test step sequence in sequence, the modules work in parallel to continuously collect data, and each module comprises a test management dispatching center, a tool integration adaptation layer, a data and configuration management center and an analysis and judgment engine; the analysis and judgment engine collects and aligns data collected by all the modules and carries out information extraction, and a multi-dimensional index calculator is used for calculating functional indexes; and comparing with a predefined passing standard to obtain a conclusion, so that automatic testing of multi-tool integration is realized. Compared with the prior art, the method has the advantages of being high in testing efficiency, accurate in judgment result, high in testing platform adaptability and the like.

Description

Technical Field

[0001] This invention relates to the field of rail transit signaling, and in particular to a method and system for implementing an automated testing platform integrating multiple tools. Background Technology

[0002] The rail transit computer interlocking system is a core control system ensuring railway traffic safety. Its safety and reliability requirements are extremely high, necessitating rigorous and comprehensive testing and verification. Currently, the industry's testing of interlocking systems faces the following technical challenges and problems: Currently, the mainstream interlocking system testing platforms mainly adopt the following technical solutions: using semi-automated testing frameworks; decentralized and independent testing tools, including network packet capture tools, serial port debugging assistants, log analysis tools, and simulator control software; manually configured testing environments, including board configuration, network parameters, simulator settings, etc.; and relying on manually written and executed test scripts, lacking unified scheduling and management.

[0003] In existing technologies, there is a lack of effective integration between various testing tools, requiring testers to frequently switch between different software interfaces. Test case execution relies on manual, step-by-step operations, making batch automation impossible; network packet capture and test execution are out of sync, resulting in poor time correlation. Test result judgment depends on manual analysis, which is heavily influenced by subjective factors; test environment preparation is complex, requiring parameter reconfiguration for each test. Testing efficiency is low: according to actual test statistics, a typical functional test includes: environment data preparation and configuration (3-5 minutes), test execution and monitoring (3-5 minutes), data collection and storage (2-3 minutes), and result analysis and report writing (15-20 minutes). When a test fails, manual analysis of information from multiple sources is required, such as network communication packets (pcap files), system operation logs (log files), simulator status records, and device serial port output.

[0004] As the scale of rail transit construction expands, the complexity of interlocking systems continues to increase, leading to an exponential increase in testing workload. Traditional testing methods can no longer meet the efficiency requirements of rapid iterative development cycles. Therefore, there is an urgent need for a highly automated and intelligent integrated testing platform to achieve automated execution and intelligent analysis of the entire testing process.

[0005] Chinese invention patent CN103092756B discloses an interactive automated testing system and method, comprising: a central test control terminal, which is connected to an automated test case editor and several agent execution terminals. The automated test case editor is connected to an interface recording module through a recorded control library file. A natural language manager is applied to both the automated test case editor and the central test control terminal. This invention provides an interactive automated testing system that significantly improves the speed of building automated software tests while greatly reducing the difficulty of building them. It also lowers the skill requirements for automated testers and solves the problem of interactive testing of network software systems that traditional testing tools cannot address. However, it still suffers from problems such as large inconsistencies in manual operation, long fault location time, subjectivity and oversight in manual interpretation, and high workload and low configuration efficiency when manually rewriting test scripts when site data changes.

[0006] In summary, there is currently a lack of a method and system for implementing an automated testing platform that integrates multiple tools to solve or partially solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a method and system for implementing an automated testing platform that integrates multiple tools, so as to solve or partially solve the problems of large differences in manual operation, long fault location time, subjectivity and omissions in manual interpretation, and large workload and low configuration efficiency when the station data changes.

[0008] The objective of this invention can be achieved through the following technical solutions: According to one aspect of the present invention, a method for implementing an automated testing platform integrating multiple tools is provided, specifically including: S1. The scheduling center receives the test case ID, loads the test step sequence, expected results and configuration parameters from the test case library and station configuration library of the data and configuration management center, and deploys the test environment through the tool integration adaptation layer and confirms that the test environment is ready. S2. The scheduling center executes the test step sequence in sequence, and the steps in the sequence are linked. Multiple modules of the platform work in parallel to continuously collect data. The modules include: a test management scheduling center, a tool integration and adaptation layer, a data and configuration management center, and an analysis and judgment engine. S3. The analysis and judgment engine collects and aligns the data collected by each module to form a unified timeline data view. The protocol and log analyzer extracts information from the aligned data and uses the multidimensional indicator calculator to calculate functional indicators based on the extracted information. S4. The rule-based automatic judgment unit compares the calculated functional indicators with predefined pass standards and obtains the final conclusion through weighted or logical operations. The report generator creates reports and evidence chains to achieve automated testing through multi-tool integration.

[0009] As a preferred technical solution, the step of confirming the test environment is ready specifically includes: the scheduling center polling and checking the status of the controlled tools and devices to confirm network connectivity, simulator login status, and normal serial communication.

[0010] As a preferred technical solution, the parallel operation of the various modules of the platform to continuously collect data specifically includes: the network capture device capturing network interaction messages, the serial port module recording the serial port output of the system under test, and the simulator control module recording the state changes of the simulator.

[0011] As a preferred technical solution, the report generator creates a report that includes a test summary, execution log, detailed data analysis results, judgment criteria, and conclusions.

[0012] As a preferred technical solution, the report generator creates a chain of evidence by associating and packaging the original test data, intermediate analysis results, and the report.

[0013] According to another aspect of the present invention, an automated testing platform system integrating multiple tools is provided, specifically including a test management and scheduling center, a tool integration and adaptation layer, a data and configuration management center, and an analysis and judgment engine.

[0014] As a preferred technical solution, the test management and scheduling center includes: The sequence editor provides a graphical interface for arranging, editing, and maintaining test sequences; The execution engine is used to parse and execute the test sequence and call the tool adapter according to predetermined logic; The resource manager is used to uniformly manage the status of software and hardware resources in the platform; The report generator is used to collect process and result data generated at each stage and automatically generate structured test reports according to predefined templates.

[0015] As a preferred technical solution, the tool integration adapter layer is used to connect the test management scheduling center and the test tools, including: The network capture adapter module is used to receive instructions from the test management and scheduling center, automatically capture, filter and store network packets, and provide real-time feedback on the packet capture status and file path. A serial communication parsing and adaptation module is used to communicate with the serial port of the interlocking device or simulator under test. The simulator cluster control adapter module is used to uniformly manage simulators, including operator display units, trackside electronic units, and area controllers. The physical interface simulation adapter module simulates physical operations in real-world scenarios by controlling external relay matrices or programmable switchgear.

[0016] As a preferred technical solution, the data and configuration management center uses a database or file system for centralized management, including: A test case library is used to store the metadata and step definitions of test cases; The station configuration library is used to store hardware configuration data, network parameters, and PLC address mapping tables for different interlocking stations. The test results and evidence library is used to store the raw data and derived analysis results generated during each test execution.

[0017] As a preferred technical solution, the analysis and judgment engine is used to analyze and judge the collected multi-source heterogeneous data, including: A multi-source data time synchronizer aligns all data to the same time base. The protocol and log analyzer has built-in parsing rules for rail transit-specific communication protocols and common log pattern recognition rules, which are used to extract key fields from raw messages and key events from system logs. Multidimensional metrics calculator, which calculates performance and functional metrics based on business rules; The rule-based automatic decision-maker draws conclusions and confidence levels based on preset thresholds and logical rules.

[0018] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) This invention manages and schedules each module through a test management and scheduling center, executes the test steps in sequence, completes the entire automated test process, solves the problems of high operator proficiency requirements and human error caused by repetitive manual operations, achieves the technical effects of improving the consistency and reliable reproducibility of test results, reducing labor costs, and improving test efficiency.

[0019] (2) This invention uses an analysis and judgment engine to collect and align the data collected by each module, extract information and perform correlation analysis on the aligned data, and perform rule-based automated analysis and judgment. This solves the problems of subjectivity and omissions in manual judgment and long fault location time, and achieves the technical effect of rapid fault location and improved accuracy of result judgment.

[0020] (3) The present invention centrally manages the configuration through a database or file system. Based on the different configurations of the target interlocking system, it automatically generates and adapts the drive data, network configuration parameters and test scripts, etc., which solves the problem of large workload and low configuration efficiency when manually rewriting test scripts when the station data changes. It achieves the technical effect of improving the universality and adaptability of the test platform, improving configuration efficiency and reducing maintenance workload. Attached Figure Description

[0021] Figure 1 This is a software-level test logic diagram for the present invention; Figure 2 This is a system-level test logic diagram for the present invention; Figure 3 This is a diagram showing the overall architecture of the system of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] Example 1 To address the problems existing in the aforementioned prior art, this embodiment provides a method for implementing an automated testing platform that integrates multiple tools. The core of this method lies in: using test management software as a unified scheduling engine, and developing a dedicated adapter module to deeply integrate traditional discrete testing tools such as packet capture tools, serial port debugging tools, and various simulator control software, and building an analysis engine, thereby achieving end-to-end automation of the entire process from test case parsing, automatic deployment of the test environment, automatic execution of test steps, automatic collection of test data to automatic analysis and report generation of test results.

[0024] The specific steps of this method include: S1. Test preparation and initialization.

[0025] S101. Test Case Parsing and Parameter Loading: The test management scheduling center receives the test case ID specified by the user and loads the corresponding test step sequence, expected results, and station-specific configuration parameters, such as interlocking area number, equipment address, and IP port, from the test case library and station configuration library of the data and configuration management center.

[0026] S102. Dynamic Environment Deployment: The scheduling center automates the deployment of the test environment through tool integration and adaptation layers. For example, the command simulator cluster controls the adaptation module to start the simulator and load the initial scene in a specific order; the command network capture adapter module pre-sets packet filtering rules.

[0027] S103. Environment Readiness Verification: The dispatch center polls and checks the status of all controlled tools and devices to confirm network connectivity, simulator login status, and normal serial communication, ensuring that the test environment is fully ready.

[0028] S2. Automatic execution of test sequences and synchronous data acquisition.

[0029] S201. Step-Driven and Tool-Linked System: The dispatch center executes each step in the test procedure sequence sequentially. The key innovation lies in the precise linkage between steps. For example, when the "send turnout control command" step is executed, two actions are triggered simultaneously: (1) Accurately start message capture through the network capture adapter module (record timestamp T1).

[0030] (2) Send control commands to the system under test through the serial communication parsing adapter module.

[0031] S202. Multi-channel parallel data acquisition: During the test execution, each adapter module works in parallel to continuously acquire data: the network snapshot captures all network interaction messages; the serial port module records all serial port outputs of the system under test; and the simulator control module records the status changes of each simulator. All data is stamped with high-precision timestamps.

[0032] S203. Execution Flow Control: The platform supports complex test logic, including conditional judgments and loop execution, all of which are automatically managed by the scheduling center. For example, if a response times out, a fault handling branch is executed, and repeated stress tests are performed.

[0033] S3. Correlation analysis of multi-source test data.

[0034] S301. Data Collection and Time Alignment: After the test steps are completed, the multi-source data time synchronizer of the analysis and judgment engine will collect the raw data files from the network, serial port, simulator, etc., and perform millisecond-level alignment based on the timestamp to form a unified time axis data view.

[0035] S302. Key Information Extraction: The protocol and log analyzer performs in-depth analysis of the aligned data. For example, it locates the response message to the control command in the pcap file and extracts its arrival timestamp T2 and safety code; it searches for and extracts "turnout positioning" related records and their timestamp T3 from the log file.

[0036] S303. Performance and Functional Indicator Calculation: The multi-dimensional indicator calculator performs calculations based on the extracted information. For example, calculating network communication latency. Internal processing latency of the computing system Verify whether the functional status is consistent with expectations.

[0037] S4. Rule-based automated result determination and report generation.

[0038] S401. Rule-based Comprehensive Judgment: The rule-based automatic judgment unit automatically compares the calculated indicators with predefined standards in the rule base. For example, setting... , And security code, the predefined pass standard is , The security code is 0xAA55. A weighted OR logical operation is used to determine the final "pass / fail" result.

[0039] S402. Automatic Generation of Structured Reports and Evidence Chains: The report generator automatically creates a complete report including a test summary, execution logs, detailed data analysis results, judgment criteria, and conclusions. Simultaneously, all raw data, intermediate analysis results, and the final report of this test are linked and packaged to form an immutable evidence chain archive, facilitating traceability and auditing.

[0040] S403. Automatic Environment Recovery: Regardless of whether the test passes or fails, the platform automatically executes a cleanup process: shutting down packet capture, resetting the simulator state, and closing the serial port, restoring the test environment to its initial state and preparing to execute the next test case.

[0041] Example 2 Based on the foregoing embodiments, this embodiment provides a multi-tool integrated automated testing platform system, which consists of four core components, including: 1. Test Management and Scheduling Center.

[0042] This section is built upon test management software, serving as the brain and central nervous system of the entire platform, responsible for the orchestration, scheduling, and monitoring of all test activities. Its main modules include: Sequence Editor: Provides a graphical interface for orchestrating, editing, and maintaining test sequences. A test sequence consists of multiple test steps, each of which can invoke different tools or perform specific operations.

[0043] Execution engine: Parses and executes the test sequence, calls each tool adapter according to the predetermined logic (sequence, branch, loop), and strictly ensures the synchronization and timing requirements between steps.

[0044] Resource Manager: Unifies the management of the status of all software and hardware resources in the platform, including the availability of tool modules, network connection status, simulator running status, etc., to ensure that the environment is ready before test execution.

[0045] Report Generator: Collects process and result data generated at each stage and automatically generates structured test reports according to predefined templates.

[0046] 2. Tool integration and adaptation layer.

[0047] This layer serves as a bridge connecting the test management and scheduling center with various specific test tools or devices. By encapsulating the complex operations of each tool into standard "Step Types" that can be called by the test management software, it enables plug-and-play functionality and unified scheduling of tools. This layer mainly includes the following adapter modules: Network packet capture adapter module: This module encapsulates the core functions of the packet capture tool. It receives instructions from the scheduling center (such as start / stop, filtering rules, and storage paths), automatically performs network packet capture, filtering, and storage, and provides real-time feedback on the packet capture status and file paths.

[0048] Serial Communication Parsing and Adaptor Module: Responsible for communicating with the serial port of the interlocking device or simulator under test. This module encapsulates functions such as serial port opening / closing, baud rate configuration, data sending / receiving, and protocol parsing (such as parsing response frames of a specific format), enabling automated command issuance and real-time monitoring and parsing of responses.

[0049] Simulator Cluster Control Adapter Module: Unifies the management of various simulators such as OC (Operator Display Unit), LEU (Railwayside Electronic Unit), and ZC (Zone Controller). This module communicates with each simulator instance via SSH, Telnet, or a dedicated API to enable batch startup or shutdown of simulators, loading of test scenario configurations, status query and synchronization, and recovery from abnormal states.

[0050] Physical interface simulation and adaptation module: By controlling external relay matrix or programmable switch equipment, it simulates physical operations such as board insertion and removal, cable connection and disconnection in real-world scenarios, thereby automating related fault injection tests or hardware configuration change tests.

[0051] 3. Data and Configuration Management Center.

[0052] This section provides data support for the platform, using a database or file system for centralized management, including: Test case library: Stores metadata and step definitions for all test cases. Each test case is associated with a specific interlocking site, test scenario, and test objective.

[0053] Station configuration library: Stores hardware configuration data, network parameters, PLC address mapping tables, etc. for different interlocking stations (such as Jiamusi Railway Station and Hongqiao West Metro Station). Supports configuration templates and version management.

[0054] Test Results and Evidence Library: Centrally stores all raw data (pcap files, log files, serial port data) and derived analysis results generated during each test execution, forming a traceable chain of evidence.

[0055] 4. Analysis and Judgment Engine.

[0056] This section is the core embodiment of the platform's intelligence level, responsible for the automatic analysis and comprehensive judgment of the massive amounts of multi-source heterogeneous data collected during the testing process, including: Multi-source data time synchronizer: Due to the differences in timestamps in network packet capture, system logs, serial port data, etc., this module uses a high-precision algorithm to align all data to the same time base, laying the foundation for subsequent correlation analysis.

[0057] Protocol and Log Analyzer: Built-in parsing rules for rail transit-specific communication protocols (such as RSSP-II) and common log pattern recognition rules, which can automatically extract key fields (such as message type, security code, and sequence number) from raw messages and key events (such as "turnout positioning completed") from system logs.

[0058] Multidimensional metrics calculator: Automatically calculates performance and functional metrics based on business rules, such as instruction response latency (from sending to receiving network confirmation), system processing latency (from receiving network confirmation to log recording), CPU or memory resource utilization, etc.

[0059] Automatic Rule-Based Decision Maker: This module has a built-in configurable rule library. It receives analyzed structured data and, based on preset thresholds and logical rules (such as "response latency < 50ms AND correct security code AND consistent functional status"), automatically determines a "pass / fail" conclusion and its confidence level.

[0060] like Figure 1 As shown, the test process begins with a test script and is driven by the test management software engine. The test management software interacts with the underlying test environment through middleware interfaces. Its key logic involves comparing two inputs: first, the expected behavior or state set in the test script; and second, the actual responses and data obtained by directly monitoring the software under test (running on the test platform) and its interactive objects (such as simulators) through serial port tools and packet capture tools. Through automatic comparison, the system generates test logs and obtains test results. This logic focuses on verifying the logical correctness and interface compliance of the software module itself.

[0061] like Figure 2As shown, compared to software-level testing, system-level testing has a more complete integration testing logic. Its process also begins with test scripts and is scheduled by test management software, but the introduced instrumentation technology can actively collect state data at key points within the software, serving as a crucial input for comparison. Simultaneously, simulator scripts are used to drive more complex simulator behaviors that closely resemble real-world operating scenarios. The test management software comprehensively compares various data streams from instrumentation points, various tools (serial ports, packet capture), and the test platform itself to complete a more comprehensive verification of system functions and interactions. Ultimately, this process not only produces test results but also generates detailed test logs, providing a complete chain of evidence for system-level integration verification, fault location, and performance evaluation.

[0062] like Figure 3 As shown, the system of this invention adopts a clear five-layer design: the user interface layer provides an interactive entry point for test case editing, execution monitoring, and result analysis; its core is the test management engine, which integrates sequence orchestration, step scheduling, resource management, and report generation, acting as the command center of the entire platform; the tool integration and adaptation layer encapsulates heterogeneous tools such as packet capture tools, serial port parsing, and simulator control into standardized services, enabling unified invocation; the data management system centrally stores and manages test cases, configuration parameters, and historical results, supporting test reuse and traceability; and the underlying analysis engine is responsible for automatically parsing and judging the collected logs and messages, realizing intelligent output of test results. This architecture achieves an end-to-end automated closed loop from user operation to tool execution, data processing, and analysis.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for implementing an automated testing platform integrating multiple tools, characterized in that, The method specifically includes: S1. The scheduling center receives the test case ID, loads the test step sequence, expected results and configuration parameters from the test case library and station configuration library of the data and configuration management center, and deploys the test environment through the tool integration adaptation layer and confirms that the test environment is ready. S2. The scheduling center executes the test steps sequentially, with the steps in the sequence linked together. Multiple modules of the platform work in parallel to continuously collect data. The modules include: a test management scheduling center, a tool integration and adaptation layer, a data and configuration management center, and an analysis and judgment engine. S3. The analysis and judgment engine collects and aligns the data collected by each module to form a unified timeline data view. The protocol and log analyzer extracts information from the aligned data and uses the multidimensional indicator calculator to calculate functional indicators based on the extracted information. S4. The rule-based automatic judgment unit compares the calculated functional indicators with predefined pass standards and obtains the final conclusion through weighted or logical operations. The report generator creates reports and evidence chains to achieve automated testing through multi-tool integration.

2. The method for implementing an automated testing platform integrating multiple tools according to claim 1, characterized in that, The steps for confirming the test environment is ready specifically include: the scheduling center polling and checking the status of the controlled tools and devices to confirm network connectivity, simulator login status, and normal serial communication.

3. The method for implementing an automated testing platform integrating multiple tools according to claim 1, characterized in that, The platform's modules work in parallel to continuously collect data, specifically including: a network snapshot device to capture network interaction messages, a serial port module to record the serial port output of the system under test, and a simulator control module to record the simulator's state changes.

4. The method for implementing an automated testing platform integrating multiple tools according to claim 1, characterized in that, The report generator creates reports that include a test summary, execution logs, detailed data analysis results, judgment criteria, and conclusions.

5. The method for implementing a multi-tool integrated automated testing platform according to claim 1, characterized in that, The report generator creates a chain of evidence by associating and packaging the raw test data, intermediate analysis results, and the report.

6. A multi-tool integrated automated testing platform system, characterized in that, The system specifically includes a test management and scheduling center, a tool integration and adaptation layer, a data and configuration management center, and an analysis and judgment engine.

7. The automated testing platform system integrating multiple tools according to claim 6, characterized in that, The test management and scheduling center includes: The sequence editor provides a graphical interface for arranging, editing, and maintaining test sequences; The execution engine is used to parse and execute the test sequence and call the tool adapter according to predetermined logic; The resource manager is used to uniformly manage the status of software and hardware resources in the platform; The report generator is used to collect process and result data generated at each stage and automatically generate structured test reports according to predefined templates.

8. The automated testing platform system integrating multiple tools according to claim 6, characterized in that, The tool integration and adaptation layer is used to connect the test management and scheduling center with the test tools, including: The network capture adapter module is used to receive instructions from the test management and scheduling center, automatically capture, filter and store network packets, and provide real-time feedback on the packet capture status and file path. A serial communication parsing and adaptation module is used to communicate with the serial port of the interlocking device or simulator under test. The simulator cluster control adapter module is used to uniformly manage simulators, including operator display units, trackside electronic units, and area controllers. The physical interface simulation adapter module simulates physical operations in real-world scenarios by controlling external relay matrices or programmable switchgear.

9. The automated testing platform system integrating multiple tools according to claim 6, characterized in that, The data and configuration management center uses a database or file system for centralized management, including: A test case library is used to store the metadata and step definitions of test cases; The station configuration library is used to store hardware configuration data, network parameters, and PLC address mapping tables for different interlocking stations. The test results and evidence library is used to store the raw data and derived analysis results generated during each test execution.

10. The automated testing platform system integrating multiple tools according to claim 6, characterized in that, The analysis and judgment engine is used to analyze and judge the collected multi-source heterogeneous data, including: A multi-source data time synchronizer aligns all data to the same time base. The protocol and log analyzer has built-in parsing rules for rail transit-specific communication protocols and common log pattern recognition rules, which are used to extract key fields from raw messages and key events from system logs. Multidimensional metrics calculator, which calculates performance and functional metrics based on business rules; The rule-based automatic decision-maker draws conclusions and confidence levels based on preset thresholds and logical rules.

Images