A configuration software compatibility test method and system

Abstract

The present application relates to the technical field of configuration software, and provides a configuration software compatibility test method, comprising the following steps: S100, establishing a configuration software compatibility test design model; S200, starting from two dimensions of compatibility result influence, test cycle and resource demand, establishing a two-dimensional distribution diagram of test content influencing compatibility test quality and efficiency; obtaining key test content with great compatibility result influence, test cycle and resource demand in the two-dimensional distribution diagram; S300, based on the key test content with great compatibility result influence, test cycle and resource demand obtained from the two-dimensional distribution diagram, performing gray-box automatic test based on bottom design logic for configuration software new and old version compatibility test and configuration software and matching industrial application software compatibility test. The compatibility test of industrial configuration software can be comprehensively, systematically, accurately and efficiently carried out, the quality of the compatibility test of the software product is improved, and unpredictable serious consequences caused by software upgrading in the field are avoided.

Description

Technical Field

[0001] This invention relates to the technical field of configuration software, and more particularly to a method and system for testing the compatibility of configuration software. Background Technology

[0002] Industrial control systems operate in complex environments and rely on SCADA (Supervisory Control and Server) software. This software, centered around the control system, provides users with functions such as hardware configuration, control program programming and debugging, and system operation monitoring. It is a crucial tool for ensuring system operation and meeting customer needs. Industrial control systems are complex in composition, and the accompanying SCADA software possesses relatively complex functions. Influenced by factors such as changing user requirements, technological upgrades, defect repairs, and hardware / software replacements, SCADA software frequently undergoes version upgrades. For industrial applications, SCADA software upgrades are an extremely serious task, demanding near-stringent quality requirements for software compatibility. Any compatibility issues that could potentially cause production accidents are unacceptable. This necessitates that control system developers conduct rigorous compatibility testing for industrial SCADA software upgrades to ensure the safety of on-site software upgrades.

[0003] Software compatibility testing methods are a crucial aspect of software testing. Existing mainstream methods are primarily based on product application scenarios, evaluating the compatibility of a product in a specific application domain using certain methods or technologies. These methods focus on the compatibility of a particular parameter or concrete carrier, and generally have significant limitations. Industrial control system SCADA software, with its complex structure, rich functionality, numerous interfaces and associated software, involves compatibility issues far exceeding those of typical commercial software. Currently, there is no effective compatibility testing method specifically for industrial control system SCADA software. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a method and system for testing the compatibility of industrial configuration software. This method and system can comprehensively, systematically, accurately, and efficiently conduct compatibility testing of industrial configuration software, improve the quality of software product compatibility testing, and avoid unpredictable and serious consequences caused by software upgrades in the field.

[0005] The above-mentioned objective of this invention is achieved through the following technical solutions:

[0006] A method for testing the compatibility of configuration software, comprising:

[0007] S100: Establish a configuration software compatibility test design model. The principles and reference implementation steps of the test design model include:

[0008] S101: Determine the key design elements and formulate a compatibility test plan for the configuration software compatibility test from two dimensions, including test data and environment preparation and software business logic analysis.

[0009] S102: Define the requirements for the compatibility testing process, set up the test platform, and prepare test data and equipment;

[0010] S103: Determine the specific objects of compatibility testing, including operating systems, hardware platforms, operating data, and third-party software, and conduct compatibility testing of the configuration software based on the specific objects, and collect and record test data;

[0011] S104: With the goal of not affecting on-site control and normal operation, design the judgment principles for compatibility test results, obtain the test results, and output the software compatibility table.

[0012] Furthermore, the configuration software compatibility testing method also includes performing compatibility testing on the configuration software based on the test design model, and continuously iterating through steps S101-S104.

[0013] Furthermore, the configuration software compatibility testing method also includes:

[0014] S200: Based on the two dimensions of compatibility result impact, test cycle, and resource requirements, establish a two-dimensional distribution map of test content that affects the quality and efficiency of compatibility testing; and identify key test content with significant compatibility result impact, test cycle, and resource requirements from the two-dimensional distribution map.

[0015] Furthermore, the configuration software compatibility testing method also includes:

[0016] S300: Based on the key test content that has a significant impact on the compatibility results obtained from the two-dimensional distribution map, and has a large test cycle and resource requirements, gray-box automated testing based on the underlying design logic is carried out, including the compatibility test between the old and new versions of the configuration software and the compatibility test between the configuration software and the supporting industrial application software.

[0017] Furthermore, the gray-box automated testing based on the underlying design logic specifically includes:

[0018] S301: Based on the underlying design logic, identify the differences between the old and new versions of the configuration software and between the configuration software and the supporting industrial application software from the two dimensions of interface modules and data files, and identify the modified DLL files and shared data files.

[0019] S302: Based on the identified modified DLL files, develop a test driver, define the interface modules, interface functions, parameter definitions, and dependencies; based on the identified shared data files, develop a script file for shared file compatibility testing, list the files, clarify dependencies, and test from four aspects: creation, reading, writing, and deletion.

[0020] S303: Prepare test data and design automated test cases;

[0021] S304: Execute automated test cases based on the compatibility testing environment, modify test drivers and scripts, test data and test cases according to the running results, and continuously iterate until stability is achieved. Determine whether the automated testing method has reached stability based on the business logic requirements of the software under test.

[0022] S305: Continuously run stable automated test cases to determine the results of compatibility automated tests.

[0023] Further, in step S303, test data is prepared and automated test cases are designed, specifically as follows:

[0024] Prepare test data based on the business logic of the object under test, including initialization data, configuration data, real-time data, historical data, alarm data, and fault injection data, and design simulation methods for static and dynamic data.

[0025] The design of automated test cases is based on the Robot Framework, covering test case design including designing test case content based on business logic, designing test case combination relationships based on design logic, and designing test case execution flow based on the tool framework.

[0026] Further, in step S305, the results of the automated compatibility test are determined, specifically as follows:

[0027] The results of compatibility automated testing are judged from the perspectives of data processing results, control logic correctness, communication interaction stability, functional consistency, and achievement of business logic goals.

[0028] A configuration software compatibility testing system for performing the configuration software compatibility testing method described above includes:

[0029] The model building module is used to build a design model for compatibility testing of configuration software;

[0030] The 2D distribution map creation module is used to create a 2D distribution map of test content that affects the quality and efficiency of compatibility testing from two dimensions: the impact of compatibility results, the test cycle, and resource requirements.

[0031] The gray-box automated testing module is used to perform gray-box automated testing based on the underlying design logic for key test content that has a significant impact on compatibility results obtained from the two-dimensional distribution map, and which has high testing cycle and resource requirements. This includes compatibility testing of new and old versions of configuration software and compatibility testing of configuration software with supporting industrial application software.

[0032] A computer device includes a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method described above.

[0033] A computer-readable storage medium storing computer code that, when executed, performs the method described above.

[0034] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0035] (1) By establishing a configuration software compatibility test design model and proposing a systematic test method based on the model, the software compatibility can be evaluated systematically, comprehensively and accurately, ensuring that the software upgrade meets the requirements for safe operation of the control system. The test conclusions obtained provide effective guidance for the field application of the control system.

[0036] (2) By considering the impact of compatibility results, testing cycle, and resource requirements, a two-dimensional distribution map of test content affecting the quality and efficiency of compatibility testing is established. Key test content with significant impact on compatibility results, high testing cycle, and high resource requirements is identified from this two-dimensional distribution map. Extensive practical experience in control system configuration software compatibility testing is summarized to identify the test content that most significantly affects the efficiency and quality of configuration software compatibility testing, and methods are improved accordingly.

[0037] (3) Gray-box automated testing based on the underlying design logic is conducted on key test contents that have a significant impact on compatibility results obtained from the two-dimensional distribution map, including compatibility testing between new and old versions of the configuration software and compatibility testing between the configuration software and its supporting industrial application software. This changes the traditional manual black-box testing method based on the external characteristics of software functions, and significantly improves the accuracy, quality, and efficiency of the test. Attached Figure Description

[0038] Figure 1 This is an overall flowchart of the configuration software compatibility testing method of the present invention;

[0039] Figure 2 This is a model diagram of the method for ensuring compatibility of the control system configuration software according to the present invention;

[0040] Figure 3A general flowchart illustrating the implementation of the method model for compatibility testing of control system configuration software according to the present invention;

[0041] Figure 4 This is a network structure diagram of the application test platform for conducting compatibility testing of control system configuration software according to the present invention.

[0042] Figure 5 This is a flowchart of an embodiment of the control system configuration software compatibility testing method of the present invention;

[0043] Figure 6 This is a two-dimensional distribution diagram of test contents that affect the quality and efficiency of the compatibility test of the control system configuration software of the present invention.

[0044] Figure 7 This is a schematic diagram of the traditional testing method for the compatibility of new and old versions of the control system configuration software and the compatibility of the configuration software with other supporting industrial software.

[0045] Figure 8 This is a schematic diagram of a gray-box automated testing method based on underlying design logic for testing the compatibility of new and old versions of control system configuration software, as well as the compatibility of configuration software and other supporting industrial software.

[0046] Figure 9 This is an overall structural diagram of the configuration software compatibility testing system of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0049] The terminology involved in this invention is explained as follows:

[0050] DCS: Distributed Control System;

[0051] PLC: Programmable Logic Controller;

[0052] Configuration software, also known as configuration monitoring system software, refers to specialized software for data acquisition and process control. It also refers to the software platform and development environment at the monitoring layer of an automatic control system. It provides users with general-purpose software tools for quickly building monitoring functions for industrial automatic control systems.

[0053] For control systems applied in industrial settings, the core of configuration software compatibility testing is to study the functionality, interfaces, and compatibility with supporting products of the configuration software under field conditions. The key is to conduct effective, specialized design that closely reflects actual industrial applications, studying the impact of the configuration software upgrade on system functionality and performance to ensure production safety. The method proposed in this invention is as follows:

[0054] One key aspect is the proposal of a design model for the compatibility of control system configuration software based on field application conditions. Based on the model, a systematic testing method is proposed, which can systematically, comprehensively, and accurately evaluate software compatibility, ensuring that software upgrades meet the requirements for safe operation of the control system. The test conclusions obtained provide effective guidance for the field application of the control system.

[0055] The second key point is the proposal of a two-dimensional matrix factor analysis method to identify the critical test content affecting the quality and efficiency of configuration software compatibility testing. Configuration compatibility testing based on the aforementioned methods involves a wide testing scope, numerous test contents, and requires a long testing cycle and substantial testing resources. This invention proposes a two-dimensional matrix factor analysis method, summarizing extensive practical experience in control system configuration software compatibility testing, to identify the test content that most significantly impacts the efficiency and quality of configuration software compatibility testing, thereby improving the method.

[0056] Thirdly, based on the findings of the above methods, a gray-box automated testing method and apparatus based on underlying design logic are proposed to improve testing quality and effectiveness. Through the aforementioned two-dimensional matrix factor analysis, the key testing content affecting the quality and efficiency of configuration software compatibility testing—compatibility between new and old versions, and compatibility between the configuration software and other supporting industrial software—is identified. A gray-box automated testing method and apparatus based on underlying design logic are proposed, changing the traditional manual black-box testing method based on external software functionalities, significantly improving testing accuracy, quality, and efficiency.

[0057] This invention proposes a configuration software compatibility testing method and system for conducting compatibility tests on industrial control configuration software, ensuring that the software product compatibility meets the requirements for safe operation of the control system in the field. Its application scope includes compatibility testing and evaluation of the host computer configuration software of industrial control systems (DCS, PLC), and is used for compatibility testing and judgment after software version upgrades, providing technical support for product launch and subsequent field applications. The objectives of the technical solution proposed in this invention include:

[0058] (1) A method model for compatibility testing of configuration software under field application conditions is proposed to ensure the systematicness, comprehensiveness and accuracy of the design, process, content and result judgment of compatibility testing of configuration software, and to ensure the quality of compatibility testing.

[0059] (2) Based on the method model, formulate a comprehensive configuration software compatibility testing process to guide the systematic development of compatibility testing.

[0060] (3) A two-dimensional matrix factor analysis method is proposed to analyze the key test contents that affect the quality and efficiency of configuration software compatibility testing, and to improve the test method to enhance quality and efficiency.

[0061] (4) A gray-box automated testing method and device based on underlying design logic is proposed, which realizes automated testing of key test contents of control system configuration software compatibility, and improves the accuracy, quality and efficiency of testing.

[0062] First Embodiment

[0063] like Figure 1 As shown, this embodiment provides a method for testing the compatibility of configuration software, including:

[0064] S100: Establish a design model for configuration software compatibility testing, such as Figure 2 As shown, the principles and reference implementation steps of the test design model include:

[0065] S101: Determine the key design elements and formulate a compatibility test plan for the configuration software compatibility test from two dimensions: test data and environment preparation, and software business logic analysis. Test data and environment preparation includes new and old version controllers, new and old version configurations, field configurations, and auxiliary object versions. Software business logic analysis includes data interaction, data transmission, interoperability, and dependencies.

[0066] S102: Define the requirements for the compatibility testing process, build the test platform and prepare test data and equipment, ensure that the compatibility testing content and process comply with the control system application rules, and design software compatibility test cases accordingly, defining the relevant requirements for operation steps, observation and data recording.

[0067] S103: Determine the specific objects of compatibility testing, including operating systems, hardware platforms, operating data, and third-party software, and conduct compatibility testing of the configuration software based on the specific objects, and collect and record test data.

[0068] S104: Conduct compatibility testing, judging from the key performance indicators of the control system, and determine whether the tested object meets the compatibility requirements and can replace the previous version of the product for upgrading. Software compatibility evaluation should aim to "not affect on-site control and normal operation," designing principles for judging the results of compatibility tests to achieve online upgrades, normal operation in conjunction with other components of the control system, and no impact on the working status of system components. Obtain the test results and output a software compatibility table.

[0069] Compatibility testing of the configuration software is performed based on the test design model, such as... Figure 3 As shown, steps S101-S104 are executed iteratively to continuously accumulate relevant information and experience to guide the field application of the control system. With the release of the main version of the control system configuration software, compatibility testing is a crucial component and requires comprehensive testing. The method involves building a control system application test platform based on a simulated field environment, such as... Figure 4 Based on the compatibility testing method model proposed in this invention, testing is conducted, and the implementation process is as follows: Figure 5 .

[0070] Furthermore, the configuration software compatibility testing method of this embodiment also includes:

[0071] S200: Based on the two dimensions of compatibility result impact, test cycle, and resource requirements, establish a two-dimensional distribution map of test content that affects the quality and efficiency of compatibility testing; and identify key test content with significant compatibility result impact, test cycle, and resource requirements from the two-dimensional distribution map.

[0072] Specifically, referring to the test design model proposed in this invention, conducting comprehensive compatibility testing of control system configuration software requires a complex control system hardware environment, the design of numerous test cases, and typically employs manual testing methods, which consumes a long testing cycle and significant testing resources. Based on the methodological model, this invention proposes a two-dimensional matrix factor analysis method. Based on the inductive analysis of numerous configuration software compatibility test results, as shown in Table 1, a two-dimensional distribution diagram of test content affecting the quality and efficiency of compatibility testing is derived from two dimensions: the impact of compatibility results, testing cycle, and resource requirements (time and manpower). Figure 6 As shown, the key test areas that urgently need improvement in testing methods are: compatibility between new and old versions, and compatibility between configuration software and other supporting industrial software.

[0073] Table 1

[0074]

[0075] Furthermore, analyzing the conclusions obtained in step S200 above, regarding the compatibility of new and old versions of the control system configuration software, and the compatibility of the configuration software with other supporting industrial software, the shortcomings of traditional manual testing methods are analyzed, such as... Figure 7 As shown, the configuration software compatibility testing method of this embodiment is as follows: Figure 8 As shown, it also includes:

[0076] S300: Based on the key test content that has a significant impact on the compatibility results obtained from the two-dimensional distribution map, and has a large test cycle and resource requirements, gray-box automated testing based on the underlying design logic is carried out, including the compatibility test between the old and new versions of the configuration software and the compatibility test between the configuration software and the supporting industrial application software.

[0077] Furthermore, the gray-box automated testing based on the underlying design logic specifically includes:

[0078] S301: Based on the underlying design logic, identify the differences between the old and new versions of the configuration software and between the configuration software and the supporting industrial application software from the two dimensions of interface modules and data files, and identify the modified DLL files and shared data files.

[0079] S302: Based on the identified modified DLL files, develop a test driver, define the interface modules, interface functions, parameter definitions, and dependencies; based on the identified shared data files, develop a script file for shared file compatibility testing, list the files, clarify dependencies, and test from four aspects: creation, reading, writing, and deletion.

[0080] S303: Prepare test data and design automated test cases. Prepare test data based on the business logic of the object under test, including initialization data, configuration data, real-time data, historical data, alarm data, and fault injection data, and design simulation methods for static and dynamic data; design automated test cases based on the Robot Framework, including test case design based on business logic, test case combination relationships based on design logic, and test case execution flow based on the tool framework.

[0081] S304: Automated test execution and correction. Automated test cases are executed based on the compatibility testing environment. Test drivers, scripts, test data, and test cases are corrected based on the execution results, and continuous iteration is performed until stability is achieved. The stability of the automated testing method is determined based on the business logic requirements of the software under test.

[0082] S305: Continuously run stable automated test cases to evaluate the results of compatibility automated tests. The evaluation considers factors including data processing results, control logic correctness, communication interaction stability, functional consistency, and achievement of business logic goals.

[0083] Second Embodiment

[0084] like Figure 9 As shown, this embodiment provides a configuration software compatibility testing system for performing the configuration software compatibility testing method as described in the first embodiment, comprising:

[0085] Model building module 1 is used to build a design model for compatibility testing of configuration software.

[0086] Module 2, which establishes a two-dimensional distribution map, is used to establish a two-dimensional distribution map of test content that affects the quality and efficiency of compatibility testing from two dimensions: the impact of compatibility results, the test cycle, and resource requirements.

[0087] The gray-box automated testing module 3 is used to perform gray-box automated testing based on the underlying design logic for key test contents that have a significant impact on compatibility results obtained from the two-dimensional distribution map, such as the compatibility test between the old and new versions of the configuration software and the compatibility test between the configuration software and its supporting industrial application software.

[0088] A computer-readable storage medium stores computer code that, when executed, performs the methods described above. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0089] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for testing the compatibility of configuration software, characterized in that, include: S100: Establish a configuration software compatibility test design model. The principles and reference implementation steps of the test design model include: S101: Determine the key design elements and formulate a compatibility test plan for the configuration software compatibility test from two dimensions, including test data and environment preparation and software business logic analysis. S102: Define the requirements for the compatibility testing process, set up the test platform, and prepare test data and equipment; S103: Determine the specific objects of compatibility testing, including operating systems, hardware platforms, operating data, and third-party software, and conduct compatibility testing of the configuration software based on the specific objects, and collect and record test data; S104: With the goal of not affecting on-site control and normal operation, design the judgment principles for compatibility test results, obtain the test results, and output the software compatibility table; S200: Based on the two dimensions of compatibility result impact, test cycle, and resource requirements, establish a two-dimensional distribution map of test content that affects the quality and efficiency of compatibility testing; and identify the key test content with a large impact on compatibility results, test cycle, and resource requirements from the two-dimensional distribution map. S300: Based on the key test content that has a significant impact on the compatibility results obtained from the two-dimensional distribution map, and has a large test cycle and resource requirements, gray-box automated testing based on the underlying design logic is carried out, including the compatibility test between the old and new versions of the configuration software and the compatibility test between the configuration software and the supporting industrial application software.

2. The configuration software compatibility testing method according to claim 1, characterized in that, Also includes: The configuration software is subjected to compatibility testing based on the test design model, and steps S101-S104 are continuously iterated.

3. The configuration software compatibility testing method according to claim 1, characterized in that, The gray-box automated testing based on the underlying design logic specifically includes: S301: Based on the underlying design logic, identify the differences between the old and new versions of the configuration software and between the configuration software and the supporting industrial application software from the two dimensions of interface modules and data files, and identify the modified DLL files and shared data files. S302: Based on the identified modified DLL file, develop a test driver, specifying the interface modules, interface functions, parameter definitions, and dependencies; Based on the identified shared data files, a script file for shared file compatibility testing was developed, listing the files, sorting out the dependencies, and testing from four aspects: creation, reading, writing, and deletion. S303: Prepare test data and design automated test cases; S304: Execute automated test cases based on the compatibility testing environment, modify test drivers and scripts, test data and test cases according to the running results, and continuously iterate until stability is achieved. Determine whether the automated testing method has reached stability based on the business logic requirements of the software under test. S305: Continuously run stable automated test cases to determine the results of compatibility automated tests.

4. The configuration software compatibility testing method according to claim 3, characterized in that, In step S303, test data is prepared and automated test cases are designed, specifically as follows: Prepare test data based on the business logic of the object under test, including initialization data, configuration data, real-time data, historical data, alarm data, and fault injection data, and design simulation methods for static and dynamic data. The design of automated test cases is based on the Robot Framework, covering test case design including designing test case content based on business logic, designing test case combination relationships based on design logic, and designing test case execution flow based on the tool framework.

5. The configuration software compatibility testing method according to claim 3, characterized in that, In step S305, the results of the automated compatibility test are determined, specifically as follows: The results of compatibility automated testing are judged from the perspectives of data processing results, control logic correctness, communication interaction stability, functional consistency, and achievement of business logic goals.

6. A configuration software compatibility testing system for performing the configuration software compatibility testing method as described in any one of claims 1-5, characterized in that, include: The model building module is used to build a design model for compatibility testing of configuration software; The 2D distribution map creation module is used to create a 2D distribution map of test content that affects the quality and efficiency of compatibility testing from two dimensions: the impact of compatibility results, the test cycle, and resource requirements. The gray-box automated testing module is used to perform gray-box automated testing based on the underlying design logic for key test content that has a significant impact on compatibility results obtained from the two-dimensional distribution map, and which has high testing cycle and resource requirements. This includes compatibility testing of new and old versions of configuration software and compatibility testing of configuration software with supporting industrial application software.

7. A computer device comprising a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing computer code, wherein when the computer code is executed, the method of any one of claims 1 to 5 is performed.

Images