Automated testing method, device, computer equipment and storage medium

Abstract

The present application relates to an automated testing method, device, computer equipment and storage medium. The method comprises: obtaining a test recording file of at least one test case and manual test result information of the test case, wherein the test recording file is a macro file recorded by the test system when the test case is manually tested and the test is successful; calling the test system to perform automated testing on the object under test according to the test recording file to obtain automated test result information; based on the test redundancy corresponding to the manual test result information, performing information comparison between the automated test result information and the manual test result information; when the automated test result information and the manual test result information are consistent, determining that the test result of the test case of the object under test is a test pass. The use of this method can reduce the complexity, test duration and test cost of automated testing, and improve the efficiency of automated testing.

Description

Technical Field

[0001] The present application relates to the field of control and detection technology, and in particular to an automated testing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] Software function testing is an indispensable part of the development and use of software control systems, and plays a vital role in improving software quality, reliability, stability, and user satisfaction. To test the functions of software control systems, a dedicated test system, such as CANoe (Controller Area Network openenvironment, bus development environment) is usually used in conjunction with corresponding hardware tools to achieve automated testing of software control systems.

[0003] Among them, the test system needs to rely on test scripts written in advance according to test cases to run automated tests, that is, automated testing requires manual development of test scripts and debugging. Since the script writing process requires code writing by testers, the current automated testing has the problems of high testing requirements, long testing time, and high testing costs. Especially when the number of test case items increases dramatically, the efficiency of automated testing will also drop significantly. Summary of the invention

[0004] Based on this, it is necessary to provide an automated testing method, device, computer equipment, computer-readable storage medium and computer program product that can reduce the complexity, duration and cost of automated testing and improve the efficiency of automated testing in response to the above technical problems.

[0005] In a first aspect, the present application provides an automated testing method, the method comprising:

[0006] Acquire a test recording file of at least one test case and manual test result information of the test case, wherein the test recording file is a macro file recorded by the test system when the test case is manually tested and the test succeeds;

[0007] Calling the test system to perform automated testing on the object under test according to the test recording file to obtain automated testing result information;

[0008] Based on the test redundancy corresponding to the manual test result information, performing information comparison between the automated test result information and the manual test result information;

[0009] When the automated test result information is consistent with the manual test result information, it is determined that the test result of the use case of the object under test is a test pass.

[0010] In the above embodiment, the test personnel can first perform manual testing on the test case on the test system, and record the operation steps of the test process in a macro file through the test system. When the test is successful, the macro file is saved to obtain the test recording file of the test case. When the function test is performed on the object under test, the test tool can directly obtain the test recording file of the test case and the manual test result information of the test case, call the test system to perform automated testing on the object under test according to the test recording file, obtain automated test result information, and compare the automated test result information with the manual test result information based on the test redundancy corresponding to the manual test result information. Since the manual test result is the test result obtained when the manual test is successful, therefore, when the automated test result information is consistent with the manual test result information, it can be determined that the test result of the test case of the object under test is the test passed. The entire test process does not require manual development of test scripts and debugging of test cases. For each test case, it is only necessary to record the manual test process, which effectively reduces the complexity of automated testing, test time and test cost. Even if the test case items increase dramatically, the time required for the manual operation recording process will be much lower than the development time of the test script, which effectively improves the detection efficiency of automated testing.

[0011] In some embodiments, the calling of the test system to perform automated testing on the object under test according to the test recording file to obtain automated testing result information includes:

[0012] Obtaining a system configuration file of the test system;

[0013] According to the file storage address of the test recording file, the system configuration file is updated to obtain an updated system configuration file;

[0014] Based on the updated system configuration file, the test system is called to perform automated testing on the object under test according to the test recording file to obtain automated testing result information.

[0015] In the above embodiment, the test tool can control the test system to accurately perform automated testing on the object under test according to the test recording file by updating the system configuration file of the test system, thereby improving the accuracy and test efficiency of the automated testing.

[0016] In some embodiments, the number of the test cases is multiple; the updating of the system configuration file according to the file storage address of the test recording file to obtain the updated system configuration file includes:

[0017] Obtaining the test sequence information of each of the test cases, and determining the running sequence of each of the test recording files;

[0018] Based on the running order of each of the test recording files, the storage address of each of the test recording files is written into the system configuration file to obtain an updated system configuration file.

[0019] In the above embodiment, the storage address of each test recording file is written into the system configuration file based on the running order of each test recording file. When the test system is subsequently called to perform automated testing on the object under test based on the updated system configuration file, each test recording file can be sequentially batch run. Even if there are multiple test cases that need to be automated tested, the test accuracy and efficiency of the automated test can also be improved.

[0020] In some embodiments, the method further comprises:

[0021] Acquire at least one target observation signal for the object under test;

[0022] Determine a first signal test result of the target observation signal from the manual test result information;

[0023] The test redundancy of the target observation signal is determined according to the first signal test result.

[0024] In the above embodiment, by acquiring at least one target observation signal for the object under test and taking the first signal test result obtained during manual testing of the target observation signal as a benchmark, the test redundancy of the target observation signal is determined, thereby improving the accuracy of determining the test redundancy, leaving a test margin for subsequent result comparison, reducing the probability of misjudgment due to differences in the test process, and further improving the test accuracy of the automated test.

[0025] In some embodiments, the comparing the automated test result information with the manual test result information based on the test redundancy corresponding to the manual test result information includes:

[0026] Determine a second signal test result of the target observation signal from the automated test result information;

[0027] Comparing the first signal test result with the second signal test result based on the test redundancy;

[0028] When the first signal test result is consistent with the second signal test result, it is determined that the automated test result information is consistent with the manual test result information.

[0029] In the above embodiment, the signal test result comparison between the first signal test result and the corresponding second signal test result is performed based on the test redundancy, which can effectively reduce the probability of misjudgment caused by differences in the test process and further improve the test accuracy of the automated test.

[0030] In some embodiments, comparing the first signal test result with the second signal test result based on the test redundancy includes:

[0031] Determining a normal test interval of the target observation signal based on the test redundancy and the first signal test result;

[0032] Determine a result difference value between the first signal test result and the second signal test result;

[0033] When the result difference value belongs to the normal test interval, it is determined that the first signal test result is consistent with the second signal test result.

[0034] In the above embodiment, the normal test interval is determined for the target observation signal by testing redundancy and the first signal test result, and the result difference value is then directly compared with the normal test interval, which can effectively improve the signal test result comparison efficiency and accuracy.

[0035] In some embodiments, determining a result difference value between the first signal test result and the second signal test result includes:

[0036] Determine first signal change coordinate information of the target observation signal according to the first signal test result, and determine second signal change coordinate information of the target observation signal according to the second signal test result;

[0037] Generate a comparison line graph based on the first signal change coordinate information and the second signal change coordinate information;

[0038] The result difference value between the first signal test result and the second signal test result is determined according to the comparison line graph.

[0039] In the above embodiment, by generating a comparison line graph, the first signal test result and the second signal test result of the target observation signal are compared for the difference in results, which effectively improves the comparison efficiency and comparison intuitiveness of the test results, thereby improving the test efficiency of the automated test.

[0040] In some embodiments, there are multiple target observation signals, and each of the target observation signals and each comparison line Figure 1 Correspondingly, the method further includes:

[0041] A test report for the object under test is generated based on the use case test results of the object under test, the automated test result information, the test cases, and the comparison line graphs of each of the target observation signals.

[0042] In the above embodiment, when generating a test report of the object under test, in order to improve the intuitiveness of the information display of the test report, each target observation signal and the individual comparison curve Figure 1 One-to-one correspondence, that is, a comparison line graph will only contain the comparison between the first signal test result and the second signal test result of a target observation signal. This can reduce the situation where the intuitiveness of the comparison line graph is reduced due to large differences in the test results of each target observation signal, and effectively improve the intuitiveness of the information display of the test report.

[0043] In some embodiments, the acquiring of at least one target observation signal for the object under test includes:

[0044] Loading a signal database file in the test system;

[0045] generating a signal selection page based on the signal database file;

[0046] In response to a signal selection operation triggered on the signal selection page, at least one target observation signal for the object under test is acquired.

[0047] In the above embodiment, by loading the signal database file of the test system and generating a signal selection page based on the signal database file, it is convenient for detection personnel to select the corresponding target observation signal from a large number of signals, thereby improving the accuracy and convenience of selecting the target observation signal.

[0048] In some embodiments, the calling of the test system to perform automated testing on the object under test according to the test recording file to obtain automated testing result information includes:

[0049] Determining the estimated duration of the automated test of the object under test according to the file recording duration of the test recording file;

[0050] Calling the test system to perform automated testing on the object under test according to the test recording file, and obtaining the test duration of the automated test in real time;

[0051] When the test duration reaches the expected duration of the automated test, determining that the test system meets the automated test end condition;

[0052] The automated test result information of the tested object is obtained based on the test system.

[0053] In the above embodiment, the testing tool can obtain the enabling conditions for stopping the automated testing of the testing system by determining the estimated duration of the automated testing of the object under test. According to the enabling conditions for stopping the automated testing, the timing for stopping the automated testing of the testing system can be accurately determined, thereby increasing the speed of obtaining the automated testing result information of the object under test from the testing system and reducing the duration of the automated testing.

[0054] In some embodiments, there are multiple test recording files; determining the estimated duration of the automated test of the object under test according to the recording duration of the test recording files includes:

[0055] Counting the recording time of each test recording file to obtain the theoretical estimated test time of the object under test;

[0056] Based on the theoretical test estimated duration and the preset correspondence between the test estimated duration and the duration redundancy, determining the duration redundancy that matches the theoretical test estimated duration;

[0057] The estimated duration of the automated test of the object under test is determined based on the estimated duration of the theoretical test and the duration redundancy.

[0058] In the above embodiment, when there are multiple test recording files, the theoretical test estimated duration is obtained by counting the file recording duration of each test recording file, and the matching duration redundancy is determined for the theoretical test estimated duration. This can effectively reduce the difference between the running time of the test tool and the running time of the test system, which leads to the deviation of the theoretical test estimated duration, effectively improve the accuracy of the automated test estimated duration, and thus improve the speed of obtaining the automated test result information of the object under test from the test system, thereby reducing the automated test duration.

[0059] In a second aspect, the present application also provides an automated testing device, the device comprising:

[0060] An information acquisition module, used to acquire a test recording file of at least one test case and manual test result information of the test case, wherein the test recording file is a macro file recorded by the test system when the test case is manually tested and the test succeeds;

[0061] The automated testing module is used to call the test system to perform automated testing on the object under test according to the test recording file to obtain automated testing result information;

[0062] A result comparison module is used to compare the automated test result information with the manual test result information based on the test redundancy corresponding to the manual test result information; if the automated test result information is consistent with the manual test result information, determine that the use case test result of the object under test is a passed test.

[0063] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0064] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0065] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0066] With the above-mentioned automated testing method, apparatus, computer equipment, storage medium and computer program product, the testing personnel can first perform manual testing on the test case on the test system, and record the operating steps of the test process in a macro file through the test system. When the test is successful, the macro file is saved to obtain the test recording file of the test case. When performing functional testing on the object under test, the testing tool can directly obtain the test recording file of the test case and the manual test result information of the test case, call the test system to perform automated testing on the object under test according to the test recording file, obtain automated test result information, and compare the automated test result information with the manual test result information based on the test redundancy corresponding to the manual test result information. Since the manual test result is the test result obtained when the manual test is successful, when the automated test result information is consistent with the manual test result information, it can be determined that the test result of the use case of the object under test is a passed test. The entire testing process does not require manual development of test scripts and debugging of test cases. For each test case, you only need to record the manual testing process, which effectively reduces the complexity, duration and cost of automated testing. Even if the number of test case items increases dramatically, the time required for manual operation recording will be much shorter than the development time of the test script, effectively improving the detection efficiency of automated testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A diagram of an application environment of an automated testing method in some embodiments;

[0068] Figure 2 A schematic diagram of a flow chart of an automated testing method in some embodiments;

[0069] Figure 3 A schematic diagram of a process of calling a test system to perform automated testing on a tested object according to a test recording file to obtain automated testing result information in some embodiments;

[0070] Figure 4 Schematic diagram of the process flow of the automated testing method in some other embodiments;

[0071] Figure 5 A schematic diagram of a process of comparing a first signal test result with a second signal test result based on test redundancy in some embodiments;

[0072] Figure 6 A schematic diagram of a process for determining a result difference value between a first signal test result and a second signal test result in some embodiments;

[0073] Figure 7 A schematic diagram of a process of obtaining at least one target observation signal for a measured object in some embodiments;

[0074] Figure 8 A schematic diagram of a flow chart of calling a test system to perform automated testing on a tested object according to a test recording file to obtain automated testing result information in some other embodiments;

[0075] Fig. 9 A schematic diagram of a process for determining an estimated duration of an automated test of a tested object according to a file recording duration of a test recording file in some embodiments;

[0076] Fig.10 Schematic diagram of the process flow of the automated testing method in some other embodiments;

[0077] Fig.11 It is a structural block diagram of an automated testing device in some embodiments;

[0078] Fig.12 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0079] The following embodiments of the technical solution of the present application will be described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0081] Reference to "embodiments" herein means that the specific features, structures, or characteristics described in conjunction with the embodiments may be included in at least some embodiments of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0082] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0083] In the description of the embodiments of the present application, the technical terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the embodiments of the present application.

[0084] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0085] Software function testing is an important means to improve software quality, meet user needs, and reduce the maintenance cost of control systems. When testing the functions of software control systems, special test systems are usually used, such as CANoe (Controller Area Network open environment) and corresponding hardware tools to achieve automated testing of software control systems.

[0086] When the test system performs automated testing on the object under test, such as the vehicle control system, battery control system, etc., it is often necessary for the test personnel to write the corresponding test scripts according to the test cases in advance and perform automated testing based on the test scripts. However, the development and debugging of the test scripts increase the code writing requirements for the test personnel, and also increase the time and cost of automated testing.

[0087] In order to improve the efficiency of automated testing of the object under test, the test personnel can first perform manual testing on the test case on the test system, operate the global variables in the test system, and record the operation steps of the test process in a macro file through the test system. When the test is successful, save the macro file to obtain the test recording file of the test case. When performing functional testing on the object under test, the test tool can directly obtain the test recording file of the test case and the manual test result information of the test case, call the test system to perform automated testing on the object under test according to the test recording file, obtain the automated test result information, and compare the automated test result information with the manual test result information. Since the manual test result is the test result obtained when the manual test is successful, therefore, when the automated test result information is consistent with the manual test result information, it can be determined that the test result of the case of the object under test is the test passed. The entire test process does not require manual development of test scripts and debugging of test cases. For each test case, it is only necessary to record the manual test process, which effectively reduces the complexity of automated testing, test time and test cost. Even if the test case items increase dramatically, the time required for the manual operation recording process will be much lower than the development time of the test script, which effectively improves the detection efficiency of automated testing.

[0088] The automated testing method provided in the embodiments of the present application can be applied to Figure 1In the application environment shown. Among them, the test tool 102 communicates with the test system 104 through the network, and the data storage system can store the data that the test tool 102 needs to process. The data storage system can be integrated on the test tool 102, or it can be placed on the cloud or other network servers. The test personnel can manually test the test case through the test system 104, operate the global variables in the test system 104, and record the operating steps of the test process through the test system 104. In the case of a successful test, save the macro file to obtain the test recording file of the test case. When performing a functional test on the object under test, the test tool 102 obtains a test recording file of at least one test case and the manual test result information of the test case, calls the test system 104 to perform an automated test on the object under test according to the test recording file, and obtains the automated test result information. When the automated test result information is consistent with the manual test result information, it is determined that the test result of the use case of the object under test is a test pass.

[0089] Among them, the test tool 102 is a tool developed and generated based on programming language software. For example, the tester can generate the test tool 102 based on python software or matlab software. The test tool 102 is connected to the test system 104 through a preset communication interface, and can realize information transmission, loading, and system scheduling functions. At the same time, the test tool 102 can also parse the result information generated after the test system 104 is tested, and complete the information comparison between the automated test result information and the manual test result information.

[0090] The test system 104 can be a system used to develop, test, and analyze the object under test, that is, the control system bus, such as a CANoe (Controller Area Network open environment, bus development environment) system, which supports the entire control system development process from demand analysis to system implementation, and can provide rich functions and configuration options, such as message recording and playback, simulated message sending, and establishment of simulation projects.

[0091] In some embodiments, Figure 2 As shown, an automated testing method is provided, which is applied to Figure 1 The test tool 102 in the example is used as an example to illustrate, and the following steps are included:

[0092] S202: Obtain a test recording file of at least one test case and manual test result information of the test case.

[0093] Among them, the test case is the guidance information data designed to test whether a certain function of the object under test meets specific requirements, which can include information such as test input, execution conditions and expected results. The test case is a guide for testing work, which can help testers clarify the test objectives, methods, input data, expected outputs, etc., so that the quality of the object under test after the test is successful meets expectations.

[0094] The test recording file is a macro file recorded by the test system when the test case is manually tested and the test is successful. For each test case, the tester can perform manual testing on the test system, that is, manually operate the global variables in the test system to obtain the corresponding manual test results. In the process of the tester manually testing the test case through the test system, the macro recording function of the test system can be used to record the entire operation process of the manual test to obtain the candidate test file of the test case. When the tester judges the manual test result and determines that the test is successful, the candidate test file can be determined as the test recording file of the test case, and the manual test result information is generated according to the manual test result.

[0095] Since the manual test result information is information data generated by the test system according to the manual test result, the information format of the manual test result information is determined according to the test system. For example, when the test system is a CANoe system, the corresponding manual test result information is a Log file. The manual test result information may include the manual test results of each test signal.

[0096] In some optional embodiments, when an automated test is required for the object under test, the test tool may obtain a test recording file of at least one test case and manual test result information of the test case. It is understandable that the number of test cases that the test tool needs to obtain may be determined by the tester according to actual test requirements.

[0097] In some embodiments, the test recording file of the test case can be added by the test personnel through the test tool. After the test personnel obtains the test recording file of at least one test case based on the test system, they can start the test tool, select the test recording file to be automatically executed and upload it based on the test tool, and the test tool can obtain the test recording file of at least one test case.

[0098] S204, calling the test system to perform automated testing on the object under test according to the test recording file, and obtaining automated testing result information.

[0099] The automated test result information is the result information data obtained after the test system performs automated testing on the object under test according to the test recording file. Similarly, the information format of the automated test result information is determined according to the test system. For example, when the test system is a CANoe system, the corresponding automated test result information is also a Log file. The automated test result information can include the automated test results of each test signal.

[0100] In some optional embodiments, after obtaining a test recording file of at least one test case, the test tool may call the test system to perform automated testing on the object under test according to the operation process recorded in the test recording file to obtain automated testing result information.

[0101] S206: When the automated test result information is consistent with the manual test result information, determine that the test result of the use case of the object under test is a passed test.

[0102] The use case test result of the object under test refers to the test result information of the object under test when the test case is used to test the object under test. When there is only one test case, the use case test result of the object under test refers to the test result information obtained by using the test case to test the object under test. When there are multiple test cases, the use case test result of the object under test may include the test result information obtained by using each test case to test the object under test.

[0103] In some of the embodiments, for each test case, the test result of the object under test may include a test pass and a test fail.

[0104] Since the manual test result information is the result information data obtained when the test case is manually tested and the test is successful, in some optional embodiments, after obtaining the automated test result information, the testing tool can compare the automated test result information with the manual test result information. If the comparison is consistent, it means that the automated test result of the object under test is consistent with the manual test result obtained when the test is successful, and it is determined that the case test result of the object under test is a passed test.

[0105] In some of the embodiments, when the automated test result information is inconsistent with the manual test result information, it means that the automated test result of the object under test is inconsistent with the manual test result obtained when the test is successful, and the testing tool can determine that the use case test result of the object under test is a test failure.

[0106] In some of the embodiments, the testing tool may parse the automated test result information and the manual test result information, and compare the information data obtained after the parsing to determine whether the automated test result information and the manual test result information are consistent.

[0107] In some of the embodiments, an information comparison model is pre-set in the testing tool. The testing tool can input the automated test result information and the manual test result information into the information comparison model to compare the result information to determine whether the automated test result information and the manual test result information of the object under test are consistent.

[0108] In the above-mentioned automated testing method, the test personnel can first perform manual testing on the test case on the test system, and record the operation steps of the test process in a macro file through the test system. When the test is successful, the macro file is saved to obtain the test recording file of the test case. When the function test is performed on the object under test, the test tool can directly obtain the test recording file of the test case and the manual test result information of the test case, call the test system to perform automated testing on the object under test according to the test recording file, obtain the automated test result information, and compare the automated test result information with the manual test result information. Since the manual test result is the test result obtained when the manual test is successful, therefore, when the automated test result information is consistent with the manual test result information, it can be determined that the test result of the test case of the object under test is the test passed. The entire test process does not require manual development of test scripts and debugging of test cases. For each test case, it is only necessary to record the manual test process, which effectively reduces the complexity of automated testing, test time and test cost. Even if the test case items increase dramatically, the time required for the manual operation recording process will be much lower than the development time of the test script, which effectively improves the detection efficiency of automated testing.

[0109] In some of the embodiments, the automated testing method further includes: performing information comparison between the automated testing result information and the manual testing result information based on the test redundancy corresponding to the manual testing result information.

[0110] Among them, test redundancy is a preset parameter used to offset the process differences between the manual test process and the automated test process, and can represent the possible result differences between the manual test results and the automated test results due to process differences.

[0111] In some optional embodiments, the testing tool may perform information comparison between the automated test result information and the manual test result information based on the test redundancy corresponding to the manual test result information.

[0112] In some of the embodiments, a redundancy determination model may be pre-set in the test tool. After obtaining the manual test result information, the test tool may input the manual test result information into the redundancy determination model to determine the test redundancy corresponding to the manual test result information.

[0113] In other embodiments, a preset correspondence between test result information and redundancy is pre-set in the test tool. After obtaining the manual test result information, the test tool can call the preset correspondence between the test result information and redundancy to find the test redundancy that matches the manual test result information.

[0114] In the above embodiment, by comparing the automated test result information with the manual test result information through the test redundancy corresponding to the manual test result information, the test result differences caused by the process differences between the manual test process and the automated test process can be offset to a certain extent, thereby improving the accuracy of the result comparison.

[0115] The calling of the test system by the test tool is a key step in realizing the automated testing of the object under test based on the test tool. Figure 3 As shown, S204, calling the test system to perform automated testing on the object under test according to the test recording file, and obtaining automated test result information, including:

[0116] S302, obtaining a system configuration file of the test system.

[0117] Among them, the system configuration file of the test system is the configuration file of the test system project, which can also be understood as an engineering file. It can contain the project settings, configuration information and related parameter settings of the test system, and is the basis for starting and running the corresponding test projects of the test system. In automated testing, the system configuration file of the test system usually contains the configuration information of the test case, such as test steps, expected results, etc. The test system can automatically execute the test case and collect and analyze the test results by reading the corresponding file content of the system configuration file. For example, when the test system is a CANoe system, the system configuration file can be the cfg (Configuration File) file of the CANoe system.

[0118] In some optional embodiments, the testing tool may obtain a system configuration file of the testing system in order to mobilize the testing system to perform automated testing on the object under test according to the test recording file.

[0119] S304: Update the system configuration file according to the file storage address of the test recording file to obtain an updated system configuration file.

[0120] The file storage address of the test recording file is the storage address of the test recording file in the test tool.

[0121] In some optional embodiments, the test tool writes the file storage address of the test recording file into a system configuration file of the test system according to the file storage address of the test recording file, updates the information of the system configuration file, and obtains an updated system configuration file.

[0122] S306, based on the updated system configuration file, calling the test system to perform automated testing on the object under test according to the test recording file, and obtaining automated testing result information.

[0123] In some optional embodiments, after completing the update of the system configuration file, the test tool can trigger the automatic operation of the test system and call the test system. The test system can read the system configuration file, obtain the test recording file according to the file storage address of the test recording file in the updated system configuration file, and perform automated testing on the object under test according to the test recording file to obtain automated test result information.

[0124] In the above embodiment, the test tool can control the test system to accurately perform automated testing on the object under test according to the test recording file by updating the system configuration file of the test system, thereby improving the accuracy and test efficiency of the automated testing.

[0125] In some embodiments, there are multiple test cases. S304, according to the file storage address of the test recording file, the system configuration file is updated to obtain an updated system configuration file, including: obtaining the test sequence information of each test case, and determining the running order of each test recording file. Based on the running order of each test recording file, the storage address of each test recording file is written into the system configuration file to obtain an updated system configuration file.

[0126] Among them, the test sequence information of each test case specifies the execution order of the corresponding test case when the object under test is subjected to automated testing. It is understandable that the test sequence information of each test case can be predetermined by the test personnel according to the actual test requirements. For example, the test personnel can generate corresponding test sequence information according to the execution order of each test case when it is manually executed, and upload the test sequence information to the test tool. Alternatively, the test personnel can upload each test case to the test tool according to the execution order of each test case, and the test tool determines the test sequence information of each test case according to the upload order of each test case.

[0127] In some optional embodiments, the test tool can obtain the test sequence information of each test case, and determine the running order of each test recording file corresponding to each test case according to the execution order of each test case specified by the test sequence information of each test case. Then, based on the running order of each test recording file, the storage address of each test recording file is written into the system configuration file to obtain an updated system configuration file.

[0128] In the above embodiment, the storage address of each test recording file is written into the system configuration file based on the running order of each test recording file. When the test system is subsequently called to perform automated testing on the object under test based on the updated system configuration file, each test recording file can be sequentially batch run. Even if there are multiple test cases that need to be automated tested, the test accuracy and efficiency of the automated test can also be improved.

[0129] Comparison of the automated test results with the manual test results is also a key step in determining the test results of the object being tested. Figure 4 As shown, the automated testing method also includes:

[0130] S402: Acquire at least one target observation signal for the object under test.

[0131] Among them, the target observation signal refers to the test signal that needs to be compared when the result information is compared. It can be understood that the test result information of all test signals exists in both the automated test result information and the manual test result information. Among these test signals, there are signals that do not need to be compared, or signals that cannot be compared, such as signals with different signal values ​​at the same time point under the same working conditions. Therefore, in order to reduce misjudgment during information comparison and improve test accuracy, it is necessary to determine the target observation signal of the object under test from each test signal. It can be understood that the number and type of target observation signals can be determined according to the actual observation needs of the object under test.

[0132] In some optional embodiments, the test tool may acquire at least one target observation signal for the object under test.

[0133] In some of the embodiments, the testing personnel can determine at least one target observation signal of the object under test in advance based on the actual observation requirements of the object under test, and set a corresponding target observation identifier for the target observation signal. The testing tool only needs to identify whether there is a target observation identifier for each test signal to determine at least one target observation signal for the object under test.

[0134] S404: Determine a first signal test result of the target observation signal from the manual test result information.

[0135] In some optional embodiments, the test tool may parse the manual test result information and determine the first signal test result of each target observation signal in the manual test result information according to at least one target observation signal of the object under test.

[0136] S406: Determine the test redundancy of the target observation signal according to the first signal test result.

[0137] In some optional embodiments, in order to offset to a certain extent the differences in test results caused by the process differences between the manual testing process and the automated testing process, the testing tool can determine the test redundancy of the target observation signal based on the first signal test result, that is, the signal test result obtained by the manual testing process.

[0138] In some of the embodiments, a preset correspondence between signal test results and test redundancy is pre-set in the test tool, and the test tool can search for a test redundancy that matches the first signal test result in the preset correspondence between signal test results and test redundancy through the first signal test result.

[0139] In some of the embodiments, the test tool may also pre-set a method for calculating test redundancy, and the test tool may bring the first signal test result into the method for calculating test redundancy to obtain a test redundancy that matches the first signal test result.

[0140] It should be noted that the test redundancy may include the redundancy of the signal change time and / or the redundancy of the signal change value.

[0141] In the above embodiment, by acquiring at least one target observation signal for the object under test and taking the first signal test result obtained during manual testing of the target observation signal as a benchmark, the test redundancy of the target observation signal is determined, thereby improving the accuracy of determining the test redundancy, leaving a test margin for subsequent result comparison, reducing the probability of misjudgment due to differences in the test process, and further improving the test accuracy of the automated test.

[0142] In some embodiments, based on the test redundancy corresponding to the manual test result information, the automated test result information and the manual test result information are compared, including:

[0143] Determine the second signal test result of the target observation signal from the automated test result information, compare the first signal test result with the second signal test result based on test redundancy, and when the first signal test result is consistent with the second signal test result, determine that the automated test result information is consistent with the manual test result information.

[0144] Among them, test redundancy can characterize the possible difference in results between the first signal test result and the second signal test result due to process differences. Since there is a difference in time between manual testing and automated testing, during the testing process, there may be inevitable process differences in the testing process due to the influence of various non-functional test-related factors. Process differences may cause differences between the first signal test result in manual testing and the second signal test result obtained by automated testing. If the first signal test result and the second signal test result are directly compared, it is also easy to cause misjudgment and reduce the accuracy of automated testing.

[0145] In some embodiments, the test tool can perform information analysis on the automated test result information, and determine the second signal test result of each target observation signal in the automated test result information based on at least one target observation signal of the object under test. For each target observation signal, the test tool can compare the first signal test result of the target observation signal with the second signal test result based on the test redundancy to obtain an accurate comparison result that offsets the process difference. When it is determined based on the accurate comparison result that the first signal test results of all target observation signals are consistent with the corresponding second signal test results, it is determined that the automated test results of the object under test are consistent with the manual test result information.

[0146] In the above embodiment, by obtaining at least one target observation signal for the object under test, and comparing the first signal test result of the target observation signal obtained during manual testing with the second signal test result obtained during automated testing, the comparison result between the automated test result information and the manual test result information of the object under test can be determined. This can effectively reduce the probability of misjudgment in the comparison of result information, thereby improving the test accuracy of the object under test.

[0147] In some optional embodiments, the test tool may compare the first signal test result with the second signal test result based on the test redundancy, so as to obtain an accurate comparison result that offsets the process difference.

[0148] In the above embodiment, the signal test result comparison between the first signal test result and the corresponding second signal test result based on the test redundancy can effectively reduce the probability of misjudgment caused by differences in the test process, and further improve the test accuracy of the automated test. Figure 5 As shown, comparing the first signal test result with the second signal test result based on the test redundancy includes:

[0149] S502: Determine a normal test interval of the target observation signal based on the test redundancy and the first signal test result.

[0150] Among them, the normal test interval is an interval parameter used to offset the process difference between the manual test process and the automated test process, and to determine whether the second signal test result is consistent with the first signal test result. It can be determined in real time by the test tool based on the first signal test result.

[0151] In some optional embodiments, after determining the test redundancy, the test tool may determine a test normal interval of the target observation signal based on the test redundancy and the first signal test result.

[0152] Taking the target observation signal as the temp signal and the test redundancy as 10% as an example, when the first signal test result obtained during manual testing is that the temp signal changes from 1 to 3 in 10s, it can be determined based on the test redundancy that the normal test interval corresponding to the temp signal changing from 1 to 3 is 9s~11s.

[0153] S504: Determine a result difference value between the first signal test result and the second signal test result.

[0154] In some optional embodiments, the testing tool may determine, based on the first signal test result and the second signal test result, a result difference value of the signal test result in a manual test case and in an automated test case.

[0155] S506: When the result difference value belongs to the normal test range, determine that the first signal test result is consistent with the second signal test result.

[0156] In some optional embodiments, the testing tool may compare the result difference value with the test normal interval of the target observation signal. When it is determined that the result difference value belongs to the test normal interval, it can be considered that after excluding the test difference caused by the difference in the test process, the first signal test result is consistent with the second signal test result.

[0157] In some of the embodiments, when it is determined that the result difference value does not belong to the normal test range, it can be considered that even if the test differences caused by the differences in the test process are excluded, the test results obtained by the automated test are still consistent with the test results obtained by the manual test when the test is successful, indicating that the signal test of the object under test does not pass, and the test tool determines that the test of the object under test has failed.

[0158] In the above embodiment, the normal test interval is determined for the target observation signal by testing redundancy and the first signal test result, and the result difference value is then directly compared with the normal test interval, which can effectively improve the signal test result comparison efficiency and accuracy.

[0159] In order to improve the intuitiveness of the comparison, in some embodiments, such as Figure 6As shown, S504, determining a result difference value between the first signal test result and the second signal test result, includes:

[0160] S602, determining first signal change coordinate information of the target observation signal according to the first signal test result, and determining second signal change coordinate information of the target observation signal according to the second signal test result.

[0161] The first signal test result includes each first signal change time and each first signal change value that matches each first signal change time. The second signal test result includes the second signal change time and each second signal change value that matches each second signal change time.

[0162] In some optional embodiments, the testing tool can determine the first signal change coordinate information of the target observation signal based on each first signal change time and each first signal change value in the first signal test result, and the first signal change coordinate information may include multiple coordinate pairs consisting of the first signal change time and the corresponding first signal change value, wherein the first signal change time may be the horizontal coordinate and the first signal change value may be the vertical coordinate, or the first signal change time may be the vertical coordinate and the first signal change value may be the horizontal coordinate.

[0163] At the same time, the testing tool can determine the second signal change coordinate information of the target observation signal based on each second signal change time and each second signal change value in the second signal test result. Similarly, the second signal change coordinate information may include multiple coordinate pairs consisting of the second signal change time and the corresponding second signal change value, wherein the second signal change time can be the horizontal coordinate and the second signal change value can be the vertical coordinate, or the second signal change time can be the vertical coordinate and the second signal change value can be the horizontal coordinate.

[0164] It should be noted that the coordinate attributes of the horizontal and vertical coordinates in the first signal change coordinate information and the second signal change coordinate information are consistent, that is, in the first signal change coordinate information, the first signal change time is the horizontal coordinate and the first signal change value is the vertical coordinate, then correspondingly, in the second signal change information, the second signal change time is the horizontal coordinate and the second signal change value is the vertical coordinate. Conversely, in the first signal change coordinate information, the first signal change time is the vertical coordinate and the first signal change value is the horizontal coordinate, then correspondingly, in the second signal change information, the second signal change time is the vertical coordinate and the second signal change value is the horizontal coordinate.

[0165] S604: Generate a comparison line graph based on the first signal change coordinate information and the second signal change coordinate information.

[0166] In some optional embodiments, the testing tool may generate a comparison line graph based on each first coordinate pair in the first signal change coordinate information and each second coordinate pair in the second signal change coordinate information.

[0167] It can be understood that in some embodiments, when there are multiple target observation signals, a comparison line graph may include the comparison of the first signal test results and the second signal test results of multiple target observation signals. That is, multiple comparison pairs of lines can be generated in a comparison line graph, each comparison pair of lines corresponds to a target observation signal, and each comparison pair of lines is a first line generated based on the first signal change coordinate information of the target observation signal, and a second line generated based on the second signal change coordinate information of the target observation signal.

[0168] In some of the embodiments, in order to reduce the situation where the intuitiveness of the comparison line graph is reduced due to the different signal values ​​of each target observation signal or the large difference in signal change time, the testing tool can classify the target observation signals with similar signal values ​​and small difference in signal change time into a signal set, and only include the comparison of each target observation signal in one signal set on a comparison line graph.

[0169] In some of the embodiments, when there are multiple target observation signals, in order to more clearly and intuitively display the differences between the target observation signals under manual testing and automated testing conditions, a comparison line graph only includes the comparison between the first signal test results and the second signal test results of one target observation signal, that is, only includes one comparison line graph, which includes a first line generated based on the first signal change coordinate information of the target observation signal, and a second line generated based on the second signal change coordinate information of the target observation signal.

[0170] S606: Determine the result difference between the first signal test result and the second signal test result according to the comparison line graph.

[0171] In some embodiments, the measurement tool may determine the result difference value between the first signal test result and the second signal test result of the target observation signal based on the comparison line graph.

[0172] In the above embodiment, by generating a comparison line graph, the first signal test result and the second signal test result of the target observation signal are compared for the difference in results, which effectively improves the comparison efficiency and comparison intuitiveness of the test results, thereby improving the test efficiency of the automated test.

[0173] In some embodiments, there are multiple target observation signals, and each target observation signal and each comparison line Figure 1Correspondingly, the automated testing method further includes: generating a test report for the object under test according to the use case test results of the object under test, the automated testing result information, the test cases, and the comparison line graphs of each target observation signal.

[0174] In some optional embodiments, after determining the use case test results of the object under test, the testing tool can generate a test report for the object under test based on the use case test results of the object under test, automated test result information, test cases, and comparison line charts of each target observation signal, so that the testing personnel or users can clearly understand the overall test status of the object under test.

[0175] In the above embodiment, when generating a test report of the object under test, in order to improve the intuitiveness of the information display of the test report, each target observation signal and each comparison line Figure 1 One-to-one correspondence, that is, a comparison line graph will only contain the comparison between the first signal test result and the second signal test result of a target observation signal. This can reduce the situation where the intuitiveness of the comparison line graph is reduced due to large differences in the test results of each target observation signal, and effectively improve the intuitiveness of the information display of the test report.

[0176] In some embodiments, Figure 7 As shown, S402, obtaining at least one target observation signal for the object under test includes:

[0177] S702, loading the signal database file in the test system.

[0178] Among them, the signal database file is a database file that stores test signals in the test system. Taking the test system as a CANoe system as an example, the signal database file can be a dbc (Database CAN) file loaded by the CANoe system, which can be used to define various elements in the CAN network, including nodes, messages and signals.

[0179] In some optional embodiments, the test tool may load the signal database file during the test based on a pre-set communication interface and protocol.

[0180] S704, generating a signal selection page based on the signal database file.

[0181] The signal selection page may include each test signal in the signal database file and a selection control configured for each test signal.

[0182] In some optional embodiments, the test tool may generate a signal selection page based on each test signal included in the recorded signal database file, and display the signal selection page to the tester.

[0183] S706 , in response to a signal selection operation triggered on a signal selection page, obtaining at least one target observation signal for the object under test.

[0184] In some optional embodiments, the test personnel may perform a signal selection operation on each test signal contained in the signal selection page based on the signal selection page generated by the test tool, that is, select at least one target observation signal to be observed from each test signal, and perform a trigger operation on the selection control of the target observation signal, such as clicking or staying, etc. The test tool obtains at least one target observation signal for the object under test in response to the signal selection operation triggered by the test personnel on the signal selection page.

[0185] In the above embodiment, by loading the signal database file of the test system and generating a signal selection page based on the signal database file, it is convenient for detection personnel to select the corresponding target observation signal from a large number of signals, thereby improving the accuracy and convenience of selecting the target observation signal.

[0186] In order to quickly obtain the automated test result information, the test tool needs to accurately determine the stop time of the automated test. Based on this, in some embodiments, such as Figure 8 As shown, S204, calling the test system to perform automated testing on the object under test according to the test recording file, and obtaining automated test result information, including:

[0187] S802: Determine an estimated duration of the automated test of the object under test according to the recording duration of the test recording file.

[0188] In some optional embodiments, the test tool can determine the file recording duration of the test recording file according to the file configuration information of the test recording file, and the file recording duration can also be considered as the manual test duration of the test case corresponding to the test recording file. The test tool determines the estimated duration of the automated test of the object under test based on the file recording duration of the test recording file.

[0189] In some of the embodiments, when there is only one test case, that is, only one test recording file, the testing tool can directly determine the file recording duration of the test recording file as the estimated duration of the automated test of the object under test.

[0190] S804, calling the test system to perform automated testing on the object under test according to the test recording file, and obtaining the test duration of the automated testing in real time.

[0191] In some optional embodiments, the testing tool may call the testing system to perform automated testing on the object under test according to the test recording file, and obtain the test duration of the automated testing in real time.

[0192] S806: When the test duration reaches the expected duration of the automated test, determine that the test system meets the automated test end condition.

[0193] In some optional embodiments, the testing tool can compare the test duration of the automated test obtained in real time with the expected duration of the automated test of the object under test. When the test duration reaches the expected duration of the automated test, it can be determined that the test system meets the automated test result conditions and the test system has completed the automated test of the object under test.

[0194] S808, obtaining automated test result information of the object under test based on the test system.

[0195] In some optional embodiments, when the test tool determines that the test system meets the automated test termination condition, it can obtain automated test result information of the object under test based on the test system.

[0196] In the above embodiment, the testing tool can obtain the enabling conditions for stopping the automated testing of the testing system by determining the estimated duration of the automated testing of the object under test. According to the enabling conditions for stopping the automated testing, the timing for stopping the automated testing of the testing system can be accurately determined, thereby increasing the speed of obtaining the automated testing result information of the object under test from the testing system and reducing the duration of the automated testing.

[0197] In addition to the case where the number of test recording files is one, in other embodiments, such as Fig. 9 As shown, there are multiple test recording files. S802, according to the recording time of the test recording files, determining the estimated duration of the automated test of the object under test, including:

[0198] S902, counting the recording time of each test recording file to obtain the theoretical test estimated time of the object under test.

[0199] In some optional embodiments, the test tool can count the file recording time of each test recording file to obtain the theoretical test estimated time of the object under test. For example, the test tool can directly determine the sum of the file recording time of each test recording file as the theoretical test estimated time of the object under test.

[0200] S904, based on the theoretical test estimated duration and the preset correspondence between the test estimated duration and the duration redundancy, determine the duration redundancy that matches the theoretical test estimated duration.

[0201] Among them, in order to reduce the difference between the running time of the test tool and the running time of the test system, the test personnel can add a redundant time, that is, the duration redundancy, to the theoretical test estimated duration of the object under test. The test personnel can determine the optimal duration redundancy corresponding to each test estimated duration based on experimental data or empirical data, bind the test estimated duration with the duration redundancy, generate a preset correspondence between the test estimated duration and the duration redundancy, and pre-configure the preset correspondence between the test estimated duration and the duration redundancy in the test tool so that the test tool can be used at any time.

[0202] In some optional embodiments, after obtaining the theoretical estimated test duration of the object under test, the testing tool can call the preset correspondence between the estimated test duration and the duration redundancy, and search for the duration redundancy that matches the theoretical estimated test duration from the preset correspondence.

[0203] S906, determining the estimated duration of the automated test of the object under test according to the estimated duration of the theoretical test and the duration redundancy.

[0204] In some optional embodiments, after obtaining the theoretical test estimated duration and duration redundancy, the test tool can determine the estimated duration of the automated test of the object under test according to the theoretical test estimated duration and duration redundancy. For example, the test tool can determine the sum of the theoretical test estimated duration and duration redundancy as the estimated duration of the automated test of the object under test.

[0205] In the above embodiment, when there are multiple test recording files, the theoretical test estimated duration is obtained by counting the file recording duration of each test recording file, and the matching duration redundancy is determined for the theoretical test estimated duration. This can effectively reduce the difference between the running time of the test tool and the running time of the test system, which leads to the deviation of the theoretical test estimated duration, effectively improve the accuracy of the automated test estimated duration, and thus improve the speed of obtaining the automated test result information of the object under test from the test system, thereby reducing the automated test duration.

[0206] In some embodiments, an automated testing method is provided, which is applied to Figure 1 The application scenario shown is used as an example for explanation, wherein the test tool may be developed through programming language software, and the test software may be CANoe software.

[0207] like Fig.10 As shown, the method may specifically include the following steps:

[0208] S1001, the test software responds to the manual test performed on each test case and obtains the test recording file and manual test result information of each test case.

[0209] In some optional embodiments, for each test case, the inspector can open the test project based on CANoe software, select the Automation module in Simulation, open the Macros function, select the newly created Macros file, name it with the ID of the test case to be executed, and store the Macros file in a specific folder. The inspector checks the "Start macro recording immediately" button and clicks "OK" to record the macro. Then run CANoe and click the start button of Macros. The inspector can set the relevant system variables on CANoe based on the detailed steps of the test case to perform the first manual execution test. After the test is completed, the inspector clicks the stop button of Macros to close the Macros recording.

[0210] The inspector judges the manual test results based on the expected results in the test case. If the test passes, it can be verified that the function of the object under test is normal. The inspector will change the name of the manual test log file to the executed test case ID and store it in a specific folder. If the test fails, it can be verified that the function of the object under test is abnormal. The inspector will abandon the manual test log file based on the CANoe system.

[0211] Based on the above steps, all test cases are manually tested until all test cases are manually tested successfully, and the test recording files and manual test result information of each test case are obtained.

[0212] S1002, the testing tool obtains the test recording file and manual test result information of each test case.

[0213] In some optional embodiments, the inspector can open a test tool developed based on a programming language software, select to add the test recording file to be executed automatically, that is, the Macros file of macro recording, and select the manual test result information corresponding to the test recording file, that is, the Log folder to be compared, and then click the "OK" button to upload the test recording file and manual test result information of each test case to the test tool.

[0214] S1003, the testing tool updates the system configuration file of the testing system according to the file storage address of each test recording file to obtain an updated system configuration file.

[0215] In some optional embodiments, the testing tool can change the cfg file of CANoe, and load each test recording file to be automatically executed into cfg in sequence according to the file storage address of each test recording file, so as to control CANoe to perform sequential batch execution of the test recording files recorded during manual testing.

[0216] S1004, the test tool loads the signal library file of the test system, and determines at least one target observation signal of the object under test based on the signal database file.

[0217] In some optional embodiments, the inspector can load the dbc file loaded in the CANoe system through the test tool. The test tool will generate and display a signal selection page based on the loaded dbc file. The inspector will check the target observation signal to be observed based on the signal selection page to determine at least one target observation signal of the object under test, so that after running the automated test, the target observation signal selected in the log files recorded during the manual test and the automated test can be compared.

[0218] S1005, respectively setting the time and value redundancy of each target observation signal to obtain a normal test interval for each target observation signal.

[0219] In some optional embodiments, the test tool will set the redundancy of time and value for each selected target observation signal, and obtain the normal test interval of each target observation signal. It can be understood that the redundancy of time and value is set based on the manual test result, that is, the first signal test result of the target observation signal. For example, when performing a manual test, the Temp signal changes from 1 to 3 in 10s. If the time redundancy of this signal is 10%, then during the automated test, the time for the Temp signal to change to 3 can be within 9s~11s, that is, the normal time test interval of the temp signal is 9s~11s. If the time for the temp signal to change from 1 to 3 during the automated test is within 9s~11s, then the test result of the temp signal can be considered as a pass. If the time for the temp signal to change from 1 to 3 during the automated test is outside 9s~11s, then the test result of the temp signal can be considered as a fail.

[0220] S1006: Determine the estimated duration of the automated test of the object under test according to the recording duration of each test recording file.

[0221] In some optional embodiments, the test tool can determine the estimated duration of the automated test of the object under test based on the recording duration of each test recording file. The test tool can calculate the time of each macro recorded Macros, and sum up all the times of the selected Macros to obtain the estimated duration of the automated test of the object under test, and use the estimated duration of the automated test as an enabling condition for stopping the automated test.

[0222] In other embodiments, in order to reduce the probability that the estimated duration of the automated test is inaccurate due to the difference between the running time of the test tool and the running time of the CANoe system, the test tool may add a redundant time to the summed estimated duration of the automated test.

[0223] S1007, calling the test system to perform automated testing on the object under test according to each test recording file.

[0224] In some optional embodiments, the tester clicks the automated test run button on the test tool, and the test tool can call the test system CANoe to control the start and operation of the CANoe system.

[0225] S1008, obtaining the test duration of the automated test.

[0226] In some optional embodiments, when the testing tool calls the testing system to perform automated testing on the object under test, it can obtain the test duration of the automated testing in real time.

[0227] S1009, determine whether the automated test is completed. If completed, execute S1010, if not completed, return to execute S1008.

[0228] In some optional embodiments, the testing tool can determine whether the test system meets the conditions for the end of the automated test based on the test duration and the expected duration of the automated test. If the test duration reaches the expected duration of the automated test, it is determined that the test system meets the conditions for the end of the automated test and the automated test is completed. If the test duration does not reach the expected duration of the automated test, it is determined that the test system does not meet the conditions for the end of the automated test and the automated test is not completed.

[0229] S1010, obtaining automated test result information of the object under test based on the test system.

[0230] In some optional embodiments, during the running of the automated test, CANoe will name the automatically generated Log file according to the name of each test recording file being run, and store it in a fixed folder. When the automated running is completed, that is, after all the test recording files selected to be executed are completed, the CANoe system will obtain a folder containing multiple Log files, that is, the automated test result information of the object being tested. The test tool can obtain the automated test result information of the object being tested based on CANoe.

[0231] S1011, compare the automated test result information with the manual test result information to obtain the test result of the test object.

[0232] In some optional embodiments, after obtaining the automated test result information, the test tool can compare the manual test result information with the automated test result information, that is, compare the manual test log file with the automated test log file, compare them in sequence according to the same file name, parse the manually recorded log and the automatically recorded log, and form an XY axis line graph based on time for each signal that needs to be compared, with the Y axis being a time-based value. Each XY axis line graph only represents one target observation signal, so each XY axis line graph only has two curves, which respectively represent the manual test result and the automated test result of the target observation signal. As many XY axis line graphs need to be generated as there are target observation signals that need to be compared in a test case.

[0233] The test tool will compare the manual test results based on the redundancy of time and values. If the difference between the manual test results and the automated test results falls within the normal test range, the XY axis line graph test is determined to have passed. If the difference between the manual test results and the automated test results does not fall within the normal test range, the XY axis line graph test is determined to have failed. For each test case, if all XY axis line graphs in the test case are judged to have passed the test, it can be determined that the automated test of the test case has passed. If there is an XY axis line graph in the test case that is judged to have failed the test, it is determined that the automated test of the test case has failed.

[0234] S1012, generating a test report for the object under test.

[0235] In some optional embodiments, the testing tool can automatically generate a test report based on the automated testing results, and the test report includes all the run test cases, automated testing results, and an XY axis line chart generated in the previous step.

[0236] In the above embodiment, first, after the manual test is passed by manually operating the system variables of CANoe, there is no need to manually develop test scripts and debug the scripts for the test cases, and the automatic test can be performed through the present invention, which improves the test efficiency and shortens the test cycle. Second, the test tool parses the Log file recorded by the CANoe software, and parses the database of the signal to the time axis from the Log file, which is used for automatic comparison of the results of manual test and automated test, and improves the accuracy of result determination. Third, after the test tool imports the dbc communication protocol and the CANoe system variable file, it automatically displays the signals in the dbc communication protocol and the variables of the CANoe system, and by checking it, it is determined whether to compare it later, which can be used to delete some signals that do not need to be compared or cannot be compared. Even under the same test conditions, the Y-axis values ​​of such signals at the same time point on the X-axis (time axis) are different each time, so as to reduce the misjudgment caused by the comparison of the test log data and improve the test pass rate. Fourth, through the redundancy setting, the test tool can set the deviation threshold value or percentage of the signals in the imported dbc communication protocol and CANoe system variable file, reduce the misjudgment caused by the comparison of the test log data, and improve the test pass rate.

[0237] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0238] Based on the same inventive concept, the embodiment of the present application also provides an automated testing device for implementing the automated testing method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more automated testing device embodiments provided below can refer to the limitations on the automated testing method above, and will not be repeated here.

[0239] In some embodiments, Fig.11 As shown, an automated testing device 1100 is provided, comprising: an information acquisition module 1101, an automated testing module 1102 and a result comparison module 1103, wherein:

[0240] The information acquisition module 1101 is used to acquire a test recording file of at least one test case and manual test result information of the test case. The test recording file is a macro file recorded by the test system when the test case is manually tested and the test is successful.

[0241] The automated testing module 1102 is used to call the testing system to perform automated testing on the object under test according to the test recording file, and obtain automated testing result information.

[0242] The result comparison module 1103 is used to compare the automated test result information with the manual test result information based on the test redundancy corresponding to the manual test result information; if the automated test result information is consistent with the manual test result information, the use case test result of the object under test is determined to be a passed test.

[0243] In some embodiments, the automated testing module 1102 is used to: obtain a system configuration file of the test system; update the system configuration file information according to the file storage address of the test recording file to obtain an updated system configuration file; based on the updated system configuration file, call the test system to perform automated testing on the object under test according to the test recording file to obtain automated testing result information.

[0244] In some embodiments, the number of test cases is multiple. The automated test module 1102 is also used to: obtain test sequence information of each test case, determine the running sequence of each test recording file; based on the running sequence of each test recording file, write the storage address of each test recording file into the system configuration file to obtain an updated system configuration file.

[0245] In some embodiments, the automated testing apparatus further comprises:

[0246] The target observation signal acquisition module is used to acquire at least one target observation signal for the object under test.

[0247] The signal test result determination module is used to determine the first signal test result of the target observation signal from the manual test result information.

[0248] A test redundancy determination module is used to determine the test redundancy of the target observation signal according to the first signal test result.

[0249] In some embodiments, the signal test result determination module is further used to determine the second signal test result of the target observation signal from the automated test result information. The test result comparison module is used to: compare the first signal test result with the second signal test result based on the test redundancy. When the first signal test result is consistent with the second signal test result, it is determined that the automated test result information is consistent with the manual test result information.

[0250] In some embodiments, the test result comparison module is also used to: determine the test normal interval of the target observation signal based on the test redundancy and the first signal test result; determine the result difference value between the first signal test result and the second signal test result; and when the result difference value belongs to the test normal interval, determine that the first signal test result is consistent with the second signal test result.

[0251] In some embodiments, the test result comparison module is also used to: determine the first signal change coordinate information of the target observation signal based on the first signal test result, and determine the second signal change coordinate information of the target observation signal based on the second signal test result; generate a comparison line graph based on the first signal change coordinate information and the second signal change coordinate information; determine the result difference value between the first signal test result and the second signal test result based on the comparison line graph.

[0252] In some embodiments, there are multiple target observation signals, and each target observation signal and each comparison line Figure 1 One to one. The automated testing device also includes:

[0253] The test report generation module is used to generate a test report for the object under test based on the use case test results of the object under test, the automated test result information, the test cases, and the comparison line graphs of each target observation signal.

[0254] In some embodiments, the target observation signal acquisition module is used to: load the signal database file in the test system; generate a signal selection page based on the signal database file; and acquire at least one target observation signal for the object under test in response to a signal selection operation triggered on the signal selection page.

[0255] In some embodiments, the automated testing module 1102 is used to: determine the estimated duration of automated testing of the object under test based on the file recording duration of the test recording file; call the test system to perform automated testing on the object under test based on the test recording file, and obtain the test duration of the automated testing in real time; when the test duration reaches the estimated duration of the automated testing, determine that the test system meets the conditions for ending the automated testing; and obtain automated testing result information of the object under test based on the test system.

[0256] In some embodiments, there are multiple test recording files. The automated test module 1102 is also used to: count the file recording time of each test recording file to obtain the theoretical test estimated time of the object under test; determine the time redundancy that matches the theoretical test estimated time based on the theoretical test estimated time and the preset corresponding relationship between the test estimated time and the time redundancy; determine the automated test estimated time of the object under test according to the theoretical test estimated time and the time redundancy.

[0257] Each module in the above-mentioned automated testing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0258] In some embodiments, a computer device is provided, which may be a test tool, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store test recording files, manual test result information, automated test result information, case test results and other data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an automated testing method is implemented.

[0259] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0260] In some embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, specific implementation steps of the above-mentioned automated testing method embodiment are implemented.

[0261] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the specific implementation steps of the above-mentioned automated testing method embodiment are implemented.

[0262] In some embodiments, a computer program product is provided, including a computer program, which, when executed by a processor, implements the specific implementation steps of the above-mentioned automated testing method embodiment.

[0263] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. In addition, the acquisition, storage, processing, and transmission of data are in compliance with relevant laws and regulations.

[0264] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0265] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0266] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An automated testing method, characterized in that: The method comprises: Acquire a test recording file of at least one test case and manual test result information of the test case, wherein the test recording file is a macro file recorded by the test system when the test case is manually tested and the test succeeds; Obtaining a system configuration file of the test system, and updating information of the system configuration file according to a file storage address of the test recording file to obtain an updated system configuration file; Based on the updated system configuration file, calling the test system to perform automated testing on the object under test according to the test recording file to obtain automated testing result information; Acquire at least one target observation signal for the object under test; Determine a first signal test result of the target observation signal from the manual test result information; Determining the test redundancy of the target observation signal according to the first signal test result; Determine a second signal test result of the target observation signal from the automated test result information; Comparing the first signal test result with the second signal test result based on the test redundancy; When the first signal test result is consistent with the second signal test result, determining that the automated test result information is consistent with the manual test result information; When the automated test result information is consistent with the manual test result information, it is determined that the test result of the use case of the object under test is a test pass.

2. The method according to claim 1, characterized in that The number of the test cases is multiple; the system configuration file is updated according to the file storage address of the test recording file to obtain an updated system configuration file, including: Obtaining the test sequence information of each of the test cases, and determining the running sequence of each of the test recording files; Based on the running order of each of the test recording files, the storage address of each of the test recording files is written into the system configuration file to obtain an updated system configuration file.

3. The method according to claim 1, characterized in that The comparing the first signal test result with the second signal test result based on the test redundancy includes: Determining a normal test interval of the target observation signal based on the test redundancy and the first signal test result; Determine a result difference value between the first signal test result and the second signal test result; When the result difference value belongs to the normal test interval, it is determined that the first signal test result is consistent with the second signal test result.

4. The method according to claim 3, characterized in that The determining a result difference value between the first signal test result and the second signal test result includes: Determine first signal change coordinate information of the target observation signal according to the first signal test result, and determine second signal change coordinate information of the target observation signal according to the second signal test result; Generate a comparison line graph based on the first signal change coordinate information and the second signal change coordinate information; The result difference value between the first signal test result and the second signal test result is determined according to the comparison line graph.

5. The method according to claim 4, characterized in that There are multiple target observation signals, each of which corresponds to each comparison line graph. The method further includes: A test report for the object under test is generated based on the use case test results of the object under test, the automated test result information, the test cases, and the comparison line graphs of each of the target observation signals.

6. The method according to claim 1, characterized in that The acquiring of at least one target observation signal for the object under test comprises: Loading a signal database file in the test system; generating a signal selection page based on the signal database file; In response to a signal selection operation triggered on the signal selection page, at least one target observation signal for the object under test is acquired.

7. The method according to claim 1 or 2, characterized in that: The calling of the test system to perform an automated test on the object under test according to the test recording file to obtain automated test result information includes: Determining the estimated duration of the automated test of the object under test according to the file recording duration of the test recording file; Calling the test system to perform automated testing on the object under test according to the test recording file, and obtaining the test duration of the automated test in real time; When the test duration reaches the expected duration of the automated test, determining that the test system meets the automated test end condition; The automated test result information of the tested object is obtained based on the test system.

8. The method according to claim 7, characterized in that The number of the test recording files is multiple; and determining the estimated duration of the automated test of the object under test according to the recording duration of the test recording files includes: Counting the recording time of each test recording file to obtain the theoretical estimated test time of the object under test; Based on the theoretical test estimated duration and the preset correspondence between the test estimated duration and the duration redundancy, determining the duration redundancy that matches the theoretical test estimated duration; The estimated duration of the automated test of the object under test is determined based on the estimated duration of the theoretical test and the duration redundancy.

9. An automated testing device, characterized in that: The device comprises: An information acquisition module, used to acquire a test recording file of at least one test case and manual test result information of the test case, wherein the test recording file is a macro file recorded by the test system when the test case is manually tested and the test succeeds; The automated testing module is used to obtain a system configuration file of the test system, update the system configuration file according to the file storage address of the test recording file, and obtain an updated system configuration file; based on the updated system configuration file, call the test system to perform automated testing on the object under test according to the test recording file to obtain automated testing result information; A target observation signal acquisition module, used to acquire at least one target observation signal for the object under test; A signal test result determination module, used to determine a first signal test result of the target observation signal from the manual test result information; A test redundancy determination module, used to determine the test redundancy of the target observation signal according to the first signal test result; The signal test result determination module is further used to determine a second signal test result of the target observation signal from the automated test result information; A test result comparison module is used to compare the first signal test result with the second signal test result based on the test redundancy; when the first signal test result is consistent with the second signal test result, it is determined that the automated test result information is consistent with the manual test result information; when the automated test result information is consistent with the manual test result information, it is determined that the use case test result of the object under test is a passed test.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images