A method and system for testing the adaptability of a complete machine

Abstract

The present invention provides a method and system for testing the adaptability of an ICT system, and relates to the technical field of ICT testing, including: obtaining a plurality of test tools for testing an ICT system, and obtaining a tool standard table; then obtaining the best tool for each test item; then establishing an automated test model, and establishing an automated test method for the ICT system; finally, obtaining the test bias for each test item based on the test data, and adjusting the standard test results obtained by the automated test method based on the test bias; the present invention is used to solve the problems in the prior art of low efficiency in integrating the adaptability test method for the ICT system and comprehensive testing, as well as unstable test results during multiple tests, thereby affecting the overall evaluation of the ICT system.

Description

Technical Field

[0001] The present invention relates to the field of ICT testing technology, and in particular to a ICT whole machine adaptability testing method and testing system. Background Art

[0002] Info-innovation refers to innovation in information technology applications. The Info-innovation industry mainly includes four major areas: infrastructure, including chips, storage, servers and cloud computing; basic software, including operating systems, databases and middleware; application software, including basic office software, enterprise management software, industrial software and process optimization software; information security, including security software, security hardware and security services; adaptation testing of Info-innovation complete machines refers to comprehensive verification of the functions, performance and security of software and hardware products and application systems in a Info-innovation environment to ensure that they can operate normally in the Info-innovation environment and meet user needs and expectations.

[0003] Existing improvements in the adaptability testing of trusted computing machines are usually improvements in the real-time computing framework. For example, in a Chinese patent application with publication number CN116701173A, a method for adaptability testing of a real-time computing framework based on a trusted computing environment is disclosed. The solution is to build a real-time computing framework in different domestic environments, perform adaptation judgment on the real-time computing framework based on the hardware environment, and obtain a first judgment result; construct a test case, input the test case into the real-time computing framework built in different domestic environments for testing, and obtain a second test result; based on the first judgment result and the second test result, obtain the adaptation result of the real-time computing framework. Other improvements in the adaptability testing of trusted computing machines are usually based on machine learning to evaluate the adaptability of the trusted computing system. However, the existing technology has the problems of low efficiency in integrating the adaptability testing methods and comprehensive testing of trusted computing machines, as well as unstable test results during multiple tests, resulting in the inability to quickly and effectively obtain the test results of different test items in the trusted computing machine, thereby affecting the overall evaluation of the trusted computing machine. In view of this, it is necessary to improve the existing adaptability testing of trusted computing machines. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a credible innovation machine adaptability testing method and testing system, which are used to solve the problems in the prior art of low efficiency in integrating the credible innovation machine adaptability testing method and comprehensive testing, as well as unstable test results during multiple tests, resulting in the inability to quickly and effectively obtain the test results of different test items in the credible innovation machine, thereby affecting the overall evaluation of the credible innovation machine.

[0005] In order to achieve the above-mentioned object, in a first aspect, the present invention provides a method for testing the adaptability of a complete machine, comprising:

[0006] Obtain multiple test tools for testing the whole device, and obtain a tool standard table based on the test results of the multiple test tools;

[0007] Get the best tool for each test item in the whole machine based on the tool standard table;

[0008] Establish an automated test model based on the tool standard table and test tools, and use the automated test model to establish an automated test method for the entire Xinchuang machine, and obtain standard test results based on the automated test method;

[0009] Obtain all test data after multiple tests on the entire Xinchuang machine, and obtain the test weight of each test item based on all test data, and adjust the standard test results obtained through automated testing methods based on the test weight.

[0010] Furthermore, a plurality of test tools for testing the whole device are obtained, and a tool standard table is obtained based on the test results of the plurality of test tools, including:

[0011] Obtain multiple test items that can be tested in the Xinchuang whole machine, which are recorded as test items CX1 to CXn, where n is the number of test items;

[0012] Acquire multiple test tools that can test multiple test items in the integrated circuit, and record them as test tools CG1 to test tools CGm, where m is the number of test tools; for any test tool CGm1 among test tools CG1 to CGm, record the items among test items CX1 to CXn that can be tested using test tool CGm1 as testable items of test tool CGm1; obtain the testable items of all test tools, where 1≤m1≤m, and m and m1 are positive integers.

[0013] Furthermore, obtaining multiple test tools for testing the whole device of the ICT system, and obtaining a tool standard table based on the test results of the multiple test tools also includes:

[0014] For any test tool CGm1 from test tool CG1 to test tool CGm, use test tool CGm1 to test the testable items of test tool CGm1 in the whole Xinchuang machine under standard conditions, and record the test results in a standard test table, wherein the standard test table is a table of t rows × y columns, and the top row of the standard test table is filled with test items CX1 to test items CXn from left to right except the leftmost cell, and the leftmost column of the standard test table is filled with test tool CGm1 in the second cell from top to bottom;

[0015] Based on the test method of using the test tool CGm1, all the test tools are used to test the testable items of the Xinchuang whole machine respectively, and all the obtained standard test tables are recorded as standard test table 1 to standard test table q respectively; a tool standard table is established, wherein the tool standard table is a table of m+1 rows×t columns, and the top row of the tool standard table is filled with test items CX1 to test items CXn from left to right except the leftmost cell, and the leftmost column of the tool standard table is filled with test tools CG1 to test tools CGm from top to bottom except the first cell, m=q;

[0016] Based on standard test tables 1 to q, fill in corresponding data in the tool standard table, where when there is no test result corresponding to any test tool in the column where any test item CXn1 in the tool test table is located, add a new row to the tool test table and fill in the leftmost cell of the new row with the test tool that can test the test item CXn1, 1≤n1≤n, n1 and n are positive integers.

[0017] Furthermore, based on the tool standard table, the best tools for obtaining each test item in the Xinchuang machine include:

[0018] For any test item CXn1 among the test items CX1 to CXn, the test tools in the tool standard table that can test the test item CXn1 are recorded as testable tools KCG1 to testable tools KCGs, where s is the number of testable tools; other test items initiated when testing the test item CXn1 are recorded as associated items n1~1 to associated items n1~k, where k is the number of associated items; s and k are positive integers;

[0019] When the number of the associated projects is 0, the testable tools KCG1 to KCGs are respectively obtained, which have the shortest test time for testing the test project CXn1, and are recorded as the best tools for the test project CXn1;

[0020] When the number of associated projects is greater than 0, a combination method of multiple test tools that can test the test project CXn1 and all associated projects is obtained based on the tool standard table, and they are recorded as test tool combination CZ1 to test tool combination CZg, respectively, where g is the number of test tool combinations, wherein all test tools in any test tool combination can test the test project CXn1 and all associated projects;

[0021] Based on the tool efficiency algorithm, the efficiency value of each test tool combination when testing the test item CXn1 and all the associated items is obtained, and all the tools in the test tool combination with the smallest efficiency value are recorded as the best tools for the test item CXn1; the tool efficiency algorithm includes: Where F is the efficiency value, L is the number of test tools in the test tool combination, L′ is the time used to test the test item CXn1 using the test tool combination, Pi is the time consumed by the i-th test tool in the test tool combination during the test process, and Rmax is the maximum memory occupied during the test using the test tool combination; i≤L;

[0022] Based on the analysis method of test item CXn1, the best tools corresponding to all test items are obtained.

[0023] Furthermore, an automated test model is established based on the tool standard table and the test tool, and an automated test method for the whole ICT system is established using the automated test model. Obtaining standard test results based on the automated test method includes:

[0024] Establish an automated test model and store the tool standard table and all test tools in the automated test model;

[0025] In the automated testing model, an automated testing method is established based on the best tools for all test items, and standard test results are obtained based on the automated testing method.

[0026] Furthermore, the automated testing method comprises:

[0027] When testing the whole Xinchuang machine, the items to be tested are recorded as items to be tested DCX1 to items to be tested DCXd, where d is the number of items to be tested;

[0028] When the best tool for any one of the items DCX1 to DCXd to be tested is able to test the items DCX1 to DCXd to be tested, d1 is any number from 1 to d, the best tool for the item DCXd1 to be tested is used to test the items DCX1 to DCXd to be tested, and the test results are recorded as the standard test results of the items DCX1 to DCXd to be tested.

[0029] Furthermore, the automated testing method further comprises:

[0030] When the best tools for all the items to be tested cannot test the items to be tested DCX1 to DCXd, the best tools for the items to be tested DCX1 to DCXd are used to test them respectively;

[0031] For any one of the items to be tested DCXd1 to the items to be tested DCXd, when the best tool of the items to be tested other than the item to be tested DCXd1 can test the item to be tested DCXd1, the test result of the item to be tested DCXd1 using the best tool of the item to be tested DCXd1 is recorded as the primary test result of the item to be tested DCXd1, and the test result of the item to be tested DCXd1 using the best tool of the items to be tested other than the item to be tested DCXd1 is recorded as the secondary test result, and the standard test result of the item to be tested is obtained using the primary and secondary combination algorithm, and the primary and secondary combination algorithm includes:

[0032]

[0033] Where A is the standard test result, A1 is the primary test result, U is the number of secondary test results, and Aj is the jth secondary test result;

[0034] When the best tools for all the items to be tested except the item to be tested DCXd1 cannot test the item to be tested DCXd1, the test result of the item to be tested DCXd1 using the best tool for the item to be tested DCXd1 is recorded as the standard test result.

[0035] Furthermore, all test data after multiple tests on the whole Xinchuang machine are obtained, and the test weight of each test item is obtained based on all the test data. The standard test results obtained by the automated testing method are adjusted based on the test weight, including:

[0036] Use an automated testing method to test the whole Xinchuang machine multiple times. For any test item CXn1 from test items CX1 to test items CXn, record multiple standard test results of test item CXn1 during the multiple tests as reference results KC1 to reference results KCc, where c is the number of times the test is performed;

[0037] Establish a plane rectangular coordinate system, recorded as a biased analysis coordinate system, wherein the coordinates from the origin of the X-axis of the biased analysis coordinate system to the right are the reference results KC1 to the reference results KCc, and the unit of the Y-axis is the unit of the data corresponding to the test item CXn1;

[0038] Based on the reference results KC1 to the reference results KCc, marks are made in the bias analysis coordinate system, which are recorded as bias reference points PZ1 to bias reference points PZc respectively; the best tool for test item CXn1 is used to test test item CXn1, and the test result of test item CXn1 using the best tool for test item CXn1 is recorded as the control result, and the vertical coordinates of bias reference points PZ1 to bias reference points PZc minus the control result are recorded as bias difference values ​​PC1 to bias difference values ​​PCc respectively, and when all bias difference values ​​are 0, the test bias of test item CXn1 is recorded as no bias;

[0039] When the number of positive numbers in all bias differences is greater than the number of negative numbers, the test bias of test item CXn1 is recorded as positive bias, and the average of the positive numbers in all bias differences is recorded as the bias mean;

[0040] When the number of positive numbers in all bias differences is less than the number of negative numbers, the test bias of test item CXn1 is recorded as negative bias, and the average of the negative numbers in all bias differences is recorded as the bias mean;

[0041] When the number of positive numbers in all bias differences is equal to the number of negative numbers, the test bias of test item CXn1 is recorded as uniform bias, and the average value of all bias differences is recorded as bias mean.

[0042] Further, all test data after multiple tests on the whole Xinchuang machine are obtained, and the test weight of each test item is obtained based on all the test data, and the standard test results obtained by the automated testing method are adjusted based on the test weight, which also includes:

[0043] Based on the analysis method of test item CXn1, the test weight of all test items is obtained;

[0044] For any test item, when the test bias of the test item is no bias, after the standard test result of the test item is obtained by using the automated test method, the standard test result of the test item is not adjusted;

[0045] When the test bias of the test item is not unbiased, after obtaining the standard test result of the test item using the automated test method, the value of the standard test result is adjusted to a value obtained by adding the standard test result to the bias mean of the test item.

[0046] In a second aspect, the present invention provides a whole-machine adaptability testing system for a trusted computing system, including a standard table acquisition module, an optimal tool analysis module, a method establishment module, and a comprehensive testing module;

[0047] The standard table acquisition module is used to acquire multiple test tools for testing the whole Xinchuang machine, and acquire the tool standard table based on the test results of multiple test tools;

[0048] The best tool analysis module is used to obtain the best tool for each test item in the Xinchuang machine based on the tool standard table;

[0049] The method establishment module is used to establish an automated test model based on the tool standard table and the test tool, and use the automated test model to establish an automated test method for the entire Xinchuang machine, and obtain standard test results based on the automated test method;

[0050] The comprehensive testing module is used to obtain all test data after multiple tests on the entire Xinchuang machine, and obtain the test weight of each test item based on all test data, and adjust the standard test results obtained through automated testing methods based on the test weight.

[0051] The beneficial effects of the present invention are as follows: first, a plurality of test tools for testing the ICT whole machine are obtained, and a tool standard table is obtained based on the test results of the plurality of test tools; then, the best tool for each test item in the ICT whole machine is obtained based on the tool standard table. The advantage of this is that, by obtaining the tool standard table, it is helpful to be more convenient and quick in the subsequent selection of test tools for testing the ICT whole machine, thereby improving the test efficiency; by obtaining the best tool for each test item, the test tool that is most suitable for the test item can be used preferentially in subsequent tests, so that the test results are more in line with the actual conditions of the test items, and thus the test results of each test item are more stable;

[0052] The present invention also establishes an automated testing model based on the tool standard table and the test tool, and uses the automated testing model to establish an automated testing method for the ICT system, and obtains standard test results based on the automated testing method; finally, all test data after multiple tests on the ICT system are obtained, and the test bias of each test item is obtained based on all the test data, and the standard test results obtained by the automated testing method are adjusted based on the test bias. The advantage of this is that by establishing an automated testing model and constructing an automated testing method in the automated testing model, the ICT system can be tested more intelligently and efficiently and with reduced labor costs. At the same time, the standard test results obtained using the automated testing method can combine the advantages of multiple testing methods to improve the accuracy of the test results; by obtaining the test bias, it is helpful to obtain the test error of the automated testing method, thereby adjusting the test results based on the test error and reducing the error of the actual test results.

[0053] Advantages of additional aspects of the present invention will be given in part in the description of the following specific embodiments, and in part will become apparent from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0055] Figure 1 A flow chart showing the steps and principles of a method for testing the adaptability of a complete machine of a trusted computing system according to the present invention;

[0056] Figure 2 It is a principle block diagram of a Xinchuang whole machine adaptability testing system of the present invention;

[0057] Figure 3 It is a schematic diagram of the biased analysis coordinate system of the present invention. DETAILED DESCRIPTION

[0058] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0059] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0060] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0061] Example 1

[0062] See also Figure 1 As shown, the present invention provides a method for testing the adaptability of an ICT system, including: step S1, obtaining multiple testing tools for testing the ICT system, and obtaining a tool standard table based on the test results of the multiple testing tools.

[0063] Step S1 includes the following sub-steps: Step S101, obtaining multiple test items that can be tested in the Xinchuang whole machine, respectively recorded as test items CX1 to test items CXn, where n is the number of test items;

[0064] Step S102, obtain multiple test tools that can test multiple test items in the trusted computing machine, and record them as test tools CG1 to test tools CGm, where m is the number of test tools; for any test tool CGm1 from test tools CG1 to test tools CGm, record the items in test items CX1 to test items CXn that can be tested using test tool CGm1 as test items of test tool CGm1; obtain the testable items of all test tools, where 1≤m1≤m, and m and m1 are positive integers.

[0065] In the specific implementation process, the test tools selected in this embodiment can be UninxBench, GLmark2, lozone, Stream, CPU2006 and LTP, etc., which can be used for the whole machine of Xinchuang. UnixBench is the original BYTEUNIX benchmark test suite, which was originally designed to provide users with basic indicators of Unix-like system performance. It contains many sub-tests and can be used for 2D display performance testing. GLMark2 is a widely used graphics performance testing tool for 3D display performance testing. Lozone is used for I\0 performance testing, Stream is used for memory bandwidth performance testing, CPU2006 is used for CPU performance testing, and LTP is used for stress testing. In actual testing, the test tools can be adjusted;

[0066] Step S103, for any test tool CGm1 from the test tools CG1 to the test tools CGm, use the test tool CGm1 to test the testable items of the test tool CGm1 in the ICT system under standard conditions, and record the test results in a standard test table, wherein the standard test table is a table of t rows × y columns, and the top row of the standard test table is filled with test items CX1 to test items CXn from left to right except for the leftmost cell, and the leftmost column of the standard test table is filled with the test tool CGm1 in the second cell from top to bottom, as shown in Table 1;

[0067] Table 1 Standard test table

[0068] Test item CX1 … Test Project CXn Test tool CGm1 … … …

[0069] In the specific implementation process, in this embodiment, t takes a value of 2, y takes a value of n+1, and in this embodiment, the standard situation is that the whole Xinchuang machine and the test tool are started normally and there is no external interference;

[0070] Step S104, based on the test method using the test tool CGm1, use all the test tools to test the testable items of the whole computer respectively, and record all the obtained standard test tables as standard test table 1 to standard test table q respectively; establish a tool standard table, wherein the tool standard table is a table of m+1 rows × t columns, and the top row of the tool standard table is filled with test items CX1 to test items CXn from left to right except the leftmost cell, and the leftmost column of the tool standard table is filled with test tools CG1 to test tools CGm from top to bottom except the first cell, please refer to Table 2.

[0071] Table 2 Tool Standard Table

[0072] Test item CX1 … Test Project CXn Test tool CG1 … … … … … … … Test tool CGm … … …

[0073] Step S105 , based on standard test table 1 to standard test table q, fill in corresponding data in the tool standard table.

[0074] Step S2, obtaining the best tool for each test item in the trusted computing machine based on the tool standard table.

[0075] Step S201, for any test item CXn1 among the test items CX1 to CXn, the test tools in the tool standard table that can test the test item CXn1 are recorded as testable tools KCG1 to testable tools KCGs, where s is the number of testable tools; other test items initiated when testing the test item CXn1 are recorded as associated items n1~1 to associated items n1~k, where k is the number of associated items, and s and k are positive integers;

[0076] Step S202, when the number of coordinated projects is 0, respectively obtain the testable tools KCG1 to KCGs that have the shortest testing time for the test project CXn1 and record them as the best tools for the test project CXn1.

[0077] In the specific implementation process, when the number of linked projects is 0, it means that when testing the whole credible machine, the test result of test project CXn1 can only be obtained through one testable tool and the test tool can only test the test project CXn1, that is, a one-to-one test situation is formed. Therefore, only the testable tool with the shortest time consumption needs to be selected as the best tool for test project CXn1, thereby saving time when testing the whole credible machine and improving test efficiency.

[0078] Step S203, when the number of associated projects is greater than 0, a combination method of multiple test tools that can test the test project CXn1 and all associated projects is obtained based on the tool standard table, and they are recorded as test tool combination CZ1 to test tool combination CZg, respectively, where g is the number of test tool combinations, wherein all test tools in any test tool combination can test the test project CXn1 and all associated projects;

[0079] In the specific implementation process, when the number of coordinated projects is greater than 0, it means that when the same test tool is used to test the test project CXn1, all coordinated projects can be tested at the same time, so as to achieve the effect of testing multiple test projects within the same test time, thereby improving time utilization. At the same time, for any coordinated project A of the test project CXn1, after the coordinated project A uses the best tool to obtain the test result, the standard test result of the coordinated project A can be calibrated in combination with the test result obtained as the coordinated project, so that the standard test result of the coordinated project A is more accurate;

[0080] Based on the tool efficiency algorithm, the efficiency value of each test tool combination when testing the test item CXn1 and all the associated items is obtained, and all the tools in the test tool combination with the smallest efficiency value are recorded as the best tools for the test item CXn1; the tool efficiency algorithm includes: Where F is the efficiency value, L is the number of test tools in the test tool combination, L′ is the time used to test the test item CXn1 using the test tool combination, Pi is the time consumed by the i-th test tool in the test tool combination during the test process, and Rmax is the maximum memory occupied during the test using the test tool combination;

[0081] In the specific implementation process, for example, in a data processing process, the number of test tools in a test tool combination of test project CXn1 is 3, the time used to test the test project CXn1 using the test tool combination is 30 minutes, the time consumed by each test tool in the test process is 15 minutes, 10 minutes and 20 minutes respectively, and the maximum memory occupied by the test tool combination during the test is 200MB. It can be obtained by calculation that the efficiency value is 90000. The smaller the efficiency value, the smaller the memory occupied and the test time of the test tool combination when testing the test project CXn1. Selecting a test tool combination with a smaller efficiency value can improve the test efficiency during actual testing.

[0082] Step S204: obtaining the best tools corresponding to all test items based on the analysis method of the test item CXn1.

[0083] Step S3, establish an automated testing model based on the tool standard table and the testing tool, and use the automated testing model to establish an automated testing method for the entire trusted computing machine, and obtain standard testing results based on the automated testing method.

[0084] Step S3 includes: Step S301, establishing an automated test model and storing a tool standard table and all test tools in the automated test model;

[0085] Step S302: Establish an automated testing method based on the best tools for all test items in the automated testing model, and obtain standard test results based on the automated testing method.

[0086] The automated testing method comprises:

[0087] Step S3021, when testing the whole Xinchuang machine, the items to be tested are recorded as items to be tested DCX1 to items to be tested DCXd, where d is the number of items to be tested;

[0088] Step S3022, when the best tool of any one of the items to be tested DCX1 to DCXd is able to test the items to be tested DCX1 to DCXd, d1 is any number from 1 to d, the best tool of the item to be tested DCXd1 is used to test the items to be tested DCX1 to DCXd, and the test results are recorded as the standard test results of the items to be tested DCX1 to DCXd.

[0089] In the specific implementation process, when the best tool of any item to be tested DCXd1 can test items to be tested DCX1 to DCXd, by using the best tool of item to be tested DCXd1, the number of test tools used can be saved to the maximum extent while obtaining the test results of all items to be tested, thereby reducing the startup of redundant test tools while ensuring the normal progress of the test, thereby improving the test efficiency;

[0090] Step S3023, when the best tools for all the items to be tested cannot test the items to be tested DCX1 to DCXd, use the best tools for the items to be tested DCX1 to DCXd to test them respectively;

[0091] Step S3024, for any one of the items to be tested DCXd1 to the items to be tested DCXd, when the best tool of the items to be tested other than the item to be tested DCXd1 can test the item to be tested DCXd1, the test result of the item to be tested DCXd1 using the best tool of the item to be tested DCXd1 is recorded as the primary test result of the item to be tested DCXd1, and the test result of the item to be tested DCXd1 using the best tool of the items to be tested other than the item to be tested DCXd1 is recorded as the secondary test result, and the standard test result of the item to be tested is obtained using the primary and secondary combination algorithm, and the primary and secondary combination algorithm includes:

[0092]

[0093] Among them, A is the standard test result, A1 is the main test result, U is the number of secondary test results, and Aj is the jth secondary test result.

[0094] In the specific implementation process, the primary and secondary combination algorithm can obtain more comprehensive test results after testing the same test item through multiple test tools, thereby improving the accuracy of the test. For example, in a data processing process, the test item is data bandwidth, the main test result is 6Gb / s, and multiple secondary test results are 6.1Gb / s, 6Gb / s, 5.9Gb / s, 6Gb / s and 6.1Gb / s. Through calculation, it can be obtained that the standard test result is 6.01Gb / s;

[0095] Step S3025, when the best tools for the items to be tested except the item to be tested DCXd1 cannot test the item to be tested DCXd1, the test result of the item to be tested DCXd1 using the best tool for the item to be tested DCXd1 is recorded as the standard test result.

[0096] Step S4, obtain all test data after multiple tests on the Xinchuang machine, and obtain the test weight of each test item based on all test data, and adjust the standard test results obtained by the automated testing method based on the test weight.

[0097] Step S4 includes:

[0098] Step S401, use the automated testing method to perform multiple tests on the Xinchuang whole machine, for any test item CXn1 among the test items CX1 to CXn, multiple standard test results of the test item CXn1 during the multiple tests are recorded as reference results KC1 to reference results KCc, where c is the number of times the test is performed;

[0099] Step S402, establishing a plane rectangular coordinate system, recorded as a biased analysis coordinate system, wherein the coordinates of the X-axis of the biased analysis coordinate system from the origin to the right are the reference results KC1 to the reference results KCc, and the unit of the Y-axis is the unit of the data corresponding to the test item CXn1;

[0100] Step S403, based on the reference results KC1 to the reference results KCc, punctuation is performed in the bias analysis coordinate system, which are recorded as bias reference points PZ1 to bias reference points PZc respectively; the best tool for the test item CXn1 is used to test the test item CXn1, and the test result of the test item CXn1 using the best tool for the test item CXn1 is recorded as the control result, and the vertical coordinates of the bias reference points PZ1 to the bias reference points PZc minus the control result are recorded as bias difference values ​​PC1 to bias difference values ​​PCc respectively, and when all bias differences are 0, the test bias of the test item CXn1 is recorded as no bias;

[0101] In the specific implementation process, for example, in a data processing, after multiple tests on the test item CXn1, the biased analysis coordinate system is obtained as follows Figure 3 As shown, X1 to X7 are reference results KC1 to reference results KC7, the straight line corresponding to Y0 is the straight line with the ordinate being the control result, and the points corresponding to Y1 to Y7 are the bias reference points PZ1 to the bias reference points PZ7. Through data analysis, the bias differences are -1 Gb / s, -0.2 Gb / s, 0 Gb / s, -0.1 Gb / s, 0 Gb / s, 0.09 Gb / s and 0.08 Gb / s, respectively. Through analysis, it can be obtained that the test bias of the test item CXn1 is negative bias, and the bias mean is about -0.43 Gb / s. Therefore, in step S405, after using the automated test method to obtain the standard test result of the test item CXn1, the value of the standard test result of the test item CXn1 is adjusted to the value obtained by adding -0.43 Gb / s to the standard test result;

[0102] Step S404, when the number of positive numbers in all bias differences is greater than the number of negative numbers, the test bias of the test item CXn1 is recorded as positive bias, and the average value of the positive numbers in all bias differences is recorded as the bias mean;

[0103] When the number of positive numbers in all bias differences is less than the number of negative numbers, the test bias of test item CXn1 is recorded as negative bias, and the average of the negative numbers in all bias differences is recorded as the bias mean;

[0104] When the number of positive numbers in all bias differences is equal to the number of negative numbers, the test bias of test item CXn1 is recorded as uniform bias, and the average value of all bias differences is recorded as bias mean.

[0105] Step S405, obtaining the test weights of all test items based on the analysis method of the test item CXn1;

[0106] For any test item, when the test bias of the test item is no bias, after the standard test result of the test item is obtained by using the automated test method, the standard test result of the test item is not adjusted;

[0107] When the test bias of the test item is not unbiased, after obtaining the standard test result of the test item using the automated test method, the value of the standard test result is adjusted to a value obtained by adding the standard test result to the bias mean of the test item.

[0108] Example 2

[0109] The difference between this embodiment and embodiment 1 is that step S1 further includes: step S106, after the corresponding data is filled in the tool standard table based on the standard test table, the number of data filled in the columns where the test items CX1 to the test items CXn are located is obtained in sequence, and recorded as the number of test data 1SJ to the number of test data NSJ respectively;

[0110] When any test data quantity N1SJ among the test data quantity 1SJ to the test data quantity NSJ is 0, a new row is added to the tool test table and the leftmost cell of the new row is filled with a test tool that can test the test item CXn1SJ. After the test item CXn1SJ is tested using the test tool in the new row, the test result is filled in the corresponding cell in the tool standard table;

[0111] In a specific implementation process, for example, in a data processing process, the first tool standard table obtained is shown in Table 3 below:

[0112] Table 3 First tool standard table

[0113]

[0114] From the above table, it can be seen that the number of data filled in the column where the stress test is located is 0, so a new row should be added to the tool test table and filled with the test tools that can test the stress test, and the stress test should be tested using the test tools in the new row, and then the test results should be filled in the tool standard table; the second tool standard table obtained based on the above analysis is shown in Table 4 below:

[0115] Table 4 Second tool standard table

[0116]

[0117] Example 3

[0118] See also Figure 2 As shown, the present invention also provides a Xinchuang whole machine adaptability testing system, including a standard table acquisition module, an optimal tool analysis module, a method establishment module and a comprehensive testing module;

[0119] The standard table acquisition module is used to acquire multiple test tools for testing the whole Xinchuang machine, and acquire the tool standard table based on the test results of multiple test tools; the standard table acquisition module is configured with a standard table acquisition strategy, which includes:

[0120] Obtain multiple test items that can be tested in the Xinchuang whole machine, which are recorded as test items CX1 to CXn, where n is the number of test items;

[0121] Acquire multiple test tools that can test multiple test items in the trusted computing machine, and record them as test tools CG1 to test tools CGm, where m is the number of test tools; for any test tool CGm1 among test tools CG1 to CGm, record the items among test items CX1 to CXn that can be tested using test tool CGm1 as test items of test item CXm1; acquire the testable items of all test tools.

[0122] For any test tool CGm1 from test tool CG1 to test tool CGm, use test tool CGm1 to test the testable items of test tool CGm1 in the whole Xinchuang machine under standard conditions, and record the test results in a standard test table, wherein the standard test table is a table of t rows × y columns, and the top row of the standard test table is filled with test items CX1 to test items CXn from left to right except the leftmost cell, and the leftmost column of the standard test table is filled with test tool CGm1 in the second cell from top to bottom;

[0123] Based on the testing method of using the test tool CGm1, all the test tools are used to test the testable items of the whole computer, and all the obtained standard test tables are recorded as standard test table 1 to standard test table q; a tool standard table is established, wherein the tool standard table is a table of m+1 rows × t columns, and the top row of the tool standard table is filled with test items CX1 to test items CXn from left to right except the leftmost cell, and the leftmost column of the tool standard table is filled with test tools CG1 to test tools CGm from top to bottom except the first cell.

[0124] Based on standard test tables 1 to q, fill in corresponding data in the tool standard table. When there is no test result corresponding to any test tool in the column where any test item CXn1 in the tool test table is located, add a new row to the tool test table and fill in the leftmost cell of the new row with a test tool that can test the test item CXn1.

[0125] The best tool analysis module is used to obtain the best tool for each test item in the Xinchuang machine based on the tool standard table; the best tool analysis module is configured with the best tool acquisition strategy, which includes:

[0126] For any test item CXn1 among the test items CX1 to CXn, the test tools in the tool standard table that can test the test item CXn1 are recorded as testable tools KCG1 to testable tools KCGs, where s is the number of testable tools; other test items initiated when testing the test item CXn1 are recorded as associated items n1~1 to associated items n1~k, where k is the number of associated items, and s and k are positive integers;

[0127] When the number of connected projects is 0, the testable tools KCG1 to KCGs are respectively obtained with the shortest testable tools for testing the test project CXn1, and recorded as the best tool for the test project CXn1.

[0128] When the number of associated projects is greater than 0, a combination method of multiple test tools that can test the test project CXn1 and all associated projects is obtained based on the tool standard table, and they are recorded as test tool combination CZ1 to test tool combination CZg, respectively, where g is the number of test tool combinations, wherein all test tools in any test tool combination can test the test project CXn1 and all associated projects;

[0129] Based on the tool efficiency algorithm, the efficiency value of each test tool combination when testing the test item CXn1 and all the associated items is obtained, and all the tools in the test tool combination with the smallest efficiency value are recorded as the best tools for the test item CXn1; the tool efficiency algorithm includes: Where F is the efficiency value, L is the number of test tools in the test tool combination, L′ is the time used to test the test item CXn1 using the test tool combination, Pi is the time consumed by the i-th test tool in the test tool combination during the test process, and Rmax is the maximum memory occupied during the test using the test tool combination;

[0130] Based on the analysis method of test item CXn1, the best tools corresponding to all test items are obtained.

[0131] The method establishment module is used to establish an automated test model based on the tool standard table and the test tool, and use the automated test model to establish an automated test method for the entire Xinchuang machine, and obtain standard test results based on the automated test method; the method establishment module is configured with an automated test method establishment strategy, and the automated test method establishment strategy includes: establishing an automated test model and storing the tool standard table and all test tools in the automated test model;

[0132] In the automated testing model, an automated testing method is established based on the best tools for all test items, and standard test results are obtained based on the automated testing method; the automated testing method includes: when testing the entire Xinchuang machine, the items to be tested are recorded as items to be tested DCX1 to items to be tested DCXd, where d is the number of items to be tested;

[0133] When the best tool for any one of the items DCX1 to DCXd to be tested is able to test the items DCX1 to DCXd to be tested, d1 is any number from 1 to d, the best tool for the item DCXd1 to be tested is used to test the items DCX1 to DCXd to be tested, and the test results are recorded as the standard test results of the items DCX1 to DCXd to be tested.

[0134] The automated testing method further includes: when the best tools for all the items to be tested cannot test the items to be tested DCX1 to DCXd, using the best tools for the items to be tested DCX1 to DCXd to test them respectively;

[0135] For any one of the items to be tested DCXd1 to the items to be tested DCXd, when the best tool of the items to be tested other than the item to be tested DCXd1 can test the item to be tested DCXd1, the test result of the item to be tested DCXd1 using the best tool of the item to be tested DCXd1 is recorded as the primary test result of the item to be tested DCXd1, and the test result of the item to be tested DCXd1 using the best tool of the items to be tested other than the item to be tested DCXd1 is recorded as the secondary test result, and the standard test result of the item to be tested is obtained using the primary and secondary combination algorithm, and the primary and secondary combination algorithm includes: Among them, A is the standard test result, A1 is the main test result, U is the number of secondary test results, and Aj is the jth secondary test result.

[0136] The automated testing method further includes: when the best tools for the items to be tested except the item to be tested DCXd1 cannot test the item to be tested DCXd1, recording the test result of the item to be tested DCXd1 using the best tool for the item to be tested DCXd1 as the standard test result.

[0137] The comprehensive testing module is used to obtain all test data after multiple tests on the entire Xinchuang machine, and obtain the test weight of each test item based on all test data, and adjust the standard test results obtained through automated testing methods based on the test weight.

[0138] The comprehensive test module is configured with a calibration test strategy, which includes: using an automated test method to perform multiple tests on the Xinchuang whole machine, and for any test item CXn1 from test items CX1 to test items CXn, multiple standard test results of test item CXn1 during the multiple tests are recorded as reference results KC1 to reference results KCc, where c is the number of times the test is performed;

[0139] Establish a plane rectangular coordinate system, recorded as a biased analysis coordinate system, wherein the coordinates from the origin of the X-axis of the biased analysis coordinate system to the right are the reference results KC1 to the reference results KCc, and the unit of the Y-axis is the unit of the data corresponding to the test item CXn1;

[0140] Based on the reference results KC1 to the reference results KCc, marks are made in the bias analysis coordinate system, which are recorded as bias reference points PZ1 to bias reference points PZc respectively; the best tool for test item CXn1 is used to test test item CXn1, and the test result of test item CXn1 using the best tool for test item CXn1 is recorded as the control result, and the vertical coordinates of bias reference points PZ1 to bias reference points PZc minus the control result are recorded as bias difference values ​​PC1 to bias difference values ​​PCc respectively, and when all bias difference values ​​are 0, the test bias of test item CXn1 is recorded as no bias;

[0141] When the number of positive numbers in all bias differences is greater than the number of negative numbers, the test bias of test item CXn1 is recorded as positive bias, and the average of the positive numbers in all bias differences is recorded as the bias mean;

[0142] When the number of positive numbers in all bias differences is less than the number of negative numbers, the test bias of test item CXn1 is recorded as negative bias, and the average of the negative numbers in all bias differences is recorded as the bias mean;

[0143] When the number of positive numbers in all bias differences is equal to the number of negative numbers, the test bias of test item CXn1 is recorded as uniform bias, and the average value of all bias differences is recorded as bias mean.

[0144] Based on the analysis method of test item CXn1, the test weight of all test items is obtained;

[0145] For any test item, when the test bias of the test item is no bias, after the standard test result of the test item is obtained by using the automated test method, the standard test result of the test item is not adjusted;

[0146] When the test bias of the test item is not unbiased, after obtaining the standard test result of the test item using the automated test method, the value of the standard test result is adjusted to a value obtained by adding the standard test result to the bias mean of the test item.

[0147] Working principle: First, obtain multiple test tools for testing the entire ICT machine, and obtain the tool standard table; then select the best tool for each test item; then establish an automated test model, and establish an automated test method for the entire ICT machine; finally, obtain the test emphasis of each test item based on the test data, and adjust the standard test results obtained through the automated test method based on the test emphasis.

[0148] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present invention may adopt the form of a computer program product implemented on one or more computer-usable storage media in which a computer-usable program code is included. Wherein, the storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM for short), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), erasable programmable read-only memory (Erasable Programmable Read 0nly Memory, EPROM for short), programmable read-only memory (Programmable Red-Only Memory, PROM for short), read-only memory (Read-Only Memory, ROM for short), magnetic storage, flash memory, disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] The above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for testing the adaptability of a complete machine, characterized in that: include: Obtain multiple test tools for testing the whole device, and obtain a tool standard table based on the test results of the multiple test tools; Get the best tool for each test item in the whole machine based on the tool standard table; Establish an automated test model based on the tool standard table and test tools, and use the automated test model to establish an automated test method for the entire Xinchuang machine, and obtain standard test results based on the automated test method; Obtain all test data after multiple tests on the Xinchuang machine, and obtain the test weight of each test item based on all test data, and adjust the standard test results obtained by the automated testing method based on the test weight; Obtain multiple test tools for testing the whole Xinchuang machine, and obtain the tool standard table based on the test results of multiple test tools, including: Obtain multiple test items that can be tested in the Xinchuang whole machine, which are recorded as test items CX1 to CXn, where n is the number of test items; Obtain multiple test tools that can test multiple test items in the ICT system, and record them as test tools CG1 to test tools CGm, where m is the number of test tools; for any test tool CGm1 from test tools CG1 to test tools CGm, m1 is any number from 1 to m, and record the items in test items CX1 to test items CXn that can be tested using test tool CGm1 as testable items of test tool CGm1; obtain the testable items of all test tools, where 1≤m1≤m, and m and m1 are positive integers; Obtain multiple test tools for testing the whole device, and obtain the tool standard table based on the test results of multiple test tools, which also includes: For any test tool CGm1 from test tool CG1 to test tool CGm, use test tool CGm1 to test the testable items of test tool CGm1 in the whole Xinchuang machine under standard conditions, and record the test results in a standard test table, wherein the standard test table is a table of t rows × y columns, and the top row of the standard test table is filled with test items CX1 to test items CXn from left to right except the leftmost cell, and the leftmost column of the standard test table is filled with test tool CGm1 in the second cell from top to bottom; Based on the test method of using the test tool CGm1, all the test tools are used to test the testable items of the Xinchuang whole machine respectively, and all the obtained standard test tables are recorded as standard test table 1 to standard test table q respectively; a tool standard table is established, wherein the tool standard table is a table of m+1 rows×t columns, and the top row of the tool standard table is filled with test items CX1 to test items CXn from left to right except the leftmost cell, and the leftmost column of the tool standard table is filled with test tools CG1 to test tools CGm from top to bottom except the first cell, m=q; Based on standard test tables 1 to q, fill in corresponding data in the tool standard table, wherein, when there is no test result corresponding to any test tool in the column where any test item CXn1 in the tool test table is located, add a new row in the tool test table and fill in the leftmost cell of the new row with the test tool that can test the test item CXn1, 1≤n1≤n, n1 and n are positive integers; The best tools for obtaining each test item in the Xinchuang machine based on the tool standard table include: For any test item CXn1 among the test items CX1 to CXn, the test tools in the tool standard table that can test the test item CXn1 are recorded as testable tools KCG1 to testable tools KCGs, where s is the number of testable tools; other test items initiated when testing the test item CXn1 are recorded as associated items n1~1 to associated items n1~k, where k is the number of associated items; s and k are positive integers; When the number of the associated projects is 0, the testable tools KCG1 to KCGs are respectively obtained, which have the shortest test time for testing the test project CXn1, and are recorded as the best tools for the test project CXn1; When the number of associated projects is greater than 0, a combination method of multiple test tools that can test the test project CXn1 and all associated projects is obtained based on the tool standard table, and they are recorded as test tool combination CZ1 to test tool combination CZg, respectively, where g is the number of test tool combinations, wherein all test tools in any test tool combination can test the test project CXn1 and all associated projects; Based on the tool efficiency algorithm, the efficiency value of each test tool combination when testing the test item CXn1 and all the associated items is obtained, and all the tools in the test tool combination with the smallest efficiency value are recorded as the best tools for the test item CXn1; the tool efficiency algorithm includes: Where F is the efficiency value, L is the number of test tools in the test tool combination, L' is the time used to test the test item CXn1 using the test tool combination, Pi is the time consumed by the i-th test tool in the test tool combination during the test process, and Rmax is the maximum memory occupied during the test using the test tool combination; i≤L; Based on the analysis method of test item CXn1, the best tools corresponding to all test items are obtained.

2. According to claim 1, a method for testing the adaptability of a complete computer is characterized in that: An automated test model is established based on the tool standard table and test tools, and an automated test method for the entire Xinchuang machine is established using the automated test model. Standard test results obtained based on the automated test method include: Establish an automated test model and store the tool standard table and all test tools in the automated test model; In the automated testing model, an automated testing method is established based on the best tools for all test items, and standard test results are obtained based on the automated testing method.

3. According to claim 2, a method for testing the adaptability of a complete computer is characterized in that: The automated testing method comprises: When testing the whole Xinchuang machine, the items to be tested are recorded as items to be tested DCX1 to items to be tested DCXd, where d is the number of items to be tested; When the best tool for any one of the items DCX1 to DCXd to be tested is able to test the items DCX1 to DCXd to be tested, d1 is any number from 1 to d, the best tool for the item DCXd1 to be tested is used to test the items DCX1 to DCXd to be tested, and the test results are recorded as the standard test results of the items DCX1 to DCXd to be tested.

4. According to the method for testing the adaptability of a complete computer system, the method is characterized in that: The automated testing method further comprises: When the best tools for all the items to be tested cannot test the items to be tested DCX1 to DCXd, the best tools for the items to be tested DCX1 to DCXd are used to test them respectively; For any one of the items to be tested DCXd1 to the items to be tested DCXd, when the best tool of the items to be tested other than the item to be tested DCXd1 can test the item to be tested DCXd1, the test result of the item to be tested DCXd1 using the best tool of the item to be tested DCXd1 is recorded as the primary test result of the item to be tested DCXd1, and the test result of the item to be tested DCXd1 using the best tool of the items to be tested other than the item to be tested DCXd1 is recorded as the secondary test result, and the standard test result of the item to be tested is obtained using the primary and secondary combination algorithm, and the primary and secondary combination algorithm includes: Where A is the standard test result, A1 is the primary test result, U is the number of secondary test results, and Aj is the jth secondary test result; When the best tools for all the items to be tested except the item to be tested DCXd1 cannot test the item to be tested DCXd1, the test result of the item to be tested DCXd1 using the best tool for the item to be tested DCXd1 is recorded as the standard test result.

5. According to the method for testing the adaptability of a complete ICT system according to claim 4, it is characterized in that: Obtain all test data after multiple tests on the Xinchuang machine, and obtain the test weight of each test item based on all test data. Adjust the standard test results obtained by the automated test method based on the test weight, including: Use an automated testing method to test the whole Xinchuang machine multiple times. For any test item CXn1 from test items CX1 to test items CXn, record multiple standard test results of test item CXn1 during the multiple tests as reference results KC1 to reference results KCc, where c is the number of times the test is performed; Establish a plane rectangular coordinate system, recorded as a biased analysis coordinate system, wherein the coordinates from the origin of the X-axis of the biased analysis coordinate system to the right are the reference results KC1 to the reference results KCc, and the unit of the Y-axis is the unit of the data corresponding to the test item CXn1; Based on the reference results KC1 to the reference results KCc, marks are made in the bias analysis coordinate system, which are recorded as bias reference points PZ1 to bias reference points PZc respectively; the best tool for test item CXn1 is used to test test item CXn1, and the test result of test item CXn1 using the best tool for test item CXn1 is recorded as the control result, and the vertical coordinates of bias reference points PZ1 to bias reference points PZc minus the control result are recorded as bias difference values ​​PC1 to bias difference values ​​PCc respectively, and when all bias difference values ​​are 0, the test bias of test item CXn1 is recorded as no bias; When the number of positive numbers in all bias differences is greater than the number of negative numbers, the test bias of test item CXn1 is recorded as positive bias, and the average of the positive numbers in all bias differences is recorded as the bias mean; When the number of positive numbers in all bias differences is less than the number of negative numbers, the test bias of test item CXn1 is recorded as negative bias, and the average of the negative numbers in all bias differences is recorded as the bias mean; When the number of positive numbers in all bias differences is equal to the number of negative numbers, the test bias of test item CXn1 is recorded as uniform bias, and the average value of all bias differences is recorded as bias mean.

6. According to claim 5, a method for testing the adaptability of a complete computer is characterized in that: Obtain all test data after multiple tests on the Xinchuang machine, and obtain the test weight of each test item based on all test data. Adjust the standard test results obtained by the automated test method based on the test weight, which also includes: Based on the analysis method of test item CXn1, the test weight of all test items is obtained; For any test item, when the test bias of the test item is no bias, after the standard test result of the test item is obtained by using the automated test method, the standard test result of the test item is not adjusted; When the test bias of the test item is not unbiased, after obtaining the standard test result of the test item using the automated test method, the value of the standard test result is adjusted to a value obtained by adding the standard test result to the bias mean of the test item.

7. A system for testing the adaptability of an ICT system, used to implement a method for testing the adaptability of an ICT system as described in any one of claims 1 to 6, characterized in that: It includes standard table acquisition module, optimal tool analysis module, method establishment module and comprehensive testing module; The standard table acquisition module is used to acquire multiple test tools for testing the whole Xinchuang machine, and acquire the tool standard table based on the test results of multiple test tools; The best tool analysis module is used to obtain the best tool for each test item in the Xinchuang machine based on the tool standard table; The method establishment module is used to establish an automated test model based on the tool standard table and the test tool, and use the automated test model to establish an automated test method for the entire Xinchuang machine, and obtain standard test results based on the automated test method; The comprehensive testing module is used to obtain all test data after multiple tests on the entire Xinchuang machine, and obtain the test weight of each test item based on all test data, and adjust the standard test results obtained through automated testing methods based on the test weight.

Images