Performance testing method based on engine matrix module

Abstract

The invention relates to the technical field of data analysis and processing, and specifically comprises a performance detection method based on an engine matrix module, comprising: collecting program feature information, indexing similar and anti-class test cases, testing a detection engine, analyzing performance parameters, obtaining a performance detection result of a detection engine on a program set, solving the technical problems that the performance detection process of the engine matrix module ignores the adaptability and performance difference of the engine itself to a specific program set, the detection of some types of programs is more accurate, the detection performance of other types of programs is poor, and the performance detection lacks universality and consistency, realizes comprehensive testing of the detection engine using test cases, constructs a test matrix containing various performance indicators, comprehensively evaluates the performance of the detection engine on different program sets through the engine performance matrix, identifies the strengths and weaknesses of the engine and optimizes them, ensures the universality and consistency of the performance detection, and further improves the technical effect of the accuracy of the detection result.

Description

Technical Field

[0001] The present invention relates to the technical field related to data analysis and processing, and in particular to a performance detection method based on an engine matrix module. Background Art

[0002] In the field of modern software development and security, automated security detection tools (such as detection engines) are increasingly used to quickly identify and fix security vulnerabilities in code, component insecurity, database risks, interface exposure and other issues.

[0003] However, different types of software and applications have unique architectures, which means that a single detection strategy cannot be fully and efficiently applied to all scenarios. The conventional performance detection process of the engine matrix module ignores the adaptability and performance differences of the detection engine itself to a specific set of programs, resulting in deviations, omissions or false positives in the detection results, reducing the accuracy of the detection.

[0004] To sum up, the performance detection process of the engine matrix module in the prior art ignores the adaptability and performance differences of the engine itself to a specific set of programs, detects some types of programs more accurately, but performs poorly on other types of programs, and the performance detection lacks universality and consistency. Technical problems. Summary of the invention

[0005] The present application provides a performance detection method based on an engine matrix module, aiming to solve the technical problems that the performance detection process of the engine matrix module in the prior art ignores the adaptability and performance differences of the engine itself to a specific set of programs, is more accurate in detecting some types of programs, but performs poorly on other types of programs, and lacks universality and consistency in performance detection.

[0006] In view of the above problems, the technical solution to implement this application is:

[0007] The present application provides a performance detection method based on an engine matrix module, wherein the method includes: obtaining a set of programs that are currently to be security-detected by a detection engine, and collecting multiple program feature information of multiple detection programs in the program set, wherein the detection engine includes a detection module matrix, and the detection module matrix includes M detection modules, where M is an integer greater than 1; according to the multiple program feature information, indexing to obtain multiple similar test cases and multiple anti-class test cases; dividing the multiple similar test cases to obtain a first similar test case set and a second similar test case set, combining the multiple anti-class test cases, testing the detection engine to obtain a first similar test result set, a second similar test result set, and an anti-class test result set, wherein the first similar test result set includes M first similar A test result cluster, each first similar test result cluster includes N performance test results, N is an integer greater than 1; the first similar test result set, the second similar test result set and the reverse test result set are processed to construct a first similar test matrix, a second similar test matrix and a reverse test matrix, and the detection reproduction performance parameters are obtained by processing; the first similar test matrix and the second similar test matrix are merged to obtain a similar test matrix, and the performance analysis of the M detection modules is performed to obtain a detection performance parameter set; according to the multiple program feature information, the detection performance requirement parameter set of the program set is analyzed and obtained, and the basic performance detection result is calculated in combination with the detection performance parameter set, and the performance detection result of the detection engine relative to the program set is obtained by processing in combination with the detection reproduction performance parameter.

[0008] In summary, one or more technical solutions provided in the present application solve the technical problems that the performance detection process of the engine matrix module ignores the adaptability and performance differences of the engine itself to a specific program set, detects some types of programs more accurately, and performs poorly on other types of programs, and the performance detection lacks universality and consistency. It realizes the use of test cases to comprehensively test the detection engine, and constructs a test matrix containing various performance indicators. By constructing an engine performance matrix containing multi-dimensional performance indicators, the performance of the detection engine on different program sets is comprehensively evaluated, the strengths and weaknesses of the engine are identified and optimized, and deviations, omissions or false alarms caused by engine bias are avoided, thereby ensuring the universality and consistency of performance detection, and thereby improving the technical effect of improving the accuracy of detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flow chart of a performance detection method based on an engine matrix module is provided for this application;

[0010] Figure 2A flowchart of indexing and obtaining multiple similar test cases and multiple anti-class test cases in a performance detection method based on an engine matrix module is provided for this application. DETAILED DESCRIPTION

[0011] The present application is described in detail below with reference to the accompanying drawings. Figure 1 As shown, the present application provides a performance detection method based on an engine matrix module, wherein the method includes:

[0012] S1: Obtain a set of programs that are currently to be security-checked by a detection engine, and collect multiple program feature information of multiple detection programs in the program set, wherein the detection engine includes a detection module matrix, and the detection module matrix includes M detection modules, where M is an integer greater than 1; S2: According to the multiple program feature information, index and obtain multiple similar test cases and multiple anti-class test cases.

[0013] The detection engine is used to perform automated security detection on developed software programs. However, the detection engine may have different detection performance for different programs and have a certain bias. Based on this, before performing program detection, the present application first detects the performance of the detection engine on the current program, identifies the strengths and weaknesses of the engine and optimizes them to make the performance of the detection engine more balanced and reduce bias, thereby ensuring the accuracy of the detection results. Specifically, a set of programs that need to be security-checked is clearly identified, including newly developed applications, system updates, or third-party components. Utilize automated tools or manual review to extract from the program set multiple program feature information of multiple detection programs in the program set described below, including code feature information (analyzing code structure, programming habits, function complexity, etc.), component feature information (identifying the third-party libraries used, frameworks, and their version information), database feature information (database type, table structure, query mode, etc.), and interface feature information (API interface definition, calling method, data transmission protocol, etc.).

[0014] In the actual application process, it also includes the configuration of the detection engine and the detection module matrix. Further, according to the pre-set security detection requirements, M detection modules are configured, including at least: source code security detection module (checks whether there are known vulnerability patterns, unsafe coding practices, etc. in the code), component security detection module (verifies whether third-party components have known security vulnerabilities), security vulnerability scanning detection module (automatically scans the software for vulnerabilities that may be exploited), database security testing module (checks the security of database configuration and access control), API interface testing module (ensures that API calls meet security standards and there is no risk of unauthorized access).

[0015] According to the multiple program feature information, multiple similar test cases and multiple anti-class test cases are indexed and obtained. Specifically, the database stores historical test cases and feature information, and the historical test cases cover different types of software security testing scenarios. From the database, according to the multiple program feature information, a similarity analysis algorithm (such as cosine similarity, Jaccard similarity, etc.) is used to calculate the similarity between the target program features and the features of each case in the test case library. Based on the similarity analysis results, multiple similar test cases that are closest to the target program features and multiple anti-class test cases with the lowest similarity are selected. Multiple similar test cases and multiple anti-class test cases are indexed and obtained, aiming to cover the most likely and least likely situations, so as to comprehensively evaluate the performance of the detection engine.

[0016] Through the above steps, the adaptability and performance of the detection engine to different program sets are systematically evaluated, providing strong support for software security detection.

[0017] S3: Divide the multiple similar test cases to obtain a first similar test case set and a second similar test case set, and test the detection engine in combination with the multiple anti-class test cases to obtain a first similar test result set, a second similar test result set, and an anti-class test result set, wherein the first similar test result set includes M first similar test result clusters, each first similar test result cluster includes N performance test results, and N is an integer greater than 1; S4: Process the first similar test result set, the second similar test result set, and the anti-class test result set to construct a first similar test matrix, a second similar test matrix, and an anti-class test matrix, and process to obtain detection reproduction performance parameters.

[0018] The multiple similar test cases are divided to obtain a first similar test case set and a second similar test case set. Specifically, according to the previously indexed multiple similar test cases and multiple anti-class test cases, ensure that they cover all expected test scenarios; the similar test cases are randomly divided into a first similar test case set and a second similar test case set according to specific rules (such as functional similarity), ensuring that the first similar test case set and the second similar test case set can independently reflect the characteristics of similar programs.

[0019] The detection engine is tested to obtain a first set of similar test results, a second set of similar test results, and a set of counter-test results. Specifically, a parallel testing method is adopted to perform security checks on the first set of similar test cases, the second set of similar test cases, and the counter-test cases, respectively, and a comprehensive security analysis is performed using the M modules of the detection engine. For example, the mean of N performance parameters of M modules of multiple test cases in the test results of the first set of similar test cases is calculated to obtain the first set of similar test results, the second set of similar test results, and the counter-test results; the performance parameters of each test case on the M modules, such as speed, accuracy, recall rate, etc., are recorded in detail to ensure the comprehensiveness and accuracy of the data.

[0020] The first set of similar test results, the second set of similar test results and the reverse test result set are processed to construct the first similar test matrix, the second similar test matrix and the reverse test matrix. Specifically, the average value of the N performance parameters of the M modules in each set of test results (the first set of similar test results, the second set of similar test results and the reverse test result set) is calculated to form three test result sets, which are: the first set of similar test results: M first similar test result clusters, each cluster containing N average performance parameters; the second set of similar test results: the second set of similar test results with the same structure as the M first similar test result clusters; the reverse test result set: a set of test results reflecting the reverse test case; and then, according to the calculated mean data, the first similar test matrix, the second similar test matrix and the reverse test matrix are constructed. The columns of the matrix represent different performance test results (speed, accuracy, recall rate, etc.), and the rows represent different detection modules (source code security detection, component security detection, etc.). Further, the data in the matrix is ​​normalized to ensure that the performance parameters of different dimensions can be directly compared.

[0021] Obtain detection reproduction performance parameters. Specifically, on the one hand, perform similarity analysis, calculate the similarity between the first similar test matrix and the second similar test matrix, evaluate the reproduction performance of similar test cases, and expect to obtain a higher similarity to prove the engine's consistency detection capability for similar programs; on the other hand, perform anti-class performance comparison, compare the similarity between the similar matrix and the anti-class test matrix, and analyze the performance stability of the detection engine when facing programs with different similarities. Even if the anti-class program has a low similarity with the target program, the engine should exhibit stable and reasonable detection performance; then, based on the similarity analysis results of the similar and anti-class matrices, calculate the detection reproduction performance parameters, and evaluate the adaptability and performance of the detection engine to the current program set.

[0022] In the actual application process, it also includes identifying the performance bottlenecks and advantage modules of the detection engine based on the detection and reproduction performance parameters, providing a basis for engine optimization; organizing test data and analysis results, and preparing detailed test reports, including performance evaluation, problem discovery and improvement suggestions.

[0023] Through the above steps, the performance of the detection engine can be objectively evaluated, and performance optimization work can be guided to ensure that the detection engine can effectively respond to diverse security detection needs.

[0024] S5: Fusion the first similar test matrix and the second similar test matrix to obtain a similar test matrix, conduct performance analysis on the M detection modules, and obtain a detection performance parameter set; S6: Analyze and obtain the detection performance requirement parameter set of the program set based on the multiple program feature information, calculate and obtain the basic performance detection result in combination with the detection performance parameter set, and process and obtain the performance detection result of the detection engine relative to the program set in combination with the detection reproduction performance parameter.

[0025] The first similar test matrix and the second similar test matrix are fused to obtain a similar test matrix. Specifically, the mean of the elements at each corresponding position of the first similar test matrix and the second similar test matrix (i.e., each performance test result of each detection module) is calculated to fuse the two matrices. The generated new matrix is ​​called a similar test matrix, which aims to eliminate the randomness caused by the division of the test set, thereby more stably reflecting the detection performance of similar programs.

[0026] Perform performance analysis on the M detection modules to obtain a set of detection performance parameters. Specifically, based on the fused similar test matrix, perform performance analysis on the M detection modules. The specific method may be to calculate the difference or ratio of the performance indicators of each module (such as average speed, accuracy, recall rate, etc.) compared with the average value of all modules, so as to quantify the relative performance of each module. For example, calculate the detection efficiency ratio (ratio of actual detection efficiency to theoretical optimal efficiency) or performance index of each module. Furthermore, determine the key areas and priorities of detection based on the characteristic information of the program collection, such as code complexity, component type, database interaction mode, number and complexity of API interfaces, etc.; use historical detection data and known performance requirements to build a requirement analyzer, which is used to analyze the specific detection requirements of the current program collection, including determining the relative importance of the M detection modules in the current detection task, and forming a set of detection performance requirement parameters.

[0027] Based on the multiple program feature information, the detection performance requirement parameter set of the program set is analyzed and obtained, and combined with the detection performance parameter set, a basic performance detection result is calculated. Specifically, the detection performance parameter set and the detection performance requirement parameter set are combined to score the performance of each detection module. In a feasible implementation method, the performance parameter of the detection module is divided by the corresponding requirement parameter to obtain the module's performance ratio, and these performance ratios are weighted and summed according to the weight of the requirement parameter to obtain the basic performance detection result. The basic performance detection result reflects the overall performance of the entire detection engine under the premise of considering the characteristics of the detection task.

[0028] In combination with the detection reproduction performance parameters, the performance test results of the detection engine relative to the program set are processed and obtained. Specifically, the detection reproduction performance parameters are introduced to adjust the basic performance test results. Preferably, considering the universal applicability and stability of the engine to programs of different similarities, the basic results are corrected by adjusting factors or directly correcting them to ensure that the final performance test results can more accurately reflect the performance of the detection engine in actual applications.

[0029] Through the above steps, the performance of the detection engine for a specific set of programs is systematically analyzed and evaluated, providing a basis for engine optimization and application selection.

[0030] Further, obtaining a set of programs to be currently security-checked by a detection engine, and collecting multiple program feature information of multiple detection programs in the program set, the method of the present application includes:

[0031] Obtain a program set that is currently to be security-checked by a detection engine; collect code feature information, component feature information, database feature information, and interface feature information of multiple detection programs in the program set as multiple program feature information.

[0032] To ensure the comprehensive and accurate collection of program feature information, it is preferred to obtain the set of programs that are currently to be security-checked by the detection engine. Specifically, the goals of security testing are clarified, including whether the purpose of the test is to discover potential security vulnerabilities, compliance checks or performance optimization, etc., as well as the scope of testing, such as whether it covers a specific stage in the entire software development life cycle or the entire code base; based on the purpose of the test, determine the set of programs that need to pass the security test, which involves pulling the code of a specified branch or tag from a version control system (such as Git), or extracting a running service instance from a production environment.

[0033] The code feature information, component feature information, database feature information and interface feature information of multiple detection programs in the program collection are collected as multiple program feature information. Specifically, in order to standardize the collection process, specific collection standards for code features, component features, database features and interface features need to be defined in advance. For example, code features include code complexity and code style consistency; component features focus on third-party library versions and dependencies; database features involve SQL query modes and data access permissions; interface features include API usage frequency and security configuration; select or customize and develop suitable tools and scripts to automatically collect the required feature information, including static code analysis tools (such as SonarQube), component scanning tools (such as OWASP Dependency-Check), database auditing tools (such as SQL VulnerabilityAssessment) and API testing tools (such as Postman or Swagger Inspector), so as to systematically and effectively obtain multi-dimensional program feature information of multiple detection programs in the program collection, laying a solid foundation for subsequent security detection and performance analysis.

[0034] In the actual application process, it also includes collecting code feature information, and further, using static code analysis tools to scan the code base to collect code feature information such as code quality, complexity, and potential vulnerabilities; collecting component feature information, and further, running component scanning tools to detect the version of dependent libraries and identify whether there are known security vulnerabilities; collecting database feature information, and further, using database audit tools to analyze database query statements, evaluate data access control and potential risks, and collecting interface feature information, and further, using API testing tools to call all public interfaces, record response time, security configuration (such as authentication, encryption), etc.; integrating the various feature information collected into a data set in a unified format for subsequent analysis; storing data in a secure data warehouse or database to ensure data traceability and security; conducting preliminary verification of the collected feature information to confirm that there are no omissions or errors, and backtracking and re-collecting when necessary; inviting a security expert system to review key feature information to ensure the accuracy and completeness of the information.

[0035] Furthermore, if Figure 2 As shown, according to the plurality of program feature information, multiple similar test cases and multiple anti-class test cases are obtained by indexing, and the method of the present application includes:

[0036] Obtain a test case library, wherein the test case library includes multiple historical test cases and multiple historical test case feature information of the detection engine; based on the multiple program feature information, traverse and analyze the similarity with the multiple historical test case feature information; index and obtain multiple historical test cases with the greatest similarity as multiple similar test cases, and index and obtain multiple historical test cases with the least similarity as multiple anti-class test cases.

[0037] Establish a database containing multiple historical test cases and their feature information that have been executed by the detection engine in the past, and name it the test case library. The historical test cases should cover various types and complexities to ensure broad representativeness. For each historical test case in the test case library, ensure that its feature information (such as code structure, component usage, database interaction, API call mode, etc.) has been labeled and recorded according to unified standards for comparison.

[0038] According to the plurality of program feature information, traverse and analyze the similarity with the plurality of historical test case feature information. Specifically, for the current set of programs to be detected, use a predefined feature extraction method to convert the code feature information, component feature information, database feature information and interface feature information into quantifiable and comparable feature vectors; determine a mathematical model or algorithm for measuring the similarity between the program feature information and the historical test case feature information, such as cosine similarity, Jac card coefficient or edit distance, and select the most appropriate algorithm according to the nature of the feature; traverse the historical test cases, use the selected similarity calculation method for each historical test case in the test case library, and compare it with the feature vector of the current program to be detected, and record the similarity score of each historical test case with the current program feature information.

[0039] The multiple historical test cases with the greatest similarity are obtained by indexing, as multiple similar test cases, and the multiple historical test cases with the least similarity are obtained by indexing, as multiple anti-class test cases. Specifically, the historical test cases are sorted according to the calculated similarity scores, and several of the test cases with the highest similarity are selected as similar test cases, which are closest to the program to be tested in structure and function; at the same time, several of the test cases with the lowest similarity are selected as anti-class test cases, which are quite different from the program to be tested. Furthermore, a similarity threshold is set to determine which test cases are very similar or extremely dissimilar, so as to more accurately divide the similar and anti-class test cases.

[0040] In the actual application process, it also includes creating indexes for the selected similar test cases and anti-class test cases to facilitate quick retrieval and subsequent use. It efficiently selects the most relevant similar test cases and anti-class test cases with significant differences from the historical test case library based on the feature information of the program to be detected, providing strong support for the performance analysis and optimization of the detection engine.

[0041] Further, the first similar test result set, the second similar test result set and the reverse test result set are processed to construct a first similar test matrix, a second similar test matrix and a reverse test matrix, and the detection recurrence performance parameters are obtained. The method of the present application includes:

[0042] Normalize the data in the first similar test result set, the second similar test result set, and the inverse test result set; and construct a first similar test matrix, a second similar test matrix, and an inverse test matrix based on the normalization result, wherein the first similar test matrix is ​​as follows:

[0043] Calculate the similarity between the first similar test matrix and the second similar test matrix to obtain similarity and detection reproducibility of the same type; calculate the mean similarity of the first similar test matrix, the second similar test matrix and the reverse test matrix to obtain reverse detection similarity; calculate and obtain detection reproducibility performance parameters based on the similar detection reproducibility and reverse detection similarity.

[0044] Select appropriate normalization methods, such as maximum-minimum normalization, Z-score standardization, etc., to eliminate the differences in dimensions and magnitudes between data and ensure that various performance indicators are comparable on the same scale; normalize all performance test result data in the first set of similar test results, the second set of similar test results, and the reverse set of similar test results, including calculating the maximum, minimum, or average value and standard deviation of each performance parameter, and adjust each value accordingly to fall within a uniform range (such as between 0 and 1).

[0045] Based on the results of the normalization process, a first similar test matrix, a second similar test matrix and an anti-class test matrix are constructed. Specifically, a first similar test matrix is ​​constructed, and further, based on the normalized data, a first similar test matrix is ​​constructed. The rows of the matrix represent different detection modules (M), the columns represent the types of performance test results (N), and each element is the normalized score of the corresponding detection module on a certain performance test; a second similar test matrix is ​​constructed, and further, the normalized data of the second similar test case is used to construct the second similar test matrix in the same way as the first similar test matrix; an anti-class test matrix is ​​constructed, and further, an anti-class test matrix is ​​constructed, and the processing method is similar to that of the similar test matrix, but the data comes from the anti-class test results.

[0046] In a feasible implementation method, the similarity between the first similar test matrix and the second similar test matrix is ​​calculated to obtain the similarity and the reproducibility of the same detection. Specifically, the similarity between the first similar test matrix and the second similar test matrix is ​​calculated using a matrix similarity calculation method (such as Pearson correlation coefficient, cosine similarity, etc.), which helps to evaluate the consistency and stability of the same test results; based on the similarity results, the consistency of the same test results is analyzed to obtain the detection reproducibility, that is, the repeatability of the detection results on similar programs.

[0047] In a feasible implementation method, the mean similarity of the first similar test matrix, the second similar test matrix and the anti-class test matrix is ​​calculated to obtain the anti-class detection similarity. Specifically, the similarity between the first similar test matrix and the anti-class test matrix, and the similarity between the second similar test matrix and the anti-class test matrix are calculated respectively, and the mean similarity between the first similar test matrix and the anti-class test matrix, and the mean similarity between the second similar test matrix and the anti-class test matrix are taken to evaluate the universal applicability and consistency of the detection engine for different (or opposite) programs.

[0048] According to the reproducibility of the same type of detection and the similarity of the anti-type detection, the detection reproducibility performance parameters are calculated. Specifically, based on the reproducibility of the same type of detection and the similarity of the anti-type detection, the performance stability and adaptability of the detection engine in the face of different program characteristics are comprehensively analyzed. According to the reproducibility of the same type of detection and the similarity of the anti-type detection, the detection reproducibility performance parameters are quantified, which involves weighted average, proportional adjustment or other mathematical transformation of the reproducibility and similarity, so as to finally obtain one or a group of parameters reflecting the performance stability and generalization ability of the detection engine, and systematically analyze the performance of the detection engine in various situations, so as to provide data support for optimizing the detection process and improving the detection efficiency and accuracy.

[0049] Further, the first similar test matrix and the second similar test matrix are merged to obtain a similar test matrix, and the performance analysis of the M detection modules is performed to obtain a detection performance parameter set. The method of the present application includes:

[0050] The means of the elements at the same position in the first similar test matrix and the second similar test matrix are calculated, and matrix fusion is performed to obtain the similar test matrix, which is as follows:

[0051] Among them, R is the same test matrix, X 11 is the first fusion performance test result of the first detection module, X M1 is the first fusion performance test result of the Mth detection module, X 1N is the Nth fusion performance test result of the first detection module, X MNis the Mth fusion performance test result of the Mth detection module; according to the similar test matrix, the performance analysis of the M detection modules is performed to obtain a set of detection performance parameters.

[0052] The first similar test matrix and the second similar test matrix are fused, and M detection module performance analyses are performed based on the fused similar test matrices to obtain a set of detection performance parameters. Specifically, the first similar test matrix and the second similar test matrix have been properly preprocessed and normalized, and the first similar test matrix and the second similar test matrix have the same structure, that is, both have M rows and N columns, wherein M is the number of detection modules, and N is the type of performance test result. For each detection module and each performance test result, the average value of the elements at corresponding positions in the first similar test matrix and the second similar test matrix is ​​calculated.

[0053] The calculated mean is used to fill a new matrix R to form a similar test matrix, which combines the average performance of the first similar test matrix and the second similar test matrix. Each row represents the performance of each detection module, and each column corresponds to the average performance test result of a performance test. Then, according to the similar test matrix, the performance analysis of the M detection modules is performed to obtain a detection performance parameter set. Specifically, the performance parameters of all detection modules are sorted to obtain a detection performance parameter set. The detection performance parameter set reflects the comprehensive performance of each detection module of the detection engine, wherein the performance parameter of each detection module is a key indicator for evaluating its effectiveness, and the set is the effectiveness of the entire engine performance matrix.

[0054] Furthermore, according to the same type of test matrix, the performance analysis of the M detection modules is performed, and the method of the present application includes:

[0055] According to the same type test matrix, the M detection performance parameters of the M detection modules are calculated to obtain a detection performance parameter set, which is as follows:

[0056] Among them, K i is the detection performance parameter of the i-th detection module among M detection modules, X maxj is the maximum value of the fusion performance test results in the jth column, X ij is the jth fusion performance test result of the i-th detection module, X minj is the minimum value of the fusion performance test results in the jth column.

[0057] According to the homogeneous test matrix, the performance analysis of M detection modules is performed, and a detection performance parameter set is calculated. Specifically, after the homogeneous test matrix (denoted as R) is constructed, the homogeneous test matrix is ​​a matrix of M rows and N columns, where M represents the number of detection modules and N represents the type of performance test results; initialize variables, define the initialization performance parameters of each detection module, and record the maximum value of each column, including, X max1 , X max2 ,……,X maxj ,……,X maxN , where X maxj is the maximum value of the jth column fusion performance test result; record the minimum value of each column, including X min1 , X min2 ,……,X minj ,……,X minN ; Traverse each column of the same test matrix (j = 1 to N) and calculate the maximum value of the column, X maxj and minimum value X minj , evaluate the performance of each module by comparing the detection result of each module with the maximum and minimum values ​​of the column; then calculate module by module: for each detection module; traverse the fusion performance detection results of the module on each column j; then, collect the calculated performance parameters of each detection module to form a detection performance parameter set, which provides a performance evaluation of M detection modules in similar tests. The closer the evaluation result is to 1, the closer the performance of the detection module is to the optimal, and vice versa.

[0058] In the actual application process, it also includes evaluating and troubleshooting the performance of each detection module based on the obtained detection performance parameter set. Further, it includes analyzing which modules perform well and which need improvement, as well as the strengths and weaknesses of the overall detection system. The performance of the detection modules is systematically quantified to provide data support for optimizing the overall performance of the detection engine.

[0059] Further, according to the plurality of program feature information, a detection performance requirement parameter set of the program set is analyzed and obtained, and a basic performance detection result is calculated and obtained in combination with the detection performance parameter set. The method of the present application includes:

[0060] Based on program detection data records, a sample program feature information set is obtained, and the relative importance of M detection module detections when different sample program feature information is detected is obtained to obtain multiple sample detection performance requirement parameter sets; using the sample program feature information set and the multiple sample detection performance requirement parameter sets, a detection performance requirement analyzer is constructed to analyze the multiple program feature information, and the mean of the multiple analysis results is calculated to obtain a detection performance requirement parameter set, wherein the detection performance requirement parameter set includes M detection performance requirement parameters; based on the detection performance parameter set and the detection performance requirement parameter set, M module detection performance scores are calculated and obtained, and a basic performance detection result is obtained by weighted calculation based on the detection performance requirement parameter set.

[0061] Based on the program detection data records, a sample program feature information set is obtained. Specifically, all relevant program detection data are collected from historical records. The program detection data records should contain program feature information and the detection results of each detection module. From the collected data, feature information of different sample programs is extracted to form a sample program feature information set, which involves code complexity, component type, database interaction mode, API interface characteristics, etc.

[0062] The relative importance of M detection modules when detecting different sample program feature information is obtained, and multiple sample detection performance requirement parameter sets are obtained. Specifically, the performance of different detection modules on different samples is analyzed to determine the relative importance of M detection modules in different scenarios. For example, if a module is particularly effective in detecting a specific type of security vulnerability, then its importance in such samples is high.

[0063] A detection performance requirement analyzer is constructed by using the sample program feature information set and the multiple sample detection performance requirement parameter sets. Specifically, based on the analysis results, a set of detection performance requirement parameters is determined for each sample program. Data can be expressed in the form of weights of M detection modules in the sample detection. The sample detection performance requirement parameter set reflects the importance differences of the detection modules in a specific context. A detection performance requirement analyzer is designed and implemented by using the sample program feature information set and the multiple sample detection performance requirement parameter sets. The detection performance requirement analyzer can receive new program feature information as input and output corresponding detection performance requirement parameters according to the learned pattern.

[0064] The plurality of program feature information are analyzed, and the mean of the plurality of analysis results is calculated to obtain a detection performance requirement parameter set. Specifically, an analyzer is applied to input the feature information of the program set to be detected into the detection performance requirement analyzer, and the analyzer performs a comprehensive analysis on the features; the analysis results are statistically processed, and the mean of the detection performance requirement parameters is calculated to form a generalized detection performance requirement parameter set, thereby ensuring that the detection performance requirement parameter set can reflect the general requirements of the entire program set.

[0065] According to the detection performance parameter set and the detection performance requirement parameter set, the detection performance scores of M modules are calculated. Specifically, for each detection module, the detection performance score of a single module is calculated according to its detection performance parameters (actual detection effect) and the corresponding detection performance requirement parameters (expected effect). For example, if the detection performance requirement parameter of a module is 0.3, and the actual detection performance parameter is 0.4, then its score is 0.4 / 0.3; according to the size of the M detection performance requirement parameters, the detection performance scores of the M modules are weighted and summed, and the corresponding weights are the respective detection performance requirement parameters, to ensure that modules with high importance have a greater impact on the final result. Furthermore, the result after weighted calculation is used as the basic performance detection result. The basic performance detection result reflects the overall detection efficiency of the detection engine for the current program set. The basic performance detection result can be used to evaluate whether the performance of the detection engine meets expectations and whether certain modules need to be adjusted or optimized.

[0066] Through the above steps, the detection engine is systematically evaluated and optimized to ensure that it can complete security detection tasks efficiently and accurately when facing different sets of programs.

[0067] Further, in combination with the detection recurrence performance parameter, the performance detection result of the detection engine relative to the program set is obtained, and the method of the present application includes:

[0068] According to the detection and reproduction performance parameters, the basic performance detection result is corrected and calculated to obtain a performance detection result.

[0069] Clarifying the detection and reproduction performance parameters means evaluating the consistency and stability of the detection results by analyzing the engine's performance on similar and anti-class test cases, including the similarity of the same test matrix (reflecting the reproducibility of the detection results on similar programs) and the anti-class test similarity (measuring the engine's generalization ability for programs with large differences); ensuring that the basic performance detection results have been obtained through the above steps, which are the original performance scores and overall evaluations of each detection module for the program set before considering the detection reproduction performance.

[0070] Specifically analyzing the similarity between the similar test matrix and the anti-class test matrix to understand the performance difference between the detection engine when detecting similar programs and dissimilar programs will help identify the stability and adaptability of the engine; designing correction rules based on the analysis results of the detection and reproduction performance parameters. For example, if the detection and reproduction performance is high, it indicates that the engine is stable and reliable, so the basic performance results are slightly adjusted or maintained; conversely, if the reproduction performance is low, it indicates that there are unstable factors, and the basic performance score needs to be appropriately lowered or re-evaluated according to the degree of deviation; quantitatively correct the basic performance test results according to the reproduction performance parameters. For example, if it is found that the anti-class detection similarity is too low, a reduction factor may be introduced to adjust the scores of modules that perform poorly in the anti-class test.

[0071] In a feasible implementation method, the corrected basic performance test results are weighted and integrated to obtain comprehensive performance test results that take into account the detection reproduction performance. When weighting, the importance of each module and the influence of the reproduction performance need to be considered again; then, the corrected performance test results are internally verified to ensure that the adjustment logic is reasonable and the results meet expectations, that is, they more accurately reflect the true performance of the detection engine on a specific set of programs; based on the verification feedback, the correction algorithm is fine-tuned until a satisfactory effect is achieved, combined with the detection reproduction performance parameters, to accurately evaluate the performance of the detection engine on a specific set of programs, and provide a scientific basis for subsequent optimization and decision-making.

[0072] In summary, the beneficial effects of the embodiments of the present application are:

[0073] 1. By building an engine performance matrix containing multi-dimensional performance indicators, the performance of the detection engine on different sets of programs is comprehensively evaluated to identify the strengths and weaknesses of the engine.

[0074] 2. Based on the program feature information, intelligently index and select the most relevant test cases, analyze the performance of the detection module by comparing similar and anti-class test cases, and then dynamically optimize the detection strategy.

[0075] 3. Classify and analyze the detection results through algorithm models, identify abnormal patterns, and ensure the accuracy and reliability of the detection results. At the same time, through matrix fusion and normalization processing, improve the calculation accuracy of detection performance parameters, optimize detection speed and resource utilization.

[0076] 4. Calculate the detection reproducibility performance parameters to ensure the stable performance of the detection engine on different program sets, thereby improving the credibility of the detection results.

[0077] 5. The method adopts the method of obtaining a test case library; based on multiple program feature information, traversing and analyzing the similarity with multiple historical test case feature information; indexing and obtaining multiple historical test cases with the greatest similarity as multiple similar test cases, and indexing and obtaining multiple historical test cases with the least similarity as multiple anti-class test cases. Based on the feature information of the program to be detected, the most relevant similar test cases and anti-class test cases with significant differences are efficiently selected from the historical test case library, providing strong support for the performance analysis and optimization of the detection engine.

[0078] In summary, any step can be stored as a computer instruction or program in an unlimited computer memory, and can be called and recognized by an unlimited computer processor, and no unnecessary restrictions are made here.

[0079] Furthermore, the above-mentioned technical scheme only reflects the preferred technical scheme of the technical scheme of the embodiment of the present application. Some changes that may be made to some parts thereof by technicians in this technical field all reflect the novel principles of the embodiment of the present application. Obviously, technicians in this field can make various changes and modifications to the present application without departing from the scope of the present application.

Claims

1. A performance detection method based on an engine matrix module, characterized in that: The method comprises: Acquire a program set that is currently to be security-checked by a detection engine, and collect multiple program feature information of multiple detection programs in the program set, wherein the detection engine includes a detection module matrix, and the detection module matrix includes M detection modules, where M is an integer greater than 1; According to the plurality of program feature information, index and obtain a plurality of similar test cases and a plurality of anti-class test cases; Divide the multiple similar test cases to obtain a first similar test case set and a second similar test case set, and test the detection engine in combination with the multiple anti-class test cases to obtain a first similar test result set, a second similar test result set, and an anti-class test result set, wherein the first similar test result set includes M first similar test result clusters, each first similar test result cluster includes N performance test results, and N is an integer greater than 1; Processing the first similar test result set, the second similar test result set and the reversed test result set, constructing a first similar test matrix, a second similar test matrix and a reversed test matrix, and processing to obtain detection reproducibility performance parameters; The first similar test matrix and the second similar test matrix are merged to obtain a similar test matrix, and performance analysis of the M detection modules is performed to obtain a detection performance parameter set; Analyze and obtain a detection performance requirement parameter set of the program set according to the plurality of program feature information, calculate and obtain a basic performance detection result in combination with the detection performance parameter set, and process and obtain a performance detection result of the detection engine relative to the program set in combination with the detection reproduction performance parameter; The first similar test matrix and the second similar test matrix are merged to obtain a similar test matrix, and the performance analysis of the M detection modules is performed to obtain a detection performance parameter set, including: The means of the elements at the same position in the first similar test matrix and the second similar test matrix are calculated, and matrix fusion is performed to obtain the similar test matrix, which is as follows: ； Among them, R is the same type test matrix, is the first fusion performance test result of the first detection module, is the first fusion performance test result of the Mth detection module, is the Nth fusion performance test result of the first detection module, is the Mth fusion performance test result of the Mth detection module; According to the same type of test matrix, the performance analysis of the M detection modules is performed to obtain a set of detection performance parameters.

2. The method according to claim 1, characterized in that Obtain a set of programs that are currently to be security-checked by the detection engine, and collect multiple program feature information of multiple detection programs in the program set, including: Get the set of programs that are currently to be tested for security by the detection engine; The code feature information, component feature information, database feature information and interface feature information of the plurality of detection programs in the program set are collected as a plurality of program feature information.

3. The method according to claim 1, characterized in that According to the plurality of program feature information, multiple similar test cases and multiple anti-class test cases are obtained by indexing, including: Acquire a test case library, wherein the test case library includes a plurality of historical test cases and a plurality of historical test case feature information of the detection engine; According to the plurality of program feature information, traversing and analyzing similarities with the plurality of historical test case feature information; The index obtains multiple historical test cases with the greatest similarity as multiple similar test cases, and the index obtains multiple historical test cases with the least similarity as multiple anti-class test cases.

4. The method according to claim 1, characterized in that: The first similar test result set, the second similar test result set and the reversed test result set are processed to construct a first similar test matrix, a second similar test matrix and a reversed test matrix, and the detection recurrence performance parameters are obtained by processing, including: Normalizing the data in the first similar test result set, the second similar test result set, and the reversed test result set; Based on the result of the normalization process, a first similar test matrix, a second similar test matrix and an inverse similar test matrix are constructed, wherein the first similar test matrix is ​​as follows: Calculate the similarity between the first similar test matrix and the second similar test matrix to obtain similarity and similar detection reproducibility; Calculating the similarity mean of the first similar test matrix, the second similar test matrix and the reverse class test matrix to obtain reverse class detection similarity; According to the same type of detection reproducibility and the opposite type of detection similarity, the detection reproducibility performance parameter is calculated and obtained.

5. The method according to claim 1, characterized in that: According to the homogeneous test matrix, the performance analysis of the M detection modules is performed, including: According to the same type test matrix, the M detection performance parameters of the M detection modules are calculated to obtain a detection performance parameter set, which is as follows: ； ； ； in, is the detection performance parameter of the i-th detection module among the M detection modules, is the maximum value of the j-th column fusion performance test results, is the jth fusion performance test result of the i-th detection module, is the minimum value of the fusion performance test results in the jth column.

6. The method according to claim 1, characterized in that Analyzing and acquiring a detection performance requirement parameter set of the program set according to the plurality of program feature information, and calculating and acquiring a basic performance detection result in combination with the detection performance parameter set, including: Based on the program detection data record, a sample program feature information set is obtained, and the relative importance of M detection modules when different sample program feature information is detected is obtained to obtain multiple sample detection performance requirement parameter sets; Using the sample program feature information set and the multiple sample detection performance requirement parameter sets, constructing a detection performance requirement analyzer, analyzing the multiple program feature information, calculating the mean of multiple analysis results, and obtaining a detection performance requirement parameter set, wherein the detection performance requirement parameter set includes M detection performance requirement parameters; According to the detection performance parameter set and the detection performance requirement parameter set, the detection performance scores of the M modules are calculated, and the basic performance detection result is obtained by weighted calculation according to the detection performance requirement parameter set.

7. The method according to claim 1, combining the detection recurrence performance parameter, processing and obtaining the performance detection result of the detection engine relative to the program set, comprising: According to the detection and reproduction performance parameters, the basic performance detection result is corrected and calculated to obtain a performance detection result.

Images