Verification method of full-performance automatic assembly line test result

Abstract

The invention relates to a verification method for a full-performance automatic assembly line test result, and the method specifically comprises the steps: collecting original test data from a plurality of sources from an automatic test platform, and carrying out the multi-dimensional normalization processing; extracting local data features of the normalized original data by using a second derivative and a mixed feature weighting formula, and performing hierarchical processing on the local data features through recursive decomposition; performing multi-scale feature mapping on the layered local data features through wavelet transform, and capturing multi-scale features of the local data features of each layer; calculating a matching error between a multi-scale feature mapping result and a standard result through nonlinear matching; a matching degree is converted into a matching degree through a self-adaptive nonlinear function, then the matching degree is converted into local errors, and the local errors are accumulated to construct a preliminary global error; and the initial global error is calibrated by using recursive correction, and the accuracy of the test result is judged according to the calibrated global error.

Description

Technical Field

[0001] The present invention relates to the field of automated testing, and in particular to a method for verifying full-performance automated assembly line test results. Background Art

[0002] As the complexity of modern software systems continues to increase, automated testing has become an important means of ensuring software quality. Automated testing can not only significantly improve testing efficiency, but also cover scenarios and boundary conditions that are difficult to reach with manual testing. Especially in large-scale software development, automated testing can quickly provide feedback on system stability, functionality, and performance issues. However, as the scale of testing expands and the test scenarios become more diverse, the complexity of test results also increases. How to effectively verify the results of automated testing and ensure their accuracy and reliability has become a key challenge in the field of software testing.

[0003] In actual applications, automated test results usually come from multiple different data sources, including functional test modules, performance test tools, and user behavior simulators. These test results not only involve a large number of test items and test dimensions, but also contain complex time series data, load test data, and simulated user behavior data. These data come in various forms, including functional indicators as well as indicators related to performance, reliability, and user experience. How to extract useful information from these multi-source, multi-dimensional data and conduct accurate verification is a difficult problem faced by the automated testing field.

[0004] There are at least the following technical problems in the existing technology: the existing technology lacks sophisticated means for processing test data, which affects the accuracy of the test results, ignores the performance of data at different levels and scales, resulting in insufficient depth and breadth of the test results, and cannot fully reflect the true performance of the system. It is impossible to effectively identify and correct the source of errors, which affects the reliability and effectiveness of the test.

[0005] The Chinese invention patent with publication number CN115757161A discloses an automated test result analysis method and device, which calculates the index value corresponding to the first test result analysis index of the software to be tested based on the identifiers of multiple first business functions, the identifier of the second business function carried by the test result of each test case, and the preset first test result analysis index. According to the identifier of the second business function carried by the test result of each test case, the test result of each test case, and the preset second test result analysis index, the index value corresponding to the second test result analysis index of each second business function is calculated. According to the index value corresponding to the first test result analysis index of the software to be tested and the index value corresponding to the second test result analysis index of each second business function, the automated test analysis result of the software to be tested is determined.

[0006] The above existing methods use multiple test samples to analyze the automated test results from the overall level of software business functions and the level of individual business functions, and improve the accuracy of test results through multi-dimensional analysis. However, it ignores that within each dimension, test data of different scales also perform differently. How to analyze the test results more comprehensively and find the source of errors more efficiently. Summary of the invention

[0007] In order to solve the above problems existing in the prior art, the present invention proposes a method for verifying the test results of a full-performance automated assembly line.

[0008] The technical solution of the present invention is as follows: On the one hand, the present invention provides a method for verifying the test results of a full-performance automated assembly line, the specific steps of which include: Collect raw test data from multiple sources from the automated testing platform and perform multi-dimensional normalization processing; The local data features of the original data after normalization are extracted using the second-order derivative and the mixed feature weighting formula, and the local data features are hierarchically processed through recursive decomposition; Multi-scale feature mapping is performed on the local data features after stratification through wavelet transform to capture the multi-scale features of local data features at each level; Calculate the matching error between the multi-scale feature mapping result and the standard result through nonlinear matching; The matching degree is converted into matching degree through an adaptive nonlinear function, and then the matching degree is converted into a local error, and the local errors are accumulated to construct a preliminary global error; The preliminary global error is calibrated using recursive correction, and the accuracy of the test results is judged based on the calibrated global error.

[0009] As a preferred implementation, the sources of the original test data include a functional test module, a performance test tool and a user behavior simulator.

[0010] As a preferred implementation, the multidimensional normalization process is specifically as follows:

[0011] In the formula, The data is processed by multidimensional normalization; is the original test result data matrix; i is the i-th test item, j is the j-th test dimension; and Represents the data matrix The minimum and maximum values ​​in .

[0012] As a preferred implementation, the second-order derivative and mixed feature weighting formula is specifically:

[0013] In the formula, The extracted local feature value represents the local feature extracted on the i-th test item and the j-th test dimension; and are the i-th test item and the j-th test dimension respectively; is the normalized data matrix The second-order derivatives are respectively and Conduct partial guidance; is the dynamic weighting coefficient; is the weight coefficient of the Euclidean distance term; Indicates that the i-th test item is in The normalized results on the feature dimensions, that is, the performance of the test items on multiple feature dimensions; Indicates that the jth test dimension is in The normalized results on feature dimensions are the performance on different feature dimensions under different test conditions; m is the number of feature dimensions.

[0014] As a preferred implementation, the decomposition formula of the step of performing hierarchical processing on local data features by recursive decomposition is as follows:

[0015] In the formula, Indicates The decomposition result of the layer, n represents the A specific local data feature in the layer, m represents the number of feature dimensions; represents the weight function; is the mean value in the Gaussian function, representing the central position of the local eigenvalue at the kth level, is the standard deviation of the Gaussian function, indicating the extent of expansion of the local eigenvalues ​​at the kth level, represents the Gaussian function.

[0016] As a preferred implementation, the step of performing multi-scale feature mapping on the layered local data features by wavelet transform is specifically as follows:

[0017] In the formula, For the The local data features of the layer are Next, at coordinates The wavelet transform coefficients at is the position coordinate of the sampling point in two-dimensional space, is the scale factor, is the mother wavelet function, Represents spatial coordinates.

[0018] As a preferred implementation, the step of calculating the matching error between the multi-scale feature mapping result and the standard result by nonlinear matching is specifically as follows:

[0019] In the formula, Indicates The characteristics of the layer are in scale Next, at coordinates The error between the wavelet transform coefficient at and the standard result is, is the standard result of historical test data.

[0020] As a preferred implementation, the step of converting the matching degree into the matching degree by using an adaptive nonlinear function is specifically as follows:

[0021] In the formula, is the matching degree, which indicates the matching result at the kth level and scale s; is the adaptive parameter in nonlinear matching; is the scaling parameter in nonlinear matching; and are the number of sampling points in the horizontal and vertical directions respectively.

[0022] As a preferred implementation, the steps of converting the matching degree into a local error and accumulating the local errors to construct a preliminary global error are specifically as follows: The calculation formula for converting the matching degree into the local error is as follows:

[0023] In the formula, is the local error value, which indicates the error aggregation result at the kth level and scale s. is a constant to prevent the denominator from being 0; The specific calculation formula of the preliminary global error is as follows:

[0024] In the formula, is the preliminary global error value, which represents the cumulative result of all local errors; is the weighting coefficient.

[0025] As a preferred implementation, the step of calibrating the preliminary global error by recursive correction is specifically as follows:

[0026] In the formula, is the final global error value, is the global error value calculated in the previous round, is the recursive correction factor.

[0027] The present invention has the following beneficial effects: 1. By introducing multidimensional normalization processing and local feature extraction, the present invention can effectively eliminate noise and outliers in the test data and improve the accuracy and precision of data analysis. In particular, through the second-order derivative and mixed feature weighting formula, the high-order change characteristics of the data are captured, making the performance of the test data in different dimensions more accurate and comprehensive; 2. The present invention realizes multi-level and multi-scale analysis of test data through recursive decomposition and multi-scale feature mapping. The hierarchical processing not only enables each sub-result to be analyzed independently, but also extracts the characteristic information of the data at different scales through wavelet transform, making the analysis process more detailed and comprehensive. This multi-level and multi-scale analysis method can capture subtle changes in test data at different levels, greatly improving the depth and breadth of data analysis.

[0028] 3. The present invention uses an adaptive nonlinear function to process matching errors. When evaluating the deviation between the test results and the standard results, the present invention can more accurately reflect the complex relationship between the data; the nonlinear matching degree calculation combines the adaptive adjustment of multiple parameters, which greatly improves the flexibility and accuracy of the matching analysis and ensures the accuracy of the error evaluation; 4. The present invention not only calculates the global error, but also ensures the gradual convergence of the error through the aggregation and recursive correction of local errors, thereby improving the reliability of the global error; through detailed error decomposition and source tracing, developers can clearly understand the source and distribution of the error, thereby providing direct guidance for the optimization of the test process. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0032] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0033] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0034] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0035] Embodiment 1: See also Figure 1 , a method for verifying the test results of a full-performance automated pipeline, the specific steps include: Collect raw test data from multiple sources from the automated testing platform and perform multi-dimensional normalization processing; In this embodiment, the data collection process must ensure the integrity and consistency of the data, so a unified timestamp and unique identifier are used to associate data items from different sources. The collected data is initially organized and archived in a structured manner for subsequent processing and analysis.

[0036] The local data features of the original data after normalization are extracted using the second-order derivative and the mixed feature weighting formula, and the local data features are hierarchically processed through recursive decomposition; Multi-scale feature mapping is performed on the local data features after stratification through wavelet transform to capture the multi-scale features of local data features at each level; Calculate the matching error between the multi-scale feature mapping result and the standard result through nonlinear matching; The matching degree is converted into matching degree through an adaptive nonlinear function, and then the matching degree is converted into a local error, and the local errors are accumulated to construct a preliminary global error; The preliminary global error is calibrated using recursive correction, and the accuracy of the test results is judged based on the calibrated global error.

[0037] In this embodiment, after the final global error calculation is completed, the error needs to be analyzed in detail and a verification report is generated. The source of each local error in the final global error value is traced, and the performance of each local error in each data dimension and scale is analyzed to understand the distribution of errors in different data dimensions, scales and levels; the results of the error distribution are visualized so that developers can more intuitively understand the distribution and source of the errors, and visualization techniques such as heat maps and three-dimensional surface maps are used to display the distribution of errors. Through visualization, developers can clearly see the distribution patterns and abnormal areas of errors in the test data, thereby providing a direct basis for the next step of improvement.

[0038] As a preferred implementation of this embodiment, the sources of the original test data include a functional test module, a performance test tool and a user behavior simulator.

[0039] In this embodiment, test data is collected from multiple sources through an automated testing platform, including a functional testing module, a performance testing tool, and a user behavior simulator. Each data source is collected in a different way. For example, the functional testing module generates specific function execution results through test scripts, the performance testing tool records the response time, throughput, and resource usage of the system under a specific load, and the user behavior simulator simulates real user operations to capture performance indicators related to user experience.

[0040] As a preferred implementation of this embodiment, the multidimensional normalization process is specifically as follows:

[0041] In the formula, The data is processed by multidimensional normalization; is the original test result data matrix; i is the i-th test item, j is the j-th test dimension; and Represents the data matrix The minimum and maximum values ​​in .

[0042] In this embodiment, the original test result data matrix obtained from the automated testing platform needs to be initially parsed to eliminate data noise and outliers, and the data is standardized through a multidimensional normalization formula to ensure that subsequent analysis will not be distorted due to differences in data magnitude.

[0043] Normalize the test data to eliminate the magnitude differences between the data so that different data points can be compared more effectively in subsequent analysis. The test item refers to the specific object or function tested in the automated testing process, which can be a module, a function, a user operation scenario, or an input-output combination in the software system; each test item represents a specific test instance, which is evaluated for a specific function or performance indicator. For example: in a web application test, the test items may be login functions, form submissions, page loading time, etc. The test dimension refers to the different angles or characteristics considered when evaluating the test items. The test dimensions can be functional indicators, performance indicators, reliability indicators, usability indicators, security indicators, etc. Each test dimension measures the performance of the test item from a different angle. For example: Time dimension: evaluate the response time, processing time, delay, etc. of a test item.

[0044] As a preferred implementation of this embodiment, the second-order derivative and mixed feature weighting formula is specifically:

[0045] In the formula, The extracted local feature value represents the local feature extracted on the i-th test item and the j-th test dimension; and are the i-th test item and the j-th test dimension respectively; is the normalized data matrix The second-order derivatives are respectively and Conduct partial guidance; is the dynamic weighting coefficient; is the weight coefficient of the Euclidean distance term; Indicates that the i-th test item is in The normalized results on the feature dimensions, that is, the performance of the test items on multiple feature dimensions; Indicates that the jth test dimension is in The normalized results on feature dimensions are the performance on different feature dimensions under different test conditions; m is the number of feature dimensions.

[0046] In this embodiment, the second-order derivatives are respectively and Partial derivatives are used to capture high-order variation features and ensure that the complexity of the data is fully utilized; is a dynamic weighting coefficient that controls the contribution of local features, is the weight coefficient of the Euclidean distance term, which is used to measure the local differences in different dimensions. By accurately capturing the local features of the data, the characteristics of different data points can be identified in the subsequent hierarchical processing.

[0047] The above-mentioned characteristic dimension m reflects the inherent change law of data in different dimensions, such as data change trend, stability or correlation.

[0048] As a preferred implementation of this embodiment, the decomposition formula of the step of performing hierarchical processing on local data features by recursive decomposition is as follows:

[0049] In the formula, Indicates The decomposition result of the layer, n represents the A specific local data feature in the layer, m represents the number of feature dimensions; represents the weight function; is the mean value in the Gaussian function, representing the central position of the local eigenvalue at the kth level, is the standard deviation of the Gaussian function, indicating the extent of expansion of the local eigenvalues ​​at the kth level, represents the Gaussian function.

[0050] In this embodiment, It is a Gaussian function, which is used to smooth the distribution of eigenvalues ​​to ensure that the decomposed data can better reflect the structural characteristics of different levels.

[0051] By decomposing the data layer by layer, the features of each layer are made more prominent, which facilitates subsequent multi-scale analysis.

[0052] As a preferred implementation of this embodiment, the step of performing multi-scale feature mapping on the layered local data features by wavelet transform is specifically as follows:

[0053] In the formula, For the The local data features of the layer are Next, at coordinates The wavelet transform coefficients at is the position coordinate of the sampling point in two-dimensional space, is the scale factor, is the mother wavelet function, Represents spatial coordinates.

[0054] In this embodiment, a multi-scale analysis method is used to extract feature information of data at different scales, thereby making subsequent matching analysis more comprehensive and accurate.

[0055] As a preferred implementation of this embodiment, the step of calculating the matching error between the multi-scale feature mapping result and the standard result by nonlinear matching is specifically as follows:

[0056] In the formula, Indicates The characteristics of the layer are in scale Next, at coordinates The error between the wavelet transform coefficient at and the standard result is, is the standard result of historical test data.

[0057] In this embodiment, the standard results are obtained through the accumulation of historical test data, which are reliable test results collected under a long-term and stable test environment. The results that have been repeatedly verified are selected as the standard as a reference under different test conditions.

[0058] As a preferred implementation of this embodiment, the step of converting the matching degree into the matching degree by using an adaptive nonlinear function is specifically as follows:

[0059] In the formula, is the matching degree, which indicates the matching result at the kth level and scale s; is the adaptive parameter in nonlinear matching; is the scaling parameter in nonlinear matching; and are the number of sampling points in the horizontal and vertical directions respectively.

[0060] In this embodiment, is an adaptive parameter in nonlinear matching, which controls the influence of the error term on the matching degree. It is a scaling parameter in nonlinear matching, which adjusts the gradient of matching calculation.

[0061] Errors are processed through nonlinear functions to ensure that the matching calculation can effectively reflect the complex relationship between data, thus providing an accurate basis for error assessment.

[0062] As a preferred implementation of this embodiment, the steps of converting the matching degree into a local error and accumulating the local errors to construct a preliminary global error are specifically as follows: The calculation formula for converting the matching degree into the local error is as follows:

[0063] In the formula, is the local error value, which indicates the error aggregation result at the kth level and scale s. is a constant to prevent the denominator from being 0; The specific calculation formula of the preliminary global error is as follows:

[0064] In the formula, is the preliminary global error value, which represents the cumulative result of all local errors; is the weighting coefficient.

[0065] In this embodiment, the local error captures the subtle errors in the matching process through squared errors and logarithmic functions, and aggregates the multi-scale matching errors into a more representative local error indicator for further accumulation and calculation of the global error.

[0066] Weighting coefficient Used to reflect the contribution of each sub-result to the overall error.

[0067] As a preferred implementation of this embodiment, the step of calibrating the preliminary global error by recursive correction is specifically as follows:

[0068] In the formula, is the final global error value, is the global error value calculated in the previous round, is the recursive correction factor.

[0069] In this embodiment, in order to correct the errors in the matching, a recursive correction term is introduced, and the error results of the previous round of calculations are brought into the current calculations to obtain the final global error value; the recursive correction ensures that the error gradually converges and reflects the actual deviation of the test results. By gradually correcting the errors, the accuracy and reliability of the global error are improved, thereby ensuring the accuracy of the final test result evaluation.

[0070] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for verifying the test results of a full-performance automated assembly line, characterized in that: The specific steps include: Collect raw test data from multiple sources from the automated testing platform and perform multi-dimensional normalization processing; The local data features of the original data after normalization are extracted using the second-order derivative and the mixed feature weighting formula, and the local data features are hierarchically processed through recursive decomposition; Multi-scale feature mapping is performed on the local data features after stratification through wavelet transform to capture the multi-scale features of local data features at each level; Calculate the matching error between the multi-scale feature mapping result and the standard result through nonlinear matching; The matching degree is converted into matching degree through an adaptive nonlinear function, and then the matching degree is converted into a local error, and the local errors are accumulated to construct a preliminary global error; The preliminary global error is calibrated using recursive correction, and the accuracy of the test results is judged based on the calibrated global error.

2. A method for verifying the test results of a full-performance automated assembly line according to claim 1, characterized in that: The sources of the original test data include functional test modules, performance test tools and user behavior simulators.

3. The method for verifying the test results of a full-performance automated assembly line according to claim 1, characterized in that: The multidimensional normalization process is specifically as follows: In the formula, The data is processed by multidimensional normalization; is the original test result data matrix; i is the number of test items, j is the number of test dimensions; and Represents the data matrix The minimum and maximum values ​​in .

4. A method for verifying the test results of a full-performance automated assembly line according to claim 3, characterized in that: The second-order derivative and mixed feature weighting formula are specifically: In the formula, The extracted local feature value represents the local feature extracted on the i-th test item and the j-th test dimension; and are the i-th test item and the j-th test dimension respectively; is the normalized data matrix The second-order derivatives are respectively and Conduct partial guidance; is the dynamic weighting coefficient; is the weight coefficient of the Euclidean distance term; Indicates that the i-th test item is in The normalized results on the feature dimensions, that is, the performance of the test items on multiple feature dimensions; Indicates that the jth test dimension is in The normalized results on feature dimensions are the performance on different feature dimensions under different test conditions; m is the number of feature dimensions.

5. A method for verifying the test results of a full-performance automated assembly line according to claim 4, characterized in that: The decomposition formula of the step of performing hierarchical processing on local data features by recursive decomposition is as follows: In the formula, Indicates The decomposition result of the layer, n represents the A specific local data feature in the layer; m represents the number of feature dimensions; represents the weight function; is the mean value in the Gaussian function, representing the central position of the local eigenvalue at the kth level, is the standard deviation of the Gaussian function, indicating the extent of expansion of the local eigenvalues ​​at the kth level, represents the Gaussian function.

6. The method for verifying the test results of a full-performance automated assembly line according to claim 1, characterized in that: The step of performing multi-scale feature mapping on the layered local data features by wavelet transform is specifically as follows: In the formula, For the The local data features of the layer are Next, at coordinates The wavelet transform coefficients at is the position coordinate of the sampling point in two-dimensional space, is the scale factor, is the mother wavelet function, Represents spatial coordinates.

7. A method for verifying the test results of a full-performance automated assembly line according to claim 6, characterized in that: The step of calculating the matching error between the multi-scale feature mapping result and the standard result by nonlinear matching is specifically as follows: In the formula, Indicates The characteristics of the layer are in scale Next, at coordinates The error between the wavelet transform coefficient at and the standard result is, is the standard result of historical test data.

8. The method for verifying the test results of a full-performance automated assembly line according to claim 1, characterized in that: The step of converting the matching degree into the matching degree by using the adaptive nonlinear function is specifically as follows: In the formula, is the matching degree, which indicates the matching result at the kth level and scale s; is the adaptive parameter in nonlinear matching; is the scaling parameter in nonlinear matching; and are the number of sampling points in the horizontal and vertical directions respectively.

9. A method for verifying the test results of a full-performance automated assembly line according to claim 8, characterized in that: The steps of converting the matching degree into local errors and accumulating the local errors to construct the preliminary global error are specifically as follows: The calculation formula for converting the matching degree into the local error is as follows: In the formula, is the local error value, which indicates the error aggregation result at the kth level and scale s. is a constant to prevent the denominator from being 0; The specific calculation formula of the preliminary global error is as follows: In the formula, is the preliminary global error value, which represents the cumulative result of all local errors; is the weighting coefficient.

10. The method for verifying the test results of a full-performance automated assembly line according to claim 1, characterized in that: The step of calibrating the preliminary global error by recursive correction is specifically as follows: In the formula, is the final global error value, is the global error value calculated in the previous round, is the recursive correction factor.

Images