Detection method and device for regression test based on application program

By analyzing application log and waveform information to generate a test anomaly information table, and combining it with coverage data for detection, the problem of low efficiency in application regression testing is solved, achieving fast and efficient detection.

CN119025415APending Publication Date: 2024-11-26SHANDONG YUNHAI GUOCHUANG CLOUD COMPUTING EQUIP IND INNOVATION CENT CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411046212.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In existing technologies, regression testing of applications is inefficient, resulting in long testing times and significant waste of resources.

Method used

By acquiring log and waveform information during test case execution, the log information is analyzed using a target classification model to generate a test anomaly information table. This is then combined with a coverage data table for detection, and the weight distribution of test cases is adjusted to optimize regression testing.

Benefits of technology

It reduces the waste of simulation resources and time, improves the efficiency and accuracy of application verification, and enables rapid detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025415A_ABST
    Figure CN119025415A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a detection method and device for a regression test based on an application program, and the method comprises the steps: obtaining log information generated by executing a test case in a process of executing the regression test on a target application program through the test case; determining a test exception information table of the target application program based on the log information, the test exception information table including target exception information corresponding to N target exception categories, and N being a natural number greater than or equal to 1; determining a coverage rate data table of the test case in the process of performing regression testing on the target application program by using the test case; and detecting the regression test executed by the target application program by using the test exception information table and the coverage rate data table. Through the application, the problem of low detection efficiency of the application program in related technologies is solved, and the effect of rapidly detecting the application program is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the computer field, and in particular, to a detection method and device based on regression testing of an application program. Background Art

[0002] Application verification is a crucial step in the design and manufacturing process of integrated circuits, which ensures the correctness of the design and the integrity of the functions. The key test scenarios in the application are tested through a large number of simulations and simulations. In the prior art, defects in the application design can usually be discovered by running a large number of test cases to cover different detection scenarios. However, for complex designs, the detection of a complex scenario may require the regression of thousands of test cases, which makes detection more and more challenging. In most detection processes, regression testing and coverage collection and analysis of a large number of test cases take a long time, resulting in low detection efficiency. Summary of the invention

[0003] The embodiments of the present application provide a detection method and device based on regression testing of an application program, so as to at least solve the problem of low detection efficiency of the application program in the related art.

[0004] According to one embodiment of the present application, a detection method based on application regression testing is provided, comprising: in the process of performing regression testing on a target application using a test case, obtaining log information generated by executing the test case; determining a test exception information table of the target application based on the log information, wherein the test exception information table includes target exception information corresponding to N target exception categories, and N is a natural number greater than or equal to 1; in the process of performing regression testing on the target application using the test case, determining a coverage data table of the test case, wherein the coverage data table includes test coverage information of executing the regression test on the target application, and the test coverage information is used to indicate the degree of testing of the target application; and detecting the regression test executed on the target application using the test exception information table and the coverage data table.

[0005] In an exemplary embodiment, determining a test exception information table of the target application based on the log information includes: performing structured processing on the log information to obtain structured log information; inputting the structured log information into a target classification model to obtain the test exception information table output by the target classification model; wherein the target classification model is a model obtained by training an initial classification model using a plurality of training log information, the initial classification model includes a parsing subnetwork, a clustering subnetwork, and a classification subnetwork, for each of the training log information, the parsing subnetwork is used to perform word segmentation processing on the training log information to obtain a training word segmentation list, the clustering subnetwork is used to cluster the training word segmentation list to obtain K word segmentation categories, and the classification subnetwork is used to determine the training log information based on the K word segmentation categories. A group is formed, and a training exception information table is generated based on the training log group, wherein the training log group includes exception information of M exception categories, wherein K is a natural number greater than or equal to 1, and M is a natural number less than or equal to K. The training log information is information of the same type as the structured log information, and the training exception information table includes exception information of the M exception categories; the target classification model also includes a storage subnetwork, wherein a database in the storage subnetwork stores preset exception information corresponding to multiple preset word segmentation categories, and the storage subnetwork is connected to the classification subnetwork, and is used for storing other exception information corresponding to other exception categories in the database when M is less than K, wherein the other exception categories are exception categories among the K word segmentation categories except the word segmentation categories matching the M exception categories.

[0006] In an exemplary embodiment, after determining the test exception information table of the target application based on the log information, the method further includes: in the process of performing regression testing on the target application using the test case, obtaining waveform information generated by executing the test case, wherein the waveform information is used to represent signal changes occurring during the regression testing; searching for an abnormal waveform matching the N target exception categories included in the test exception information table from the waveform information; searching for an abnormal log matching the N target exception categories from the log information; and locating the abnormality of performing the regression test on the target application based on the abnormal waveform and the abnormal log.

[0007] In an exemplary embodiment, in the process of performing regression testing on the target application using the test case, determining the coverage data table of the test case includes: in the process of performing regression testing on the target application using the test case, storing the acquired test information into a test case information table, wherein the test case information includes at least one of the following information: information of the test case, a test path of the regression test, the number of times the regression test is executed, and coverage file information; extracting coverage information under multiple coverage dimensions from the test case information table; and merging the coverage information under the multiple coverage dimensions to obtain the coverage data table.

[0008] In an exemplary embodiment, during the process of performing regression testing on the target application using the test case, after determining the coverage data table of the test case, the method further includes: determining the code coverage, requirement coverage and use case coverage of the test case within a first preset time period based on the coverage data included in the coverage data table, wherein the code coverage is used to indicate the proportion of the code in the target application covered by the test case executed within the first preset time period, the requirement coverage is used to indicate the proportion of the requirements of the target application covered by the test case, and the use case coverage is used to indicate the ratio between the number of test cases executed within the first preset time period and the total number of test cases in the regression test; and issuing prompt information using the code coverage, requirement coverage and use case coverage, wherein the prompt information is used to prompt adjustment of the test case.

[0009] In an exemplary embodiment, the regression test performed on the target application is detected using the test exception information table and the coverage data table, including: using the test exception information table to locate the test exception of the test case in the regression test to obtain a first exception; using the coverage data table to detect the coverage exception of the test case in the regression test to obtain a second exception; and performing exception detection on the regression test performed on the target application through the test case based on the first exception and the second exception.

[0010] In an exemplary embodiment, after performing anomaly detection on the regression test performed on the target application using the test case based on the first anomaly and the second anomaly, the method further includes: extracting anomaly parameters of the test case from the first anomaly, wherein the anomaly parameters include the number of anomalies and the level of anomalies; extracting a coverage growth rate of performing the regression test on the target application using the test case within a second preset time period from the second anomaly; and adjusting the weight distribution of the test case in the regression test based on the anomaly parameters and the coverage growth rate.

[0011] In an exemplary embodiment, the weight distribution of the test case in the regression test is adjusted based on the exception parameters and the coverage growth rate, including: when the number of exceptions is greater than a preset number of exceptions, the exception level is greater than a preset level, and the coverage growth rate is greater than a preset growth rate, the distribution weight of the test case is adjusted to a first weight; when the number of exceptions is less than a preset number of exceptions, the exception level is less than a preset level, and the coverage growth rate is less than a preset growth rate, the distribution weight of the test case is adjusted to a second weight, wherein the first weight is greater than the second weight.

[0012] According to another embodiment of the present application, a detection device based on regression testing of an application is provided, comprising: a first acquisition module, used to acquire log information generated by executing the test case in the process of performing regression testing on the target application using the test case; a first determination module, used to determine a test exception information table of the target application based on the log information, wherein the test exception information table includes target exception information corresponding to N target exception categories, and N is a natural number greater than or equal to 1; a second determination module, used to determine a coverage data table of the test case in the process of performing regression testing on the target application using the test case, wherein the coverage data table includes test coverage information of executing the regression test on the target application, and the test coverage information is used to indicate the degree of testing of the target application; a first detection module, used to detect the regression test executed on the target application using the test exception information table and the coverage data table.

[0013] In an exemplary embodiment, the first determination module includes: a first processing unit, which is used to perform structured processing on the log information to obtain structured log information; a first input unit, which is used to input the structured log information into a target classification model to obtain the test exception information table output by the target classification model; wherein the target classification model is a model obtained by training an initial classification model using multiple training log information, and the initial classification model includes a parsing subnetwork, a clustering subnetwork and a classification subnetwork. For each of the training log information, the parsing subnetwork is used to perform word segmentation processing on the training log information to obtain a training word segmentation list, the clustering subnetwork is used to cluster the training word segmentation list to obtain K word segmentation categories, and the classification subnetwork is used to determine the training log information based on the K word segmentation categories. A group is formed, and a training exception information table is generated based on the training log group, wherein the training log group includes exception information of M exception categories, wherein K is a natural number greater than or equal to 1, and M is a natural number less than or equal to K. The training log information is information of the same type as the structured log information, and the training exception information table includes exception information of the M exception categories; the target classification model also includes a storage subnetwork, wherein a database in the storage subnetwork stores preset exception information corresponding to multiple preset word segmentation categories, and the storage subnetwork is connected to the classification subnetwork, and is used for storing other exception information corresponding to other exception categories in the database when M is less than K, wherein the other exception categories are exception categories among the K word segmentation categories except the word segmentation categories matching the M exception categories.

[0014] In an exemplary embodiment, the above-mentioned device also includes: a second acquisition module, which is used to determine the test exception information table of the target application based on the log information, and then, in the process of performing regression testing on the target application using the test case, obtain waveform information generated by executing the test case, wherein the waveform information is used to represent signal changes occurring during the regression testing process; a first search module, which is used to search for abnormal waveforms matching the N target exception categories included in the test exception information table from the waveform information; a second search module, which is used to search for abnormal logs matching the N target exception categories from the log information; and a first positioning module, which is used to locate the anomaly of executing the regression test on the target application based on the abnormal waveform and the abnormal log.

[0015] In an exemplary embodiment, the above-mentioned second determination module includes: a first storage unit, used to store the acquired test information in a test case information table during the process of performing regression testing on the target application using the test case, wherein the test case information includes at least one of the following information: information of the test case, the test path of the regression test, the number of times the regression test is executed, and coverage file information; a first extraction unit, used to extract coverage information under multiple coverage dimensions from the test case information table; a first merging unit, used to merge the coverage information under the multiple coverage dimensions to obtain the coverage data table.

[0016] In an exemplary embodiment, the above-mentioned device also includes: a third determination module, which is used to determine the coverage data table of the test case during the process of performing regression testing on the target application using the test case, and then determine the code coverage, requirement coverage and use case coverage of the test case within a first preset time period based on the coverage data included in the coverage data table, wherein the code coverage is used to indicate the proportion of the code in the target application covered by the test case executed within the first preset time period, the requirement coverage is used to indicate the proportion of the requirements of the target application covered by the test case, and the use case coverage is used to indicate the ratio between the number of test cases executed within the first preset time period and the total number of test cases in the regression test; a first prompt module, which is used to issue prompt information using the code coverage, requirement coverage and use case coverage, wherein the prompt information is used to prompt the adjustment of the test case.

[0017] In an exemplary embodiment, the above-mentioned first detection module includes: a first positioning unit, used to use the test exception information table to locate the test exception of the test case in the regression test to obtain a first exception; a first detection unit, used to use the coverage data table to detect the coverage exception of the test case in the regression test to obtain a second exception; a second detection unit, used to perform anomaly detection on the regression test performed on the target application through the test case based on the first exception and the second exception.

[0018] In an exemplary embodiment, the above-mentioned device also includes: a first extraction module, which is used to extract the abnormal parameters of the test case from the first abnormality after performing abnormality detection on the regression test performed on the target application through the test case based on the first abnormality and the second abnormality, wherein the abnormal parameters include the number of abnormalities and the abnormality level; a second extraction module, which is used to extract the coverage growth rate of performing the regression test on the target application using the test case within a second preset time period from the second abnormality; and a first adjustment module, which is used to adjust the weight distribution of the test case in the regression test based on the abnormal parameters and the coverage growth rate.

[0019] In an exemplary embodiment, the above-mentioned first adjustment module includes: a first adjustment unit, used to adjust the distribution weight of the test case to a first weight when the number of exceptions is greater than a preset number of exceptions, the exception level is greater than a preset level, and the coverage growth rate is greater than a preset growth rate; a second adjustment unit, used to adjust the distribution weight of the test case to a second weight when the number of exceptions is less than a preset number of exceptions, the exception level is less than a preset level, and the coverage growth rate is less than a preset growth rate, wherein the first weight is greater than the second weight.

[0020] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0021] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0022] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0023] Through this application, since the log information in the process of executing regression testing is used as the test exception information table of the target application, and the coverage data table of the test cases in the regression testing process is statistically calculated, the regression test executed by the target application is finally detected using the test exception information table and the coverage data table. It can reduce the waste of simulation resources and time, and accelerate the convergence of application verification. Therefore, it can solve the problem of low detection efficiency of applications in related technologies and achieve the effect of rapid detection of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The hardware structure frame of the server device of the detection method based on the regression test of the application program in the embodiment of the present application is Figure 2

[0025] Figure 2 The process of the detection method based on the regression test of the application program according to the embodiment of the present application is as follows Figure 2

[0026] Figure 3 The following is a flow chart of a specific process of performing regression testing on a target application according to an embodiment of the present application. Figure 2

[0027] Figure 4 The process of collecting coverage and analyzing coverage according to the embodiment of the present application is as follows Figure 2

[0028] Figure 5 The process of performing regression testing on a chip according to the specific embodiment of the present application is as follows Figure 2

[0029] Figure 6 It is a structural block diagram of a detection device based on application program regression testing according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0032] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a detection method based on application program regression testing in an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 The server device includes a processor 102 (only one is shown) (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0033] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the detection method based on regression testing of application programs in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] In this embodiment, a detection method based on regression testing of an application is provided. Figure 2 is a flow chart of a detection method based on regression testing of an application program according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0036] Step S202, in the process of executing regression testing on the target application using the test case, obtaining log information generated by executing the test case;

[0037] Optionally, regression testing is to retest the tested software after the software code changes to confirm whether the new changes affect the original functions of the software. The detection method based on regression testing of the application provided in this embodiment is applicable to any scenario where the stability and correctness of the software function need to be confirmed. For example, when the application is updated, regression testing is required to ensure that the new version of the software does not introduce new errors or cause the old functions to fail. When new functions are added to the application, regression testing is required to ensure that the addition of new functions does not affect the normal operation of existing functions. When defects in the application are fixed, regression testing is required to ensure that the defect repair does not introduce new errors or cause other functions to fail. When the application operating environment changes, such as operating system upgrades or hardware replacements, regression testing is required to ensure the normal operation of the application in the new environment. When the data processed by the application changes, regression testing is required to ensure that the application can correctly handle the new data situation.

[0038] Optionally, the target application in this embodiment includes but is not limited to modules that implement various functions in the chip.

[0039] Optionally, after starting the regression test of the test case, the data source of each test will send a print log in real time, and the log information can be obtained. The log information includes but is not limited to the following information: test case name, test time, test status (recording the execution result of the test case, such as success (Pass), failure (Fail), error (Error) or skip (Skip)), error information (if the test case fails or an error occurs, record detailed error information, including error type, error message and stack trace), test input, test output, system information, environment variables, performance indicators, dependencies, log level, user operation, database operation, network communication, resource usage, test execution path, test report. These log information can be adjusted and customized according to the needs and scenarios of the test. By effectively recording and analyzing logs, the quality and efficiency of the test can be improved, and problems can be discovered and solved in a timely manner.

[0040] Step S204, determining a test exception information table of the target application based on the log information, wherein the test exception information table includes target exception information corresponding to N target exception categories, where N is a natural number greater than or equal to 1;

[0041] Optionally, the target abnormal information included in the test abnormal information table is information extracted from the log information in the above-mentioned process. For example, error information in the process is extracted from the log information, and the error information includes error type, simulation stage, error flag, error number, system information of specific error, etc.

[0042] Step S206, in the process of performing regression testing on the target application using the test case, determining a coverage data table of the test case, wherein the coverage data table includes test coverage information of performing regression testing on the target application, and the test coverage information is used to indicate the degree of testing of the target application;

[0043] Optionally, the coverage data table is a database table that stores test coverage information when the target application performs regression testing. Test coverage information includes, but is not limited to, the following information: code coverage, functional coverage, conditional coverage, etc. For example, the coverage file of the merged version (Merge version refers to the process of merging codes of different versions or branches together. In the software development process) is parsed, and the simulation tool (Verification Component Suite, referred to as VCS) is used to read the coverage information, which mainly includes values ​​such as module name, line coverage, FSM coverage, flip coverage, conditional coverage, and assertion coverage.

[0044] Step S208: Use the test exception information table and the coverage data table to detect the regression test executed on the target application.

[0045] Optionally, regression testing is performed on the target application, mainly testing the following aspects: Verify that the basic functions of the application are still functioning properly, including the user interface, operation process, data processing, etc. Check the performance of the application under high load, high concurrency, etc., to ensure that the performance is not negatively affected. Ensure that the application can operate normally in different operating systems, browsers, devices, etc. Check the security of the application, including data protection, anti-tampering, anti-attack, etc. Verify the consistency, accessibility and ease of use of the user interface to ensure that users can operate smoothly during use. Check the integrity, consistency and performance of the database to ensure that operations such as data storage, query and update are normal. Verify that the integration of the application with external systems or services is normal, including data exchange, interface calls, etc. Check the performance of the application under different configurations to ensure that it can operate normally in various environments. Verify the response and handling mechanism of the application when encountering errors or abnormal situations. This can reduce the risk of the application and ensure the stability and reliability of the software. Problems that may be missed during the development process can be discovered and repaired in a timely manner.

[0046] Among them, the executor of the above steps can be a terminal, a server, a specific processor set in the terminal or server, or a processor or processing device set relatively independently from the terminal or server, or it can be the pool robot itself, etc., but is not limited to this.

[0047] The execution order of step S204 and step S206 is interchangeable, that is, step S206 may be executed first, and then step S204.

[0048] Through the above steps, since the log information in the process of executing the regression test is used as the test exception information table of the target application, and the coverage data table of the test cases in the regression test process is statistically calculated, the regression test executed by the target application is finally detected using the test exception information table and the coverage data table. This can reduce the waste of simulation resources and time, and accelerate the convergence of application verification. Therefore, the problem of low detection efficiency of the application in the related technology can be solved, and the effect of rapid detection of the application can be achieved.

[0049] In an exemplary embodiment, determining the test exception information table of the target application based on the log information includes:

[0050] Step S302, performing structured processing on the log information to obtain structured log information;

[0051] Optionally, this embodiment can use Python regular expressions to structure the log information. The main purpose is to convert the log information of different test cases into structured log information. For example, the log information of different test cases can be converted into structured log information according to the following steps:

[0052] S 1. Determine the key fields of the log information: First, it is necessary to determine the key fields contained in the log information of each test case, such as the name of the test case, execution result, time consumption and other information.

[0053] S2, design a structured log format: Based on the determined key fields, design a structured log format. You can use JSON, XML and other formats to store log information.

[0054] S3, write a log conversion script: Write a script or program to parse the original test case log information and convert it into a structured log format. In the script, extract and store the key fields of the log information according to the previously designed log format.

[0055] S4, perform log conversion: run the written log conversion script to convert the log information of all test cases into a structured log format.

[0056] S5, storing and analyzing log information: storing the converted structured log information in a database or log file for subsequent analysis and monitoring of test results.

[0057] Through the above steps, the log information of different test cases can be converted into structured log information, making the log easier to manage and analyze.

[0058] Step S304, input the structured log information into the target classification model to obtain the test exception information table output by the target classification model; wherein the target classification model is a model obtained by training an initial classification model using multiple training log information, and the initial classification model includes a parsing subnetwork, a clustering subnetwork and a classification subnetwork. For each of the training log information, the parsing subnetwork is used to perform word segmentation processing on the training log information to obtain a training word segmentation list, the clustering subnetwork is used to cluster the training word segmentation list to obtain K word segmentation categories, and the classification subnetwork is used to determine a training log group based on the K word segmentation categories, and generate a training exception information table based on the training log group. The training log group The target classification model further includes abnormal information of M abnormal categories, wherein K is a natural number greater than or equal to 1, M is a natural number less than or equal to K, the training log information is information of the same type as the structured log information, and the training abnormal information table includes abnormal information of the M abnormal categories; the target classification model also includes a storage subnetwork, a database in the storage subnetwork stores preset abnormal information corresponding to multiple preset word segmentation categories, and the storage subnetwork is connected to the classification subnetwork, and is used for storing other abnormal information corresponding to other abnormal categories in the database when M is less than K, wherein the other abnormal categories are abnormal categories among the K word segmentation categories except the word segmentation categories matching the M abnormal categories.

[0059] Optionally, the above-mentioned initial classification network includes but is not limited to a network obtained using a machine learning module. The above-mentioned parsing subnetwork can be a text processing network, for example, a recurrent neural network (RNN) or a convolutional neural network (CNN), which is used to perform word segmentation on text data. This network can receive raw log information as input, and convert text data into a list after word segmentation by learning the features of pattern recognition and language processing. The purpose of word segmentation of the resultant log information and obtaining a training word segmentation list can be achieved. For example, the processed structured data is word segmented to obtain data_list and length; the instance after word segmentation is ['uvm_error', 'setup', 'DEBUG_COUNT_FLAG', '1000ns', 'Count_flag overtake 300'].

[0060] The clustering subnetwork can be composed of a neural network for unsupervised learning, and can include K-means clustering, hierarchical clustering, DBSCAN, etc. These models cluster the input data and classify similar data points into the same category, thereby realizing cluster analysis of the input data. When clustering the training word list, the clustering subnetwork will classify the words in the word list into different categories by learning the similarities and differences between the data. In this way, the word list can be effectively clustered and K word categories can be obtained. For example, the parameters of the accessed nodes of the parse tree in the clustering subnetwork are set to obtain the matching log template (including multiple categories), and the corresponding data matching cluster match_cluster is obtained by iterative comparison of the child nodes. For example, the instance looks at the length, uvm_error, setup and other information of the word structure in turn. For the efficient operation of the algorithm, the access node parameters will be restricted.

[0061] The classification subnetwork can be a network composed of a neural network model, for example, a convolutional neural network (CNN) or a recurrent neural network (RNN). These neural network models can receive input tokenized data and classify different token categories by learning the characteristics and patterns of the data. By training such a classification subnetwork, the training log group can be effectively determined and a training anomaly information table can be generated.

[0062] The storage subnetwork can be a network based on a neural network or a network based on a database system. In a neural network, the storage subnetwork can be a recurrent neural network (RNN) or a long short-term memory network (LSTM) with storage and retrieval functions. These networks can store preset exception information corresponding to multiple preset word segmentation categories, and by connecting to the classification subnetwork, the storage of exception information corresponding to other exception categories can be realized. In a database system, the storage subnetwork can be a database management system with storage and retrieval functions, which is used to store and manage exception information corresponding to preset word segmentation categories. When M is less than K, the storage subnetwork can store the exception information corresponding to other exception categories in the database by connecting to the classification subnetwork.

[0063] For example, the classification subnetwork determines whether match_cluster exists. If the judgment result is that it does not exist, it means that there is no corresponding log group in the library of the syntax tree parse_tree, and you need to use the log list data_1ist and cluster_id to create a new match_cluster; at the same time, add a new log group template to parse_tree, and update the log content and the corresponding ID to the list_group under this log group, which stores the classified log group content. If the judgment result is yes, get the most matching log template from parse_tree, then compare the current word segmentation result with the log template in the matching log group. If it is new information, update the word segmentation content and the corresponding ID to the list_group under this log group; the results of these two processes will update the clustering template parse tree storage and log feature value storage in real time.

[0064] The information included in the test abnormality information table obtained in this embodiment may be the information shown in Table 1.

[0065] Table 1:

[0066]

[0067]

[0068] This embodiment analyzes structured log information through a target classification model, and can more accurately identify and classify abnormal information.

[0069] In an exemplary embodiment, after determining the test exception information table of the target application based on the log information, the method further includes:

[0070] Step S402, in the process of executing regression testing on the target application using the test case, obtaining waveform information generated by executing the test case, wherein the waveform information is used to represent signal changes occurring during the regression testing process;

[0071] Optionally, in the process of executing regression testing on the target application using the test case, if the error information exceeds a set threshold, the simulation run of the failed cluster type test is restarted to generate waveform information, and the storage path (i.e., waveform position) and waveform opening method of the waveform information are stored in a related file. The error information, waveform position, and waveform opening method can be sent to the client, and the engineer can obtain the waveform information through the client to perform related debugging information to locate bugs and design debugging.

[0072] Step S404, searching the waveform information for abnormal waveforms that match the N target abnormal categories included in the test abnormality information table;

[0073] Optionally, the abnormal waveform includes but is not limited to: a noise waveform, a pulse waveform, a waveform distortion, a frequency abnormal waveform, a pulse amplitude abnormal waveform, and a pulse width abnormal waveform. According to the target abnormality categories included in the test abnormality information table, abnormal waveforms matching these categories can be found from the waveform information. By identifying and analyzing these abnormal waveforms, it can help determine the cause of the equipment failure or improve the signal processing algorithm.

[0074] Step S406, searching the log information for an exception log that matches the N target exception categories;

[0075] Optionally, to find exception logs that match N target exception categories from the log information, you can proceed as follows: Traverse the log information: traverse the log information line by line to find log entries containing exception information. Match each log entry to check whether it contains keywords or information that matches the target exception category. Record the successfully matched log entries, and you can output them to a new file or display them on the screen. Analyze the successfully matched log entries, view their context information and other related information to further understand the cause of the exception and how to solve it. Through the above steps, you can filter out exception logs that match N target exception categories from the log information, and locate and handle exceptions more quickly.

[0076] Step S408: locating anomalies in executing the regression test on the target application based on the abnormal waveform and the abnormal log.

[0077] Optionally, to locate the exceptions of regression testing of the target application based on the exception waveform and the exception log, you can proceed as follows: Analyze the exception waveform: First, carefully analyze the exception waveform to understand the time, frequency, duration, and other information of the exception. By analyzing the characteristics of the exception waveform, you can determine the specific circumstances and possible causes of the exception. View the exception log: At the same time, view the exception log to obtain more information about the exception. The exception log may contain the specific error message of the exception, the location where the exception occurred, the stack trace of the exception, and other information, which are very important for locating and resolving the exception. Determine the cause of the exception: Combine the information of the exception waveform and the exception log, and try to determine the root cause of the exception. The cause of the exception may be a bug in the code, a configuration problem, a resource bottleneck, etc. Perform regression testing again: According to the determined cause of the exception, write corresponding regression test cases to verify and reproduce the exception. Make sure that the regression test covers the code path and scenarios that may cause the exception, and perform the test execution. Through the above steps, you can effectively locate and resolve the exceptions in the target application based on the exception waveform and the exception log, and improve the stability and performance of the system.

[0078] Optionally, the specific process of performing anomaly detection of regression testing on the target application program in this embodiment in combination with abnormal waveform and abnormal log location is as follows: Figure 3 As shown, the following steps are included:

[0079] S30 1, start the test case to perform regression testing on the target application;

[0080] S302, obtaining the print log sent in real time by the data source of the test case to obtain log information;

[0081] S303, when new initial log information arrives, the log information is preprocessed using a regular expression, including converting the log information of different test cases into structured log information, mainly dividing the information into two parts, a constant part and a variable part, the constant part mainly includes error type, simulation stage, error flag, error number, for example, UVM error, SVA error, user-defined error, etc., the variable part mainly includes system information of specific errors, mainly including the cause of the error based on the context;

[0082] S304, parse the structured log information through the target classification model, take the preprocessed structured log information as the input of the target classification model, adopt the Drain3 algorithm, and parse the structured log information based on the fixed depth tree by following the specially designed rules encoded in the internal nodes of the tree and the leaf nodes of the tree. The specific steps include: segmenting the processed structured log information to obtain the log list data_list and the length of the list 1ength; the instance after segmentation is ['uvm_error', 'setup', 'DEBUG_COUNT_FLAG', '1000ns', 'Count_f1ag overtake 300']; set the parameters of the accessed nodes of the syntax tree parse tree to obtain the matching log template, and obtain the corresponding data matching match_cluster through iterative comparison of the child nodes. For example, the instance looks at the length, uvm_error, setup and other information of the segmentation structure in turn, and the access node parameters will be restricted for the efficient operation of the algorithm.

[0083] Determine whether match_cluster exists. If the judgment result is NONE, it means that there is no corresponding log group in the parse_tree library, and you need to use data_list and cluster_id to create a new match_cluster; at the same time, add a new log group template to parse_tree, and update the log content and the corresponding ID in 1ist_group under this log group, which stores the classified log group content. If the judgment result is yes, get the most matching log template from parse_tree, and compare the current word segmentation result with the log template in the matching log group. If it is new information, update the word segmentation content and the corresponding ID to list_group under this log group; the results of these two processes will update the clustering template parse tree storage and log feature value storage in real time.

[0084] S305, integrate the parsing learning model into the workflow of automated debugging, and extract error information from each real-time regression test print log through the machine learning module, mainly including the error_type, stage, tag, time, cause context, quantity, etc. corresponding to the test case, and record all information in the database. Cluster the error information and output the test exception information table.

[0085] S306, generating information related to the waveform file, and sending the waveform file to the client, the client performs anomaly detection of regression testing on the target application program in combination with the abnormal waveform and the abnormal log location.

[0086] This embodiment combines the abnormal waveform and the abnormal log to more comprehensively understand the specific situation of the target application when the abnormality occurs, so as to more accurately locate the problem. The abnormal waveform can reflect the various changes in the application during execution, while the abnormal log can provide detailed error information and abnormal stack to help developers determine the specific cause of the abnormality. Thus, regression testing can be performed more effectively.

[0087] In an exemplary embodiment, in the process of performing regression testing on the target application using the test case, determining the coverage data table of the test case includes:

[0088] Step S502, in the process of performing regression testing on the target application using the test case, storing the acquired test information into a test case information table, wherein the test case information includes at least one of the following information: information of the test case, a test path of the regression test, the number of times the regression test is executed, and coverage file information;

[0089] Optionally, the information of the test case includes the name, description, expected results, preconditions, steps, etc. of the test case. The test path of the regression test is the test process and test sequence. The number of times the regression test is executed includes multiple times, and multiple executions are performed to ensure the stability and reliability of the system. The coverage file information includes code coverage, function coverage, etc., which are used to measure the coverage and effect of the test. The coverage file can include test coverage reports, code coverage reports, etc.

[0090] Step S504, extracting coverage information under multiple coverage dimensions from the test case information table;

[0091] Optionally, to extract coverage information under multiple coverage dimensions, you first need to determine which coverage dimensions are included in the test case information table. Then calculate the corresponding coverage information based on the coverage of each dimension. For example, the test case information table contains three dimensions: functional coverage, code coverage, and path coverage. You can count the number of function points covered in the test case and divide it by the total number of function points to get the functional coverage. Count the number of code lines or code blocks covered in the test case and divide it by the total number of code lines or code blocks to get the code coverage. Count the number of paths covered in the test case and divide it by the total number of paths to get the path coverage.

[0092] Step S506, merging the coverage information under the multiple coverage dimensions to obtain the coverage data table.

[0093] Optionally, for example, a process for automatically collecting coverage and analyzing coverage is Figure 4 As shown, the following steps are included:

[0094] S401, start test case regression testing.

[0095] S402, test case information table arrangement, first create a test case regression test table based on the relevant regression information of the test case, the table gives the test case name, regression test path, regression number, coverage file name, merge version coverage name, merge version coverage output path, etc., which are used for automatic coverage statistics.

[0096] S403, parsing the coverage file information table, extracting information from the test case information table in S402 using Python, and extracting relevant information, mainly including test time, test case name, regression times, etc.

[0097] S404, automatic processing and merging of coverage files, using Python's subprocess and OS library to merge different versions of coverage files under different coverage paths, using the parsed information to generate different processes on the command terminal to execute coverage merging commands, and outputting the merged version of the coverage file to the specified path.

[0098] S405, coverage file parsing, parse the merge version of the coverage file, use the VC S simulation tool to read the coverage data, mainly including module name, line coverage, FSM coverage, flip coverage, conditional coverage and assertion coverage, and use the output file as the input of the next module.

[0099] S406, collating and outputting the coverage information of the merged versions, collating the information for the generated report, and outputting the coverage data table, which mainly includes the coverage information corresponding to different modules and the coverage information corresponding to different versions. And counting the contribution of different test cases to the coverage improvement, for example, after a certain number of regressions of test case 1, the growth rate of the coverage improvement of the two versions before and after, so that the contribution of different test cases to the coverage improvement can be intuitively seen.

[0100] Through the above steps, we can obtain coverage information in different dimensions, so as to comprehensively evaluate the coverage of test cases.

[0101] In an exemplary embodiment, in the process of performing regression testing on the target application using the test case, after determining the coverage data table of the test case, the method further includes:

[0102] Step S602, determining the code coverage, requirement coverage and use case coverage of the test case within a first preset time period based on the coverage data included in the coverage data table, wherein the code coverage is used to indicate the proportion of the code in the target application covered by the test case executed within the first preset time period, the requirement coverage is used to indicate the proportion of the requirements of the target application covered by the test case, and the use case coverage is used to indicate the proportion between the number of the test cases executed within the first preset time period and the total number of test cases in the regression test;

[0103] Optionally, code coverage can be generated by running test cases through a code coverage tool (e.g., JaCoCo, Emma, ​​etc.). Requirement coverage can be evaluated by comparing the inputs and expected outputs in the test cases with the requirements defined in the requirements document. Use case coverage can be evaluated by comparing the number of test cases listed in the test case design document or test plan with the number of test cases actually executed.

[0104] Step S604: Prompt information is issued using the code coverage, requirement coverage, and test case coverage, wherein the prompt information is used to prompt adjustment of the test case.

[0105] This embodiment can ensure the comprehensiveness and quality of the test by comprehensively considering code coverage, requirement coverage and use case coverage. By continuously optimizing test cases and test strategies, the test coverage can be improved, potential defects and risks can be reduced, and thus the software quality and reliability can be improved.

[0106] In an exemplary embodiment, using the test exception information table and the coverage data table to detect the regression test executed by the target application program includes:

[0107] Step S702, using the test exception information table to locate the test exception of the test case in the regression test, and obtain a first exception;

[0108] Optionally, the exception information table provides the exception information of the test case in the regression test, which can help locate the problem of the test case. According to the records in the exception information table, the exception type, exception occurrence time, exception description, etc. can be determined to analyze the cause of the problem. Possible causes include code errors, environment configuration problems, data input errors, etc. By carefully analyzing the exception information, we can determine the specific cause of the problem and take appropriate measures to solve the problem.

[0109] Step S704, using the coverage data table to detect coverage anomalies of the test case in the regression test, and obtaining a second anomaly;

[0110] Alternatively, coverage anomalies may indicate that the test cases do not adequately cover the code or that there are duplicate test cases. By analyzing the coverage data table, you can determine the coverage of the test cases and find coverage anomalies. If the coverage of a test case is significantly lower than that of other test cases, you may need to optimize the test case or add new test cases to improve the coverage. In addition, if multiple test cases cover the same part of the code, there may be duplicate test cases, and the test cases need to be streamlined or integrated. By analyzing the coverage data table, you can discover and resolve coverage anomalies of test cases in regression testing, ensure that the coverage of test cases is sufficient and effective, and thus improve software quality and testing efficiency.

[0111] Step S706: Perform anomaly detection on the regression test performed on the target application through the test case based on the first anomaly and the second anomaly.

[0112] Optionally, anomaly detection refers to detecting whether the target application will generate anomalies that are the same or similar to previously known anomalies when performing regression testing. According to the first anomaly and the second anomaly, anomaly detection can be performed by writing test cases for these anomalies. For example, for the first anomaly, a test case can be written to verify whether the application will process normally and give corresponding error prompts when illegal characters are entered. For example, some special characters or illegal inputs can be entered, and then the response of the application can be checked to see if it meets the expected error prompts. For the second anomaly, a test case can be written to verify whether the application will process correctly and give corresponding error prompts when the network connection is abnormal or timed out. For example, a network connection abnormality or timeout can be simulated, and then the response of the application can be checked to see if it meets the expected error prompts.

[0113] This embodiment can accurately detect whether a target application has similar abnormal situations in regression testing through the first exception and the second exception, and timely discover and solve these problems, thereby improving the stability and reliability of the application.

[0114] In an exemplary embodiment, after performing anomaly detection on the regression test performed on the target application through the test case based on the first anomaly and the second anomaly, the method further includes:

[0115] Step S802, extracting the abnormal parameters of the test case from the first abnormality, wherein the abnormal parameters include the abnormality quantity and the abnormality level;

[0116] Optionally, the exception levels of test cases can be divided into the following categories: High: When the test case execution fails, it will cause the system to crash or key functions to not work properly, affecting the core functions of the system. Medium: When the test case execution fails, some functions of the system cannot be used normally, but it will not cause the system to crash or key functions to not work properly. Low: When the test case execution fails, it will not affect the normal operation of the system function, but will have a certain degree of impact on the user experience or system performance. Information: When the test case execution fails, it only provides some additional information or suggestions and will not have a substantial impact on the system function. When writing test cases, you can determine the exception level based on the importance and impact of the test so that you can better handle and track problems during the test execution.

[0117] Step S804, extracting, from the second exception, a coverage growth rate of executing the regression test on the target application using the test case within a second preset time period;

[0118] Optionally, coverage growth rate=(coverage rate in the second preset time period-coverage rate in the first preset time period) / coverage rate in the first preset time period*100%. The coverage rate in the second preset time period refers to the coverage rate after the regression test is performed on the target application using the test case mentioned in the second exception, and the coverage rate in the first preset time period refers to the coverage rate after the regression test is performed on the target application using the test case mentioned in the first exception.

[0119] Step S806: adjusting the weight distribution of the test case in the regression test based on the abnormal parameter and the coverage growth rate.

[0120] Optionally, the weight distribution of the test case in the regression test is adjusted based on the exception parameters and the coverage growth rate, including: when the number of exceptions is greater than a preset number of exceptions, the exception level is greater than a preset level, and the coverage growth rate is greater than a preset growth rate, the distribution weight of the test case is adjusted to a first weight; when the number of exceptions is less than a preset number of exceptions, the exception level is less than a preset level, and the coverage growth rate is less than a preset growth rate, the distribution weight of the test case is adjusted to a second weight, wherein the first weight is greater than the second weight.

[0121] For example, the regression weights of test cases with higher error numbers and error levels are increased; the weights of test cases with higher coverage growth rates are increased, while the weights of test cases with lower coverage growth rates are decreased.

[0122] This embodiment can achieve accurate regression, save simulation resources and simulation time, and improve work efficiency by intelligently allocating weights of regression test cases.

[0123] The present application is described below in conjunction with specific embodiments:

[0124] This specific embodiment is aimed at the problem that the verification of chips (including multiple applications) in the prior art is time-consuming and causes waste of resources. A method for chip verification based on machine learning is proposed, and an automated platform for automated regression and coverage collection of a machine learning model is created. There are two main modules. The first module is used to create an automated platform based on machine learning to automatically complete the extraction, parsing and clustering analysis of log data in regression testing, and through the training and learning of the machine learning model, automatically output the statistics of error information and the generation and sending of waveforms to assist engineers in locating and debugging defects; the second module is the automated statistics, merge and analysis of coverage, and outputs the coverage data table, and reports the parsed data. The outputs of these two modules will be used as new inputs to adjust the weight distribution of test cases in regression testing, so as to intelligently allocate the weights of regression test cases, achieve accurate regression, save simulation resources and simulation time, and improve work efficiency.

[0125] like Figure 5 As shown, it is an overall flow chart of this specific embodiment, which includes the following steps:

[0126] S501, start the automated regression and coverage collection automation platform of the machine learning model, and use the test cases to perform regression testing on the application in the chip.

[0127] S502, enter the log clustering analysis, which includes:

[0128] Log information preprocessing: When engineers start regression testing of test cases, data sources of various tests will send print logs in real time. When new initial log information arrives, Python regular expressions are used to preprocess the log information, mainly to convert the log information of different test cases into structured log information.

[0129] Log analysis module: The machine learning module (i.e., the target classification model) parses the structured log information and uses the preprocessed log information as the input of the machine learning module, which includes: processing the structured log information through word segmentation, and then outputting the data_list using the Drain3 algorithm, a parsing method based on a fixed-depth tree, and parsing and classifying the processed log data by following the specially designed rules encoded in the internal nodes of the tree and the leaf nodes of the tree.

[0130] Log analysis module: The clustering template parsing number and log feature value storage repository of the machine learning module are updated. Through the processing of the clustering unit, on the one hand, the clustering template parsing number parse_tree database is updated, which is mainly used to store log group templates; on the other hand, the log content feature value database is updated, which mainly stores the specific content of the classified logs.

[0131] Log analysis module: Accelerates the location and processing of design bugs based on machine learning models. For the design defect cluster statistics table output by the machine learning module, if the design defects exceed the set threshold, the simulation run of the failed cluster type test is restarted to generate waveforms, and the waveform storage path and waveform opening method are stored in related files. The error information, waveform location, and opening method can be sent to the verification engineer, and the engineer can locate the bug and debug the design through the relevant debugging information.

[0132] S502, enter coverage statistics and analysis, including:

[0133] Coverage automatic analysis module: completes the test case information table organization, parsing and output. According to the pre-given test case information table, the coverage table is parsed using a python script, mainly extracting the test case name, regression test path, number of regressions, coverage file name, merge version coverage name, and merge version coverage output path in the project.

[0134] Coverage automatic analysis module: automatic processing and merging of coverage files, using Python's subprocess and OS library to merge different versions of coverage files under different coverage paths. Among them, Python's subprocess and OS library are tools used to execute external commands and operating system-related tasks. The subprocess library provides a simple interface to create and control subprocesses, which can execute external commands and obtain their output. It is more powerful and flexible and can handle more complex subprocess operations. The OS library provides an access interface to the operating system, including file operations, process management, environment variables, etc. It provides a series of functions to perform operating system-related tasks, such as creating directories, deleting files, and obtaining the current working directory.

[0135] Coverage automatic analysis module: coverage file parsing and output, using VCS simulation tools to read coverage data, mainly including module name, line coverage, FSM coverage, flip coverage and other values, and output coverage file statistics table.

[0136] The input for test case regression weight allocation and adjustment comes from two parts. The first part is the results of log information extraction, processing and statistics, and the measurement indicators are the number of errors and error levels corresponding to the test cases. The second part is the coverage information summary table as input, and the measurement indicator is the coverage growth rate of the test cases. These two parts of data distribute and adjust the weights of regression test cases.

[0137] This specific embodiment uses the logs in the existing regression test as a data set, organizes the data and uses it as the input of the machine learning standard model to complete the learning and training of the model, clusters and analyzes thousands of error messages that appear in the regression test, and uses relevant algorithms to merge the error messages into corresponding clusters. Users only need to analyze and repair certain error messages in the cluster to solve all the error messages in the cluster, saving energy and time. In addition, cluster analysis generally runs automatically without the need for users, and learns the causal relationship of error information, and then can autonomously identify the corresponding defect type, give corresponding reports, and accelerate the resolution of design defects. In addition, the coverage rate in the automated statistical regression test is used to complete the collection and merging of coverage rates. At the same time, the automated analysis of the coverage rate file is completed, and the report of coverage rate related information is used to guide the input of the regression test. Finally, the automatic adjustment regression test of the test case is realized according to the relevant algorithm to achieve accurate regression convergence. The input of the model is the error information statistics of the test cases and the contribution of the test cases to the coverage growth. Thousands of test cases are weighted and prioritized using relevant algorithms. Higher weights are given to test cases that contribute more to coverage growth and have more errors and defects. This can reduce the waste of simulation resources and time. Engineers can spend more time on bug location, resolution, and boundary testing, accelerating the convergence of chip verification.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0139] In this embodiment, a detection device based on regression testing of an application is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0140] Figure 6 is a structural block diagram of a detection device based on application program regression testing according to an embodiment of the present application, such as Figure 6 As shown, the device comprises:

[0141] The first acquisition module 62 is used to obtain the log information generated by executing the test case in the process of performing regression testing on the target application using the test case; the first determination module 64 is used to determine the test exception information table of the target application based on the log information, wherein the test exception information table includes target exception information corresponding to N target exception categories, and N is a natural number greater than or equal to 1; the second determination module 66 is used to determine the coverage data table of the test case in the process of performing regression testing on the target application using the test case, wherein the coverage data table includes test coverage information of executing the regression test on the target application, and the test coverage information is used to indicate the degree of testing of the target application; the first detection module 68 is used to detect the regression test executed on the target application using the test exception information table and the coverage data table.

[0142] In an exemplary embodiment, the first determination module includes: a first processing unit, which is used to perform structured processing on the log information to obtain structured log information; a first input unit, which is used to input the structured log information into a target classification model to obtain the test exception information table output by the target classification model; wherein the target classification model is a model obtained by training an initial classification model using multiple training log information, and the initial classification model includes a parsing subnetwork, a clustering subnetwork and a classification subnetwork. For each of the training log information, the parsing subnetwork is used to perform word segmentation processing on the training log information to obtain a training word segmentation list, the clustering subnetwork is used to cluster the training word segmentation list to obtain K word segmentation categories, and the classification subnetwork is used to determine the training log information based on the K word segmentation categories. A group is formed, and a training exception information table is generated based on the training log group, wherein the training log group includes exception information of M exception categories, wherein K is a natural number greater than or equal to 1, and M is a natural number less than or equal to K. The training log information is information of the same type as the structured log information, and the training exception information table includes exception information of the M exception categories; the target classification model also includes a storage subnetwork, wherein a database in the storage subnetwork stores preset exception information corresponding to multiple preset word segmentation categories, and the storage subnetwork is connected to the classification subnetwork, and is used for storing other exception information corresponding to other exception categories in the database when M is less than K, wherein the other exception categories are exception categories among the K word segmentation categories except the word segmentation categories matching the M exception categories.

[0143] In an exemplary embodiment, the above-mentioned device also includes: a second acquisition module, which is used to determine the test exception information table of the target application based on the log information, and then, in the process of performing regression testing on the target application using the test case, obtain waveform information generated by executing the test case, wherein the waveform information is used to represent signal changes occurring during the regression testing process; a first search module, which is used to search for abnormal waveforms matching the N target exception categories included in the test exception information table from the waveform information; a second search module, which is used to search for abnormal logs matching the N target exception categories from the log information; and a first positioning module, which is used to locate the anomaly of executing the regression test on the target application based on the abnormal waveform and the abnormal log.

[0144] In an exemplary embodiment, the above-mentioned second determination module includes: a first storage unit, used to store the acquired test information in a test case information table during the process of performing regression testing on the target application using the test case, wherein the test case information includes at least one of the following information: information of the test case, the test path of the regression test, the number of times the regression test is executed, and coverage file information; a first extraction unit, used to extract coverage information under multiple coverage dimensions from the test case information table; a first merging unit, used to merge the coverage information under the multiple coverage dimensions to obtain the coverage data table.

[0145] In an exemplary embodiment, the above-mentioned device also includes: a third determination module, which is used to determine the coverage data table of the test case during the process of performing regression testing on the target application using the test case, and then determine the code coverage, requirement coverage and use case coverage of the test case within a first preset time period based on the coverage data included in the coverage data table, wherein the code coverage is used to indicate the proportion of the code in the target application covered by the test case executed within the first preset time period, the requirement coverage is used to indicate the proportion of the requirements of the target application covered by the test case, and the use case coverage is used to indicate the ratio between the number of test cases executed within the first preset time period and the total number of test cases in the regression test; a first prompt module, which is used to issue prompt information using the code coverage, requirement coverage and use case coverage, wherein the prompt information is used to prompt the adjustment of the test case.

[0146] In an exemplary embodiment, the above-mentioned first detection module includes: a first positioning unit, used to use the test exception information table to locate the test exception of the test case in the regression test to obtain a first exception; a first detection unit, used to use the coverage data table to detect the coverage exception of the test case in the regression test to obtain a second exception; a second detection unit, used to perform anomaly detection on the regression test performed on the target application through the test case based on the first exception and the second exception.

[0147] In an exemplary embodiment, the above-mentioned device also includes: a first extraction module, which is used to extract the abnormal parameters of the test case from the first abnormality after performing abnormality detection on the regression test performed on the target application through the test case based on the first abnormality and the second abnormality, wherein the abnormal parameters include the number of abnormalities and the abnormality level; a second extraction module, which is used to extract the coverage growth rate of performing the regression test on the target application using the test case within a second preset time period from the second abnormality; and a first adjustment module, which is used to adjust the weight distribution of the test case in the regression test based on the abnormal parameters and the coverage growth rate.

[0148] In an exemplary embodiment, the above-mentioned first adjustment module includes: a first adjustment unit, used to adjust the distribution weight of the test case to a first weight when the number of exceptions is greater than a preset number of exceptions, the exception level is greater than a preset level, and the coverage growth rate is greater than a preset growth rate; a second adjustment unit, used to adjust the distribution weight of the test case to a second weight when the number of exceptions is less than a preset number of exceptions, the exception level is less than a preset level, and the coverage growth rate is less than a preset growth rate, wherein the first weight is greater than the second weight.

[0149] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0150] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0151] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0152] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0153] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0154] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0155] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0156] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.

[0157] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0158] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0159] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A detection method based on regression testing of an application, characterized in that: include: In the process of executing regression testing on the target application using the test case, obtaining log information generated by executing the test case; Determine a test exception information table of the target application based on the log information, wherein the test exception information table includes target exception information corresponding to N target exception categories, where N is a natural number greater than or equal to 1; In the process of performing regression testing on the target application using the test case, determining a coverage data table of the test case, wherein the coverage data table includes test coverage information of executing the regression test on the target application, and the test coverage information is used to indicate the degree of testing the target application; The regression test performed on the target application is detected using the test exception information table and the coverage data table.

2. The method according to claim 1, characterized in that Determining a test exception information table of the target application based on the log information includes: Performing structured processing on the log information to obtain structured log information; Inputting the structured log information into a target classification model to obtain the test exception information table output by the target classification model; Wherein, the target classification model is a model obtained by training an initial classification model using multiple training log information, and the initial classification model includes a parsing subnetwork, a clustering subnetwork and a classification subnetwork. For each of the training log information, the parsing subnetwork is used to perform word segmentation processing on the training log information to obtain a training word segmentation list, and the clustering subnetwork is used to cluster the training word segmentation list to obtain K word segmentation categories. The classification subnetwork is used to determine a training log group based on the K word segmentation categories, and generate a training abnormality information table based on the training log group. The training log group includes abnormal information of M abnormal categories, K is a natural number greater than or equal to 1, and M is a natural number less than or equal to K. The training log information is information of the same type as the structured log information, and the training abnormality information table includes abnormal information of the M abnormal categories; The target classification model also includes a storage subnetwork, in which a database stores preset exception information corresponding to multiple preset word segmentation categories. The storage subnetwork is connected to the classification subnetwork and is used to store other exception information corresponding to other exception categories in the database when M is less than K, wherein the other exception categories are exception categories among the K word segmentation categories except for the word segmentation categories that match the M exception categories.

3. The method according to claim 1, characterized in that After determining the test exception information table of the target application based on the log information, the method further includes: In the process of executing regression testing on the target application using the test case, acquiring waveform information generated by executing the test case, wherein the waveform information is used to represent signal changes occurring during the regression testing process; Searching for an abnormal waveform matching the N target abnormal categories included in the test abnormality information table from the waveform information; Searching for an exception log matching the N target exception categories from the log information; Anomalies in executing the regression test on the target application are located based on the abnormal waveform and the abnormal log.

4. The method according to claim 1, characterized in that In the process of performing regression testing on the target application using the test case, determining a coverage data table of the test case includes: In the process of performing regression testing on the target application using the test case, the acquired test information is stored in a test case information table, wherein the test case information includes at least one of the following information: information of the test case, a test path of the regression test, the number of times the regression test is executed, and coverage file information; Extract coverage information under multiple coverage dimensions from the test case information table; The coverage information under the multiple coverage dimensions is combined to obtain the coverage data table.

5. The method according to claim 1, characterized in that In the process of performing regression testing on the target application using the test case, after determining the coverage data table of the test case, the method further includes: Determine the code coverage, requirement coverage and use case coverage of the test case within a first preset time period based on the coverage data included in the coverage data table, wherein the code coverage is used to indicate the proportion of the code in the target application covered by the test case executed within the first preset time period, the requirement coverage is used to indicate the proportion of the requirements of the target application covered by the test case, and the use case coverage is used to indicate the proportion between the number of the test cases executed within the first preset time period and the total number of test cases in the regression test; Prompt information is issued using the code coverage, requirement coverage, and test case coverage, wherein the prompt information is used to prompt adjustment of the test case.

6. The method according to claim 1, characterized in that Detecting the regression test performed on the target application using the test exception information table and the coverage data table includes: Locating the test exception of the test case in the regression test by using the test exception information table to obtain a first exception; Using the coverage data table to detect coverage anomalies of the test case in the regression test, and obtaining a second anomaly; Anomaly detection is performed on the regression test performed on the target application through the test case based on the first anomaly and the second anomaly.

7. The method according to claim 6, characterized in that After performing anomaly detection on the regression test performed on the target application through the test case based on the first anomaly and the second anomaly, the method further includes: Extracting an abnormality parameter of the test case from the first abnormality, wherein the abnormality parameter includes an abnormality quantity and an abnormality level; Extracting, from the second exception, a coverage growth rate of executing the regression test on the target application using the test case within a second preset time period; The weight distribution of the test cases in the regression test is adjusted based on the abnormal parameters and the coverage growth rate.

8. The method according to claim 7, characterized in that Adjusting the weight distribution of the test case in the regression test based on the abnormal parameter and the coverage growth rate includes: When the number of exceptions is greater than a preset number of exceptions, the level of exceptions is greater than a preset level, and the coverage growth rate is greater than a preset growth rate, adjusting the distribution weight of the test case to a first weight; When the number of exceptions is less than a preset number of exceptions, the level of exceptions is less than a preset level, and the coverage growth rate is less than a preset growth rate, the distribution weight of the test case is adjusted to a second weight, wherein the first weight is greater than the second weight.

9. A detection device based on regression testing of an application program, characterized in that: include: A first acquisition module is used to acquire log information generated by executing the test case in the process of executing regression testing on the target application using the test case; A first determination module, configured to determine a test exception information table of the target application based on the log information, wherein the test exception information table includes target exception information corresponding to N target exception categories, where N is a natural number greater than or equal to 1; A second determination module is used to determine a coverage data table of the test case in the process of performing regression testing on the target application using the test case, wherein the coverage data table includes test coverage information of executing the regression test on the target application, and the test coverage information is used to indicate the degree of testing the target application; The first detection module is used to detect the regression test executed by the target application program by using the test exception information table and the coverage data table.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 8 when executed by a processor.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Citation Information

Cited By

  • Database test case generation method and device, server and storage medium

    CN120196555A

  • Database test case generation method, device, server and storage medium

    CN120196555B

  • Detection method and device, storage medium and electronic equipment

    CN122133574A