Code coverage rate threshold generation method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a code coverage rate threshold generation method and device, equipment, a storage medium and a program product. Belongs to the technical field of software testing. The method comprises the following steps: in response to receiving a coverage rate threshold generation instruction sent by a control terminal, obtaining a historical coverage rate, a missing test defect number, a hit frequency and a found defect number corresponding to each code block in a test record of each version of software; determining hot-spot code blocks and non-hot-spot code blocks according to the hit times corresponding to the code blocks and the found defect numbers corresponding to the code blocks; determining a hotspot code coverage rate threshold value corresponding to the hotspot code block according to the number of missed detection defects corresponding to the hotspot code block and a historical coverage rate; determining a non-hot-spot code coverage rate threshold value corresponding to the non-hot-spot code block according to the missing detection defect number and the historical coverage rate corresponding to the non-hot-spot code block; and outputting the hot-spot code coverage rate threshold and the non-hot-spot code coverage rate threshold. The method solves the problems of low test efficiency and waste of test resources.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to a method, device, equipment, storage medium and program product for generating a code coverage threshold. Background Art

[0002] After each software version update is completed, it is usually necessary to test the new version of the software. The code coverage during the test process needs to reach a certain coverage threshold to ensure that the test is sufficient to reflect the problems in the software.

[0003] At present, the selection of coverage threshold in related technologies is usually done manually. The staff checks a large amount of test coverage obtained in the test experience and selects the coverage threshold that they think is appropriate based on experience.

[0004] However, the inventors have discovered that the related technology has at least the following technical problems: In order to ensure the test effect, current testers usually select a higher test coverage threshold, which leads to low test efficiency and waste of test resources. Summary of the invention

[0005] The embodiments of the present application provide a code coverage threshold generation method, apparatus, device, storage medium and program product to solve the problems of low testing efficiency and waste of testing resources.

[0006] In a first aspect, an embodiment of the present application provides a code coverage threshold generation method, including: in response to receiving a coverage threshold generation instruction sent by a control terminal, obtaining the historical coverage, number of missed defects, number of hits and number of found defects corresponding to each code block in the test records of each version of the software; determining the hotspot code blocks and non-hotspot code blocks according to the number of hits corresponding to each code block and the number of found defects corresponding to each code block; determining the hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and the historical coverage corresponding to the hotspot code block; determining the non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and the historical coverage corresponding to the non-hotspot code block; outputting the hotspot code coverage threshold and the non-hotspot code coverage threshold.

[0007] In a possible implementation, the hotspot code coverage threshold corresponding to the hotspot code block is determined according to the number of missed defects and the historical coverage rate corresponding to the hotspot code block, including: dividing the historical coverage rate corresponding to the hotspot code block into N historical coverage rate intervals, where N is a positive integer; calculating the coverage average value corresponding to the target historical coverage rate interval according to the historical coverage rate corresponding to the target historical coverage rate interval, where the target historical coverage rate interval is any historical coverage rate interval; determining the average number of missed defects corresponding to each historical coverage rate interval according to the number of missed defects and the historical coverage rate corresponding to each hotspot code block; fitting the energy consumption defect curve using the average coverage rate and the average number of missed defects; determining the extreme points of the energy consumption defect curve; determining the extreme point with the largest average coverage rate among the extreme points as the target extreme point; and determining the coverage average value corresponding to the target extreme point as the hotspot code coverage threshold.

[0008] In one possible implementation, based on the number of missed defects and historical coverage corresponding to each hot code block, determining the average number of missed defects corresponding to each historical coverage interval includes: determining the hot code block whose historical coverage is in the target historical coverage interval as the reference code block; calculating the average number of missed defects corresponding to the reference code block to obtain the average number of missed defects corresponding to the target historical coverage interval.

[0009] In one possible implementation, determining the extreme point of the energy consumption defect curve includes: taking the derivative of the energy consumption defect curve, and determining the point corresponding to the first-order derivative of 0 as the critical point; calculating the second-order derivative value of the energy consumption defect curve corresponding to the critical point; if the second-order derivative value is not 0, determining the critical point as the extreme point; if the second-order derivative value is 0, calculating the derivatives of the two adjacent points on the left and right of the critical point; if the first-order derivatives of the adjacent points have different signs, determining the critical point as the extreme point.

[0010] In one possible implementation, the hotspot code coverage threshold corresponding to the hotspot code block is determined based on the number of missed defects and historical coverage corresponding to the hotspot code block, including: combining the historical coverages and the corresponding number of missed defects corresponding to the hotspot code block into a vector; clustering the vectors to obtain at least one cluster; and determining the largest historical coverage in the cluster containing the most vectors as the hotspot code coverage threshold.

[0011] In a possible implementation, hot code blocks and non-hot code blocks are determined based on the number of hits corresponding to each code block and the number of defects found corresponding to each code block, including: determining high-hit code blocks in each code block based on the number of hits corresponding to each code block; determining high-defect rate code blocks in each code block based on the number of defects found corresponding to each code block; if a target code block is both a high-hit code block and a high-defect rate code block, then the target code block is determined as a hot code block, otherwise the target code block is determined as a non-hot code block.

[0012] In one possible implementation, after determining the non-hotspot code coverage threshold corresponding to the non-hotspot code block based on the number of missed defects and historical coverage corresponding to the non-hotspot code block, it also includes: in response to receiving a test instruction for the software, testing the software to obtain test results, wherein the test results include the test coverage corresponding to each code block; if the test coverage corresponding to any hotspot code block of the software is less than the hotspot code coverage threshold, or the test coverage corresponding to any non-hotspot code block is less than the non-hotspot code coverage threshold, then outputting a prompt that the test coverage does not meet the standard.

[0013] In the second aspect, an embodiment of the present application provides a code coverage threshold generating device, including: a data acquisition module, used to respond to receiving a coverage threshold generating instruction sent by a control terminal, and obtain the historical coverage, number of missed defects, number of hits and number of found defects corresponding to each code block in the test records of each version of the software; a code block determination module, used to determine the hotspot code blocks and non-hotspot code blocks according to the number of hits corresponding to each code block and the number of found defects corresponding to each code block; a first threshold determination module, used to determine the hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and the historical coverage corresponding to the hotspot code block; a second threshold determination module, used to determine the non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and the historical coverage corresponding to the non-hotspot code block; a threshold output module, used to output the hotspot code coverage threshold and the non-hotspot code coverage threshold.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor executes the first aspect above and / or various possible implementations of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0017] The code coverage threshold generation method, apparatus, device, storage medium and program product provided in the embodiments of the present application determine hot code blocks and non-hot code blocks by the number of hits corresponding to the code blocks and the number of defects found corresponding to each code block, and use the number of missed defects and historical coverage to determine the hot code coverage threshold corresponding to the hot code blocks for the hot code blocks and non-hot code blocks respectively, so that the code coverage threshold used in subsequent tests is more efficient, thereby increasing test efficiency and reducing waste of test resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] Figure 1 A schematic diagram of a scenario of a code coverage threshold generation method provided in this application;

[0020] Figure 2 A schematic diagram of a flow chart of a method for generating a code coverage threshold value provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of a code coverage threshold generation device provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0023] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0025] After the software version is updated, testing the new version is a routine process, which aims to ensure that the testing activities can effectively reveal potential problems in the software. To this end, the code coverage during the testing process must meet the preset coverage threshold.

[0026] Currently, the setting of coverage thresholds mainly relies on manual judgment. Based on rich testing experience and a large amount of historical coverage data, staff members subjectively select the thresholds they think are appropriate. This method of selecting test coverage thresholds usually uses a larger test coverage threshold, which prolongs the test time and reduces the test efficiency.

[0027] In response to the above technical problems, the inventors proposed the following technical concept: by obtaining the historical coverage, number of missed defects, number of hits and number of defects found corresponding to each code block in the test records of each version of the software, the hot code blocks and non-hot code blocks in the code blocks are determined by the number of hits and the number of defects found, and the number of missed defects and historical coverage are used to determine the coverage thresholds corresponding to the hot code blocks and non-hot code blocks.

[0028] This application is used in the scenario of generating code coverage thresholds. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation portals for users to choose to authorize or refuse.

[0029] Figure 1 A schematic diagram of a scenario of a method for generating a code coverage threshold provided in this application. Figure 1 In this scenario, it includes: a control terminal 101 and a server 102.

[0030] In a specific implementation process, the control terminal 101 may include a computer, a server, a tablet, a mobile phone, a PDA (Personal Digital Assistant), a notebook, etc., which can input data.

[0031] The server 102 can be implemented using a server or a cluster of multiple servers with more powerful processing capabilities and higher security. If possible, it can also be replaced by a computer, laptop, etc. with stronger computing power.

[0032] The connection between the server 102 and the control terminal 101 can be a wired connection or a wireless connection.

[0033] The control terminal 101 is used to send a coverage threshold generation instruction to the server 102, and the server 102 is used to determine the hotspot code coverage threshold corresponding to the hotspot code block and the non-hotspot code coverage threshold corresponding to the non-hotspot code block using historical test data.

[0034] It is understandable that the scenarios illustrated in the embodiments of the present application do not constitute a specific limitation on the code coverage threshold generation method. In other feasible implementations of the present application, the above scenarios may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or arrange the components differently, which can be determined according to the actual application scenario and is not limited here. Figure 1 The scenarios shown can be implemented by hardware, software, or a combination of software and hardware.

[0035] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0036] Figure 2 A flowchart of a method for generating a code coverage threshold value provided in an embodiment of the present application is provided. The execution subject of the embodiment of the present application may be Figure 1 The server 102 in the embodiment may also be a computer and / or a mobile phone, etc., which is not particularly limited in this embodiment. Figure 2 As shown, the method includes:

[0037] S201: In response to receiving a coverage threshold generation instruction sent by a control terminal, obtaining historical coverage, number of missed defects, number of hits, and number of found defects corresponding to each code block in the test records of each version of the software.

[0038] In this step, the software may be an application program or a software system. The coverage threshold generation instruction may be received by receiving a data packet, a message, etc. By reading the historical coverage field, the number of missed defects field, the hit number field, and the number of found defects field corresponding to each code block in the test record, the historical coverage, the number of missed defects, the hit number, and the number of found defects corresponding to each code block are obtained.

[0039] S202: Determine hot code blocks and non-hot code blocks according to the number of hits corresponding to each code block and the number of found defects corresponding to each code block.

[0040] In this step, it can include determining code blocks with a large number of hit times and a large number of defects found as hot code blocks, and determining other code blocks as non-hot code blocks; it can also include taking a weighted sum of the number of hits and the number of defects found to obtain a comprehensive heat value, and determining the code blocks with a high comprehensive heat value as hot code blocks.

[0041] S203: Determine a hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and the historical coverage corresponding to the hotspot code block.

[0042] In this step, the number of missed defects and the historical coverage rate may be input into a threshold calculation formula to obtain a corresponding hotspot code coverage rate threshold.

[0043] S204: Determine a non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and the historical coverage corresponding to the non-hotspot code block.

[0044] This step is similar to the above step S203 and will not be repeated here.

[0045] S205: Outputting a hotspot code coverage threshold and a non-hotspot code coverage threshold.

[0046] In this step, it may include displaying and outputting the hotspot code coverage threshold and the non-hotspot code coverage threshold, and it may also include sending the hotspot code coverage threshold and the non-hotspot code coverage threshold to the control terminal.

[0047] From the description of the above embodiments, it can be seen that the embodiments of the present disclosure determine hot code blocks and non-hot code blocks by the number of hits corresponding to the code blocks and the number of defects found corresponding to each code block, and use the number of missed defects and historical coverage to determine the hot code coverage threshold corresponding to the hot code blocks for the hot code blocks and non-hot code blocks respectively, so that the code coverage threshold used in subsequent tests is more efficient, thereby increasing test efficiency and reducing waste of test resources.

[0048] In a possible implementation, in the above step S203, determining the hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and the historical coverage corresponding to the hotspot code block includes:

[0049] S2031: Divide the historical coverage corresponding to the hot code block into N historical coverage intervals, where N is a positive integer.

[0050] In this step, the historical coverage rate is evenly divided into N historical coverage rate intervals, or the historical coverage rate is divided into N historical coverage rate intervals according to the value size.

[0051] S2032: Calculate the average coverage rate corresponding to the target historical coverage rate interval according to the historical coverage rate corresponding to the target historical coverage rate interval, where the target historical coverage rate interval is any historical coverage rate interval.

[0052] In this step, for example, historical coverage interval A corresponds to 4 historical coverages, which are 75%, 76%, 78%, and 79%, respectively, and the average coverage corresponding to historical coverage interval A is 77%. For another example, historical coverage interval B corresponds to 5 historical coverages, which are 82%, 85%, 87%, 88%, and 88%, respectively, and the average coverage corresponding to historical coverage interval B is 86%.

[0053] S2033: Determine the average number of missed defects corresponding to each historical coverage interval according to the number of missed defects corresponding to each hotspot code block and the historical coverage.

[0054] In this step, for example, if the historical coverage corresponding to the hot code blocks A, B, and C in a certain test belongs to the historical coverage interval c, and the historical coverage interval c only corresponds to the hot code blocks A, B, and C, then the average number of missed defects corresponding to the hot code blocks A, B, and C in this test is calculated to obtain the average number of missed defects corresponding to the historical coverage interval c. For another example, if the historical coverage interval d corresponds to the hot code blocks E, F, G, and H, then the average number of missed defects corresponding to the hot code blocks E, F, G, and H is calculated to obtain the average number of missed defects corresponding to the historical coverage interval d.

[0055] S2034: Use the average coverage value and the average number of missed defects to fit the energy consumption defect curve.

[0056] In this step, the average coverage rate and the average number of missed defects corresponding to each historical coverage rate interval can be combined into coordinate points, and each coordinate point can be fitted to obtain an energy consumption defect curve. It can also include multiplying the average coverage rate by a preset value to obtain energy consumption, and taking the reciprocal of the number of missed defects, and returning the energy consumption defect curve from the energy consumption and the reciprocal of the number of missed defects.

[0057] S2035: Determine the extreme point of the energy consumption defect curve.

[0058] In this step, the extreme point of the energy consumption defect curve can be determined by using the gradient descent method, Newton's method, etc.

[0059] S2036: Determine the extreme value point with the largest average coverage rate among the extreme value points as the target extreme value point.

[0060] In this step, for example, if there are currently 3 extreme value points, the average coverage rates corresponding to these 3 extreme value points are read, and 75%, 85%, and 88% are obtained respectively, then the extreme value point with an average coverage rate of 88% is determined as the target extreme value point; for another example, if there are currently 5 extreme value points, the average coverage rates corresponding to these 5 extreme value points are read, and 60%, 66%, 70%, 85%, and 90% are obtained respectively, then the extreme value point with an average coverage rate of 90% is determined as the target extreme value point.

[0061] S2037: Determine the average coverage value corresponding to the target extreme value point as the hotspot code coverage threshold.

[0062] In this step, for example, if the average coverage rate corresponding to the target extreme point is 80%, 80% is determined as the hotspot code coverage rate threshold. For another example, if the average coverage rate corresponding to the target extreme point is 90%, 90% is determined as the hotspot code coverage rate threshold.

[0063] From the description of the above embodiments, it can be seen that the embodiment of the present disclosure divides the historical coverage corresponding to the hot code block into intervals, fits the curve, and then finds the extreme value of the curve. The coverage with the highest coverage and lower energy consumption is used as the coverage threshold, thereby reducing the resource usage of the test while ensuring the defect detection effect.

[0064] In a possible implementation, in the above step S2033, according to the number of missed defects and the historical coverage corresponding to each hotspot code block, the average number of missed defects corresponding to each historical coverage interval is determined, including:

[0065] S331: Determine a hotspot code block whose historical coverage is within a target historical coverage range as a reference code block.

[0066] In this step, for example, if the target historical coverage interval is 80% to 90%, and the historical coverage threshold corresponding to hot code block A is 85%, then hot code block A is the reference code block. For another example, if the target historical coverage interval is 90% to 100%, and the historical coverage threshold corresponding to hot code block B is 92%, then hot code block B is the reference code block.

[0067] S332: Calculate the average number of missed defects corresponding to the reference code block to obtain the average number of missed defects corresponding to the target historical coverage interval.

[0068] In this step, the number of missed defects corresponding to each reference code block is added and then divided by the total number of reference code blocks to obtain the average number of missed defects.

[0069] From the description of the above embodiments, it can be seen that the embodiments of the present disclosure find the hotspot code blocks corresponding to each historical coverage interval, and then take the average value of the number of missed defects of the corresponding hotspot code blocks to obtain the average number of missed defects corresponding to the historical coverage interval, thereby finding the number of missed defects that can represent the characteristics of the historical coverage interval as the number of missed defects used, making the subsequent process of finding the coverage threshold more accurate.

[0070] In a possible implementation, in the above step S2035, determining the extreme point of the energy consumption defect curve includes:

[0071] S351: deriving the energy consumption defect curve, and determining the point corresponding to the first-order derivative of 0 as the critical point.

[0072] In this step, for example, after taking the derivative of the energy consumption defect curve, the first-order inverses corresponding to three points are obtained to be 0, and these three points are determined as critical points. For another example, after taking the derivative of the energy consumption defect curve, the first-order inverses corresponding to five points are obtained to be 0, and these five points are determined as critical points.

[0073] S352: Calculate the second-order derivative value of the energy consumption defect curve corresponding to the critical point.

[0074] In this step, the first-order derivative of the energy consumption defect curve is calculated, the first-order derivative is differentiated again to obtain the second-order derivative, and the critical point coordinates are substituted into the second-order derivative to obtain the second-order derivative.

[0075] S353: If the second-order derivative value is not 0, the critical point is determined to be an extreme point.

[0076] In this step, for example, if the second-order derivative values ​​corresponding to critical points A, B, and C are 1, -1, and 0, respectively, then critical points A and B are determined as extreme points. For another example, if the second-order derivative values ​​corresponding to critical points D, E, and F are 10, 0.1, and 1, respectively, then all three critical points are determined as extreme points.

[0077] S354: If the second-order derivative value is 0, the derivatives of the two adjacent points on the left and right of the critical point are calculated. If the first-order derivatives of the adjacent points have different signs, the critical point is determined to be an extreme point.

[0078] In this step, if the second-order derivative value is 0, then select points within a preset range around the critical point as adjacent points, and calculate the first-order derivatives at the adjacent points to obtain two first-order derivative values. If the two values ​​are one positive and one negative, then the critical point of the price is determined as the extreme point, otherwise this point is determined as the extreme point.

[0079] It can be seen from the description of the above embodiments that the embodiments of the present disclosure determine the extreme points in the curve by calculating derivatives, thereby facilitating the subsequent use of the extreme points to find the test coverage threshold.

[0080] In a possible implementation, in the above step S203, determining the hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and the historical coverage corresponding to the hotspot code block includes:

[0081] S203B1: Combine the historical coverage rates and the corresponding number of missed defects corresponding to the hot code blocks into a vector.

[0082] In this step, each test will obtain a set of historical coverage and number of missed defects of the hot code block, and this set of historical coverage and number of missed defects are combined to obtain a vector.

[0083] S203B2: Cluster each vector to obtain at least one cluster.

[0084] In this step, the K-means clustering algorithm can be used to cluster the vectors to obtain clusters.

[0085] S2033B: Determine the maximum historical coverage rate among the clusters containing the most vectors as the hotspot code coverage rate threshold.

[0086] In this step, for example, if 4 clusters are currently obtained, and the number of vectors they contain is 5, 10, 15, and 20, respectively, then the largest historical coverage in the fourth cluster is determined as the hotspot code coverage threshold. For another example, if 5 clusters are currently obtained, and the number of vectors they contain is 3, 20, 50, 20, and 10, respectively, then the largest historical coverage in the third cluster is determined as the hotspot code coverage threshold.

[0087] From the description of the above embodiments, it can be seen that the embodiments of the present disclosure combine the historical coverage and the corresponding number of missed defects into vectors, cluster the vectors to obtain clusters, and determine the largest historical coverage in the cluster containing the most vectors as the hotspot code coverage threshold, thereby selecting a suitable historical coverage from the historical coverage generated by most tests as the coverage threshold actually used, so that the coverage threshold is sufficient for testing and not too high.

[0088] In a possible implementation, in the above step S202, determining the hot code blocks and the non-hot code blocks according to the number of hits corresponding to each code block and the number of defects found corresponding to each code block includes:

[0089] S2021: Determine a high-hit code block among the code blocks according to the number of hits corresponding to each code block.

[0090] In this step, the code blocks whose corresponding hit counts rank in the top M% among all code blocks may be determined as high-hit code blocks, or the code blocks whose corresponding hit counts rank in the top X% among all code blocks may be determined as high-hit code blocks.

[0091] Among them, M and X are positive numbers.

[0092] S2022: Determine a code block with a high defect rate among the code blocks according to the number of defects found in each code block.

[0093] In this step, the code blocks whose corresponding number of found defects ranks in the top J% among all code blocks may be determined as high-hit code blocks, or the code blocks whose corresponding number of found defects ranks in the top L may be determined as high-hit code blocks.

[0094] Among them, J and L are positive numbers.

[0095] S2023: If the target code block is both a high-hit code block and a high-defect rate code block, the target code block is determined as a hotspot code block; otherwise, the target code block is determined as a non-hotspot code block.

[0096] In this step, for example, if code block A is both a high-hit code block and a high-defect rate code block, code block A is determined as a hot code block; for example, if code block B is both a high-hit code block and a high-defect rate code block, code block B is determined as a hot code block.

[0097] It can be seen from the description of the above embodiments that the embodiments of the present disclosure screen out the code blocks that mainly need to be tested by determining the code blocks with a high number of hits and a high number of defects as hot code blocks.

[0098] In a possible implementation, after the above step S204 determines the non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and historical coverage corresponding to the non-hotspot code block, it also includes: step S230 and step S231.

[0099] S230: In response to receiving a test instruction for the software, the software is tested to obtain a test result, wherein the test result includes a test coverage rate corresponding to each code block.

[0100] In this step, the test instruction may be a trigger instruction of a test button issued by a staff member, or a command line input by a staff member, etc. The process of testing the software may include running the software corresponding to the test instruction using a test case to obtain a test result.

[0101] S231: If the test coverage rate corresponding to any hotspot code block of the software is less than the hotspot code coverage rate threshold, or the test coverage rate corresponding to any non-hotspot code block is less than the non-hotspot code coverage rate threshold, a prompt indicating that the test coverage rate does not meet the standard is output.

[0102] In this step, the prompt of the test coverage not meeting the standard may be pre-set, or the branch or line number of the hot code block may be written into the prompt template to obtain the prompt of the test coverage not meeting the standard.

[0103] For example, if the hot code coverage threshold is 90%, and the test coverage of hot code block A in the software is 85%, then a prompt indicating that the test coverage does not meet the standard will be output. For another example, if the non-hot code coverage threshold is 80%, and the test coverage of non-hot code block B in the software is 75%, then the line number of the non-hot code block is written into the prompt template, and a prompt indicating that the test coverage does not meet the standard is obtained, and a prompt indicating that the test coverage does not meet the standard is output.

[0104] From the description of the above embodiments, it can be seen that the embodiments of the present disclosure monitor the test effect by judging the compliance status after the test and giving a prompt when a code block test fails to meet the standard.

[0105] Figure 3 This is a schematic diagram of the structure of the code coverage threshold generation device provided in the embodiment of the present application. Figure 3 As shown, the code coverage threshold generating device 300 includes: a data acquisition module 301, a code block determination module 302, a first threshold determination module 303, a second threshold determination module 304, and a threshold output module 305.

[0106] The data acquisition module 301 is used to obtain the historical coverage, number of missed defects, number of hits and number of found defects corresponding to each code block in the test records of each version of the software in response to receiving the coverage threshold generation instruction sent by the control terminal;

[0107] A code block determination module 302, for determining hot code blocks and non-hot code blocks according to the number of hits corresponding to each code block and the number of defects found corresponding to each code block;

[0108] A first threshold determination module 303 is used to determine a hotspot code coverage threshold corresponding to a hotspot code block according to the number of missed defects and historical coverage corresponding to the hotspot code block;

[0109] A second threshold determination module 304 is used to determine a non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and historical coverage corresponding to the non-hotspot code block;

[0110] The threshold output module 305 is used to output the hotspot code coverage threshold and the non-hotspot code coverage threshold.

[0111] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0112] In one possible implementation, the first threshold determination module 303 is specifically used to divide the historical coverage rate corresponding to the hot code block into N historical coverage rate intervals, where N is a positive integer; according to the historical coverage rate corresponding to the target historical coverage rate interval, calculate the coverage rate average value corresponding to the target historical coverage rate interval, where the target historical coverage rate interval is any historical coverage rate interval; according to the number of missed defects and historical coverage rate corresponding to each hot code block, determine the average number of missed defects corresponding to each historical coverage rate interval; use the coverage rate average value and the average number of missed defects to fit the energy consumption defect curve; determine the extreme points of the energy consumption defect curve; determine the extreme point with the largest coverage rate average value among the extreme points as the target extreme point; and determine the coverage rate average value corresponding to the target extreme point as the hot code coverage threshold.

[0113] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0114] In one possible implementation, the first threshold determination module 303 is specifically used to determine a hotspot code block whose historical coverage is within a target historical coverage interval as a reference code block; calculate the average number of missed defects corresponding to the reference code block, and obtain the average number of missed defects corresponding to the target historical coverage interval.

[0115] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0116] In one possible implementation, the first threshold determination module 303 is specifically used to derive the energy consumption defect curve, and determine the point corresponding to the first-order derivative of 0 as the critical point; calculate the second-order derivative value of the energy consumption defect curve corresponding to the critical point; if the second-order derivative value is not 0, determine the critical point as an extreme point; if the second-order derivative value is 0, calculate the derivatives of the two adjacent points on the left and right of the critical point; if the first-order derivatives of the adjacent points have different signs, determine the critical point as an extreme point.

[0117] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0118] In one possible implementation, the first threshold determination module 303 is specifically used to combine the historical coverage rates and the corresponding number of missed defects corresponding to the hotspot code block into a vector; cluster the vectors to obtain at least one cluster; and determine the largest historical coverage rate in the cluster containing the most vectors as the hotspot code coverage rate threshold.

[0119] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0120] In one possible implementation, the code block determination module 302 is specifically used to determine a high-hit code block among each code block based on the number of hits corresponding to each code block; determine a high-defect rate code block among each code block based on the number of discovered defects corresponding to each code block; if the target code block is both a high-hit code block and a high-defect rate code block, the target code block is determined as a hotspot code block, otherwise the target code block is determined as a non-hotspot code block.

[0121] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0122] In a possible implementation, the code coverage threshold generating device 300 further includes: a coverage prompting module 306 .

[0123] The coverage prompt module 306 is used to test the software in response to receiving a test instruction for the software and obtain a test result, wherein the test result includes the test coverage corresponding to each code block; if the test coverage corresponding to any hot code block of the software is less than the hot code coverage threshold, or the test coverage corresponding to any non-hot code block is less than the non-hot code coverage threshold, then a prompt indicating that the test coverage does not meet the standard is output.

[0124] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0125] In order to implement the above embodiment, the embodiment of the present application also provides an electronic device.

[0126] refer to Figure 4 , which shows a schematic diagram of the structure of an electronic device 400 suitable for implementing the embodiment of the present application, and the electronic device 400 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (Portable Media Players, PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0127] like Figure 4 As shown, the electronic device 400 may include a processor (such as a central processing unit, a graphics processing unit, etc.) 401, and a memory 402 connected to the processor in communication, which can perform various appropriate actions and processes according to the program stored in the memory 402, the computer execution instruction, or the program loaded from the storage device 408 to the random access memory (Random Access Memory, referred to as RAM) 403, to implement the code coverage threshold generation method in any of the above embodiments, wherein the memory may be a read-only memory (Read Only Memory, referred to as ROM). In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the memory 402, and the RAM 403 are connected to each other via a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0128] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0129] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the memory 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0130] It should be noted that the computer-readable storage medium mentioned above in the present application may be a computer-readable signal medium or a computer storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable storage medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0131] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0132] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0133] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0134] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] The modules involved in the embodiments of the present application may be implemented by software or hardware. The name of the unit does not limit the module itself in some cases. For example, the first threshold determination module may also be described as a "hotspot code coverage threshold determination module".

[0136] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that may be used include, without limitation, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0137] The present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the processor executes the computer-executable instructions, the technical solution of the code coverage threshold generation method in any of the above-mentioned embodiments is implemented. The implementation principle and beneficial effects are similar to those of the code coverage threshold generation method. Please refer to the implementation principle and beneficial effects of the code coverage threshold generation method, which will not be repeated here.

[0138] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the technical solution of the code coverage threshold generation method in any of the above-mentioned embodiments. Its implementation principle and beneficial effects are similar to the implementation principle and beneficial effects of the code coverage threshold generation method. Please refer to the implementation principle and beneficial effects of the code coverage threshold generation method, which will not be repeated here.

[0140] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0141] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0142] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A code coverage threshold generation method, characterized in that: include: In response to receiving a coverage threshold generation instruction sent by the control terminal, obtaining the historical coverage, number of missed defects, number of hits, and number of found defects corresponding to each code block in the test records of each version of the software; Determine hot code blocks and non-hot code blocks according to the number of hits corresponding to each code block and the number of defects found corresponding to each code block; Determine a hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and historical coverage corresponding to the hotspot code block; Determine a non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and historical coverage corresponding to the non-hotspot code block; The hotspot code coverage threshold and the non-hotspot code coverage threshold are output.

2. The method according to claim 1, characterized in that Determining the hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and the historical coverage corresponding to the hotspot code block includes: Divide the historical coverage corresponding to the hot code block into N historical coverage intervals, where N is a positive integer; According to the historical coverage rate corresponding to the target historical coverage rate interval, calculating the average coverage rate corresponding to the target historical coverage rate interval, wherein the target historical coverage rate interval is any historical coverage rate interval; According to the number of missed defects and historical coverage corresponding to each hot code block, determine the average number of missed defects corresponding to each historical coverage interval; Using the average coverage rate and the average number of missed defects to fit an energy consumption defect curve; Determining the extreme point of the energy consumption defect curve; Determine the extreme point with the largest average coverage rate among the extreme points as the target extreme point; The average coverage value corresponding to the target extreme value point is determined as the hotspot code coverage threshold.

3. The method according to claim 2, characterized in that Determining the average number of missed defects corresponding to each historical coverage interval according to the number of missed defects corresponding to each hotspot code block and the historical coverage includes: Determine the hot code block whose historical coverage is within the target historical coverage range as the reference code block; The average number of missed defects corresponding to the reference code blocks is calculated to obtain the average number of missed defects corresponding to the target historical coverage interval.

4. The method according to claim 2, characterized in that: Determining the extreme point of the energy consumption defect curve includes: Deriving the energy consumption defect curve, and determining the point corresponding to the first-order derivative being 0 as the critical point; Calculating the second-order derivative value of the energy consumption defect curve corresponding to the critical point; If the second-order derivative value is not 0, determining the critical point as an extreme point; If the second-order derivative value is 0, the derivatives of the two adjacent points on the left and right of the critical point are calculated; if the first-order derivatives of the adjacent points have different signs, the critical point is determined to be an extreme point.

5. The method according to claim 1, characterized in that Determining the hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and the historical coverage corresponding to the hotspot code block includes: Combining each historical coverage rate and the corresponding number of missed defects corresponding to the hot code block into a vector; Cluster each vector to obtain at least one cluster; The maximum historical coverage among the clusters containing the most vectors is determined as the hotspot code coverage threshold.

6. The method according to any one of claims 1 to 5, characterized in that: Determining hot code blocks and non-hot code blocks according to the number of hits corresponding to each code block and the number of defects found corresponding to each code block includes: According to the number of hits corresponding to each code block, determine a high-hit code block among each code block; According to the number of defects found in each code block, determine the code block with high defect rate among the code blocks; If the target code block is both a high-hit code block and a high-defect rate code block, the target code block is determined as a hotspot code block; otherwise, the target code block is determined as a non-hotspot code block.

7. The method according to claim 1, characterized in that After determining the non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and the historical coverage corresponding to the non-hotspot code block, the method further includes: In response to receiving a test instruction for the software, testing the software to obtain a test result, wherein the test result includes a test coverage rate corresponding to each code block; If the test coverage rate corresponding to any hotspot code block of the software is less than the hotspot code coverage rate threshold, or the test coverage rate corresponding to any non-hotspot code block is less than the non-hotspot code coverage rate threshold, a prompt indicating that the test coverage rate does not meet the standard is output.

8. A code coverage threshold generation device, characterized in that: include: A data acquisition module, configured to obtain, in response to receiving a coverage threshold generation instruction sent by the control terminal, the historical coverage, number of missed defects, number of hits, and number of found defects corresponding to each code block in the test records of each version of the software; A code block determination module, used to determine hot code blocks and non-hot code blocks according to the number of hits corresponding to each code block and the number of defects found corresponding to each code block; A first threshold determination module is used to determine a hotspot code coverage threshold corresponding to the hotspot code block according to the number of missed defects and historical coverage corresponding to the hotspot code block; A second threshold determination module is used to determine a non-hotspot code coverage threshold corresponding to the non-hotspot code block according to the number of missed defects and historical coverage corresponding to the non-hotspot code block; A threshold output module is used to output the hotspot code coverage threshold and the non-hotspot code coverage threshold.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the code coverage threshold generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the code coverage threshold generation method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the code coverage threshold generation method according to any one of claims 1 to 7.

Images