Executable Code Branch Annotations for Objective Branch Verification

Abstract

A method for objective code branch verification performs code branch annotations to evaluate developer-specified pass conditions for each code branch, where the developer-specified pass conditions indicate whether the code branch is operating as originally intended by the code author. The method includes maintaining and dynamically updating a counter value and a pass indicator status associated with each of a plurality of annotated code branches defined within the code body during execution of the code body. The counter value for each annotated code branch represents the number of times the annotated code branch has been executed within the current runtime environment, and the pass indicator status indicates whether the pass condition identified within the code branch is satisfied or not, where satisfaction or non-satisfaction is determined based on the counter value of the associated code branch.

Description

Technical Field

[0001] This application relates to software testing, and more particularly to systems and methods for objective code branch verification. Background Art

[0002] Software testing is typically planned by developers implementing a manually designed test plan that is intended to check that each code branch in the body of the code executes as expected under the expected code branch trigger conditions. Due to the large number of situations that a particular software product (such as a device driver) may experience in the field, designing an accurate and comprehensive test plan can be laborious and costly. These manually designed test plans are often imperfect and thus cannot fully test all of the code branches that may be executable. This difficulty often results in software products being released for use that incorrectly execute code branches, albeit in a manner that is undetectable to the end user. Further, in some cases, such use may lead to the accumulation of errors, which can ultimately result in severe system errors a long time after the initial incorrect execution of the code branch. Summary of the Invention

[0003] A tool for objective code branch verification maintains a counter value and a pass indicator status associated with each of a plurality of code branches defined within a body of code. The counter value for each code branch represents the number of times that the code branch has been executed within the current runtime environment, and the pass indicator status indicates whether the current counter value for the code branch satisfies a pass condition identified within the annotation for the code branch. During the execution of the body of code, the annotation for the code branch is executed to evaluate the associated pass condition. In response to each evaluation of the pass condition, the tool dynamically updates the pass indicator status for the associated code branch based on the evaluation.

[0004] The Summary of the Invention is provided to introduce a selection of concepts that are further described below in the Detailed Description. The Summary of the Invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. These and various other features and advantages will become apparent by reading the following Detailed Description. Brief Description of the Drawings

[0005] Figure 1 An example system is illustrated that objectively verifies the correct execution of each code branch in a body of code using code branch annotations and various modules.

[0006] Figure 2 Aspects of another example system are illustrated that objectively verifies the correct execution of each code branch in a body of code using code branch annotations.

[0007] Figure 3 Aspects of another example system for objectively verifying the correct execution of each code branch in a code body using code branch annotations are explained.

[0008] Figure 4 Example operations for objectively verifying the correct execution of each code branch in a code body using code branch annotations are explained.

[0009] Figure 5 An example schematic diagram of a processing device that may be suitable for implementing aspects of the disclosed technology is explained. DETAILED DESCRIPTION

[0010] Some existing software testing products utilize a technique commonly referred to as "code coverage" to provide measurements to developers that indicate which statements in a code body have been executed (e.g., when each line of code has been executed and how many times it has been executed) and which lines of code have not been executed. Code coverage products are particularly useful in helping developers verify what percentage of the code has been executed. The granularity of code coverage reaches the line-of-code level and does not consider larger groups of lines of code that make up the functional requirements of the software. A common problem in code testing occurs when the person testing the code body may not always know whether a code branch is to be executed during nominal execution. For example, a developer may write a code branch that is not to be executed (unless the system encounters a rare error). Over time, the developer may forget the purpose and / or functionality of a particular code branch and become unsure when that code branch is to be executed.

[0011] Similarly, software products may be changed many times, and these changes may cause a particular code branch to be executed less often, more often, never, or in some other unexpected manner over time. Testers implementing variants of the software product may not be aware that these changes have occurred. Similarly, code authors (both original and new) may not be able to understand why certain code branches exist and / or whether it is safe to remove seemingly stale and unnecessary code branches.

[0012] Current code testing solutions (such as those that employ code coverage techniques) provide information to testers regarding when certain lines of code have been executed and how many times they have been executed. However, this information is insufficient in a large number of test scenarios where the tester does not necessarily know the original purpose of each line of code and / or the exact conditions under which those lines of code were originally expected to be executed. For example, a tester may not know whether the non-execution of certain lines of code under nominal runtime conditions is expected.

[0013] The technology disclosed herein provides a tool that allows a programmer to easily and objectively define code branches that include groups of one or more lines of code; measure and track when those code branches are executed; measure how and when each code branch is executed; and easily determine whether a code branch is executed as expected. Additionally, these definitions that set forth the purpose of each code branch and when and how each branch is executed are encoded in such a way that the information is automatically and accurately conveyed to testers, because the definitions are within the programming code itself rather than in a test plan document, which may easily become out of sync or obsolete as the software is changed or fixed over time. According to one implementation, the foregoing is accomplished via a code branch tracking module that provides an API via a custom handle that allows a developer to annotate each code branch in a software product (e.g., driver, application, etc.) with information that the branch tracking module uses to evaluate the effectiveness and runtime integrity of the code branch when the code branch is executed. As used herein, a code branch refers to a segment of code that is conditionally executed to perform a certain functional part of an algorithm. A code branch encompasses all lines of code within a subroutine (or function) or any conditional expression, branch, or construct (e.g., "if / else if / else" statements, "switch / case" branches, loops that can be conditionally executed, and code that is expected never to be executed) that is executed when a Boolean condition that a particular programmer executes is satisfied. For example, a code branch is the logic for the solution disclosed for a software product (driver, application, etc.). A "code branch" is defined herein to explicitly exclude code segments that are for the exclusive purpose of tracking error conditions.

[0014] According to one implementation, the disclosed technology allows a programmer to embed an annotation within each code branch in the body of the code. Each embedded code branch annotation identifies a macro, referred to herein as a "pass evaluation macro." When called during the execution of a code branch, the pass evaluation macro executes to evaluate a selected pass condition for the code branch. This evaluation generates information that indicates whether the branch "correctly" ran - regardless of whether "correctly running" involves the execution or non-execution of the code branch during a nominal runtime scenario.

[0015] According to one implementation, the execution of the above code branch annotations generates information that is dynamically tracked and managed by the code branch tracking module. Among other information, the code branch tracking module maintains a pass indicator status for each code branch. The pass indicator status indicates whether the associated pass condition is met based on the current execution count for each code branch. When this tracked information is presented to the tester (the user who is testing the software), the information can allow the user to quickly and easily understand whether each code branch is being executed according to the developer's original intent, regardless of whether that original intent is satisfied via non-execution (e.g., a code branch that does not execute in a nominal execution scenario) or via execution (e.g., where the code branch executes its logic to perform the functional operation it was written to implement at the time of original writing).

[0016] The above information that is generated via the execution of the branch annotations and tracked by the code branch tracking module effectively enhances the ability of code testers to quickly and efficiently identify code branches that are not being executed correctly, thereby allowing for the rapid fixing of both hard-to-detect coding errors and inefficiencies that might otherwise go undetected over an extended period of time. Code produced and released using the code branch tracking system disclosed herein is thus more likely to run more efficiently (e.g., faster) and in an error-free manner compared to code produced and released using traditional techniques. From the perspective of the end user (e.g., a user interacting with the released computer application), this translates into an improvement in the operation of the computer functionality, where the user experiences a more streamlined interaction with the application and is less likely to experience the frustrations of various crash scenarios such as causing system downtime and irreparable corruption of user data.

[0017] Figure 1 An example system 100 is illustrated that utilizes code branch annotations and a code branch tracking module 104 to objectively verify whether each code branch in a code body 102 is being executed according to the original intent of the developer who wrote the code branch. As used herein, "original intent" refers to the initial purpose for which the code branch was written. This initial purpose can be understood to include the expected trigger condition (e.g., the condition that causes the branch to execute) and the expected functional effect achieved via the execution of the code branch.

[0018] In different implementations, the code body 102 can take various different forms, including but not limited to system drivers, user applications, operating systems, etc. By way of example and not limitation, the code body 102 is shown as including three code branches (branch

[001] , branch

[002] , and

[003] ). Each branch continues conditionally (e.g., if / else if / else). Although not shown, it is assumed that each of these three code branches includes code that constitutes the functional solution provided by the code body 102. In the case where the code body 102 is a driver, each branch facilitates the solution exposed by the driver.

[0019] Each of the code branches

[001] ,

[002] , and

[003] in the code body 102 includes a comment (e.g., the code branch comment 108). When the code branch is executed, the comment contained in that code branch is also executed, which causes the code branch tracking module 104 to accumulate information such as Figure 1 the exemplary information shown in the branch tracking table 112. The code branch tracking module 104 includes various pass evaluation macros 110, which can be understood as functions selectively invoked by the code branch comments (e.g., comment 108) to evaluate whether each code branch is executed according to the developer's original intention.

[0020] Although the code branch comments (e.g., 108) can be added to the code at various time points, it is assumed that in at least one implementation the comments are added by the developer when the code is initially written. According to one implementation, each code branch comment 108 identifies a selected one of the pass evaluation macros 110 that can be executed to evaluate a particular developer-specified "pass condition" for the associated code branch. Each pass condition can be understood as a set of objective, verifiable, developer-specified criteria that are used as a measure to ensure that the code branch has been executed in a manner that conforms to the developer's original intention.

[0021] In one implementation, one or more of the pass evaluation macros 110 evaluate whether a pass condition for a code branch is met by comparing the execution count of the code branch (e.g., the number of times the code branch has been executed) with a set execution count specified by the developer as being sufficient to reasonably assume that the code is operating correctly. For example, as long as the associated code branch never executes, one of the pass evaluation macros 110 can evaluate that the pass condition "never executes" is met. As long as the code branch has executed exactly once, another example of a pass evaluation macro 110 can evaluate that the pass condition "executes once" is met. When the code branch has executed fewer than a threshold number of times (where the threshold is specified by the developer), another example of a pass evaluation macro 110 evaluates that the pass condition "executes fewer than" is met. In this and other scenarios (e.g., "executes more than" or "executes at least"), the threshold count for the pass condition can be indicated in the annotation 108. Further exemplary pass evaluation macros 110 of the code branch tracking module 104 are defined below with reference to Figure 2 to define.

[0022] When each code branch is executed within the code body 102, the embedded annotation is executed to determine whether the pass condition indicated by the annotation is met by the execution count of the associated code branch. Throughout this process, the code branch tracking module 104 records certain information, such as the exemplary information shown in the branch trace table 112.

[0023] In Figure 1 , the branch trace table 112 includes the branch ID of each branch that has been annotated as described above in the code body 102. During the execution of the code body 102, the code branch tracking module 104 maintains and dynamically updates the execution count value 114 for each annotated code branch. The execution count value 114 indicates the number of times the associated code branch has been executed during the current runtime scenario. Additionally, the code branch tracking module 104 also maintains and dynamically updates the pass indicator status 116 associated with each annotated code branch.

[0024] Generally, the pass / fail status of the pass condition evaluated by a selected pass evaluation macro called by an annotation in an associated code branch in the pass evaluation macro is indicated by the indicator status 116. For example, the code branch

[001] can be verified by a pass evaluation macro that evaluates the pass condition "execute at least five times". This exemplary pass condition is satisfied when the associated branch has been executed at least five times. Since the exemplary data in the branch trace table 112 indicates that the branch

[001] has been executed 10 times, the pass indicator status 116 indicates a "pass" value. Continuing with the above example, the branch

[002] can be verified by a pass evaluation macro that evaluates the pass condition "execute exactly once". Since the branch

[002] has been executed four times in this example, the pass indicator status 116 indicates a "fail" value.

[0025] It is worth noting that the pass condition for some code branches can be satisfied when the code branch has not been executed at all. For example, the branch trace table 112 indicates that the code branch

[003] has a "pass" for the pass indicator status 116 even though it has been executed zero times. For example, this may be the case if the code branch

[003] is expected to perform an aspect of error handling during an exceptional run-time scenario.

[0026] To ensure adequate evaluation of branches that are not executed in normal circumstances, such as branch

[003] , the system 100 can be configured to initially determine and populate the pass indicator status 116 for each branch at compile time before the code body 102 begins execution. For example, a compiler (not shown) of the code branch trace module 104 can analyze the annotations in the code body 102 at compile time and determine whether the associated pass condition is satisfied by a "zero" execution count value. If so, the associated pass indicator status 116 can be initially populated with a "pass" value instead of "fail". Although the pass indicator status 116 for each code branch is subject to change during the execution of the code body 102, this initial population of the pass indicator status 116 at compile time allows testers to determine whether it is normal for the branch not to be executed - a key technical advantage over existing code coverage solutions.

[0027] When a code branch is executed, the annotations contained in the code branch are also executed, which causes the code branch trace module 104 to continuously accumulate and update the information in the branch trace table 112. The accumulated information can be retrieved by the code branch trace reader 118 and presented to the display, and the tester can optionally call the code branch trace reader 118 to examine all or a portion of the accumulated information. In one implementation, the code branch trace reader 118 displays all annotations for which the associated pass condition has not been satisfied. In another implementation, the code branch trace reader 118 displays a score indicating what percentage of the annotations have passed.

[0028] In some implementations, a tester can use different filter options to command the code branch trace reader 118 to request a rendering of different subsets of the information accumulated by the code branch tracing module. For example, a tester may wish to view information only related to code branches having a "failed" value for the indicator status 116 or information related to specific code branches of interest. In this way, the accumulated information can optionally be presented to the tester, which provides a complete picture of whether each code branch in the code body 102 is executed in accordance with the pass conditions initially specified by the developer who wrote the comments.

[0029] As further reference Figure 2 As described, the comments within each code branch can include additional information that allows the tester to also understand the conditions under which each code branch is expected to execute. If the code branch itself does not execute in a manner that satisfies the pass conditions assigned to it by the developer, this allows the tester to manually perform actions that force the code branch to execute during testing to allow the branch to meet the pass conditions.

[0030] Figure 2 Aspects of another system 200 are illustrated that utilize code branch comments to objectively verify whether each branch in the code body 202 is executing according to the developer's original intent. Once instantiated in a test environment, the code branch tracing module 204 provides an API that allows a programmer to annotate each code branch in the code body (e.g., a driver) with certain information. Similar Figure 1 to the code branch tracing module 104, the code branch tracing module 204 includes an evaluation macro 210, and the evaluation macro 210 can be invoked via the execution of comments included in the code branches of the code body 202. Although the code branch tracing module 204 can be designed to execute across different computing platforms in various ways for testing different types of code, the code branch tracing module 204 shown by way of example and not limitation is a "DMF module", which is created using a driver module framework for developers to build Windows drivers. In this example, the code body 202 is a client driver.

[0031] In Figure 2 it, the code body 202 is shown to include a code branch 208, and the code branch 208 includes a code branch comment 206. When the code branch 208 is executed, the code branch comment 206 executes the evaluation macro 214 included in the code branch. By way of example and not limitation, Figure 2Table 224 is explained. Table 224 explains various pass evaluation macros other than the description of the criteria required to meet the associated pass conditions. In the example being explained, the code branch annotation 206 includes a pass evaluation macro 214 named "branchtrack_count_at_least", which is one of the multiple pass evaluation macros 210 disclosed by the code branch tracking module 204. When the code branch tracking module 204 has executed at least the "count" number of times, the pass condition of the pass rating macro 214 is evaluated as being met. In the example being explained, the "count" is the threshold execution count value 212 (e.g., value 8) included by the developer within the code branch annotation 206. This value is, for example, a value that the developer believes is sufficient to indicate with a certain reasonable certainty that the code branch is executing correctly.

[0032] As an example and not a limitation, the code branch annotation 206 is also shown as assigning a handle (e.g., a DMF handle) to the instantiated code branch tracking module 204 (e.g., "Dmf module branch tracking"). This handle provides a connection to the code that tracks the branch at runtime, and the developer can use this handle to cause the code branch tracking module 204 to perform its accounting of the execution of the code branch based on the annotation.

[0033] The code branch annotation 206 also includes the name 218 of the code branch in which the code branch annotation 206 is embedded. For example, the name 218 can be a human-readable name that allows the code branch to be uniquely identified. In some cases, the name 218 can be the name of the function in which the code branch exists along with the conditions that cause the code branch to execute. For example, the name 218 ("ButtonState.Power.Down") can be used when the code branch is embedded within a function related to a power button control and when the associated code branch is expected to be called when the physical state of the power button is "down". If retrieved and presented to the tester in association with pass status indicator data (e.g., as described in reference Figure 1 ), the name 218 provides the tester with an understanding of the conditions that the developer expects to trigger the execution of the code branch.

[0034] In some implementations, the code branch annotation may also include human-readable text that explains how the pass conditions selected by the developer for the code branch are met. For example, the code branch annotation 206 is shown as including human-readable text 220. In the example being explained, this text notifies the tester that the code branch has a pass condition that is met when the power button is pressed at least 8 times (triggering 8 executions of the branch).

[0035] Effectively, the human-readable text 220 provides the tester with means to perform actions to satisfactorily test the code branch until the associated pass condition of the branch is met. For example, during a driver test for a power button, the information presented to the tester may indicate that the code branch 208 has a "failed" status for the pass indicator. This could imply that the code branch 208 was not called when it should have been or simply that the tester has not yet performed the actions identified by the developer as sufficient to ensure that the code branch is operating correctly. The human-readable text 220 (e.g., "Press the power button at least 8 times") informs the tester what is required to meet the developer-defined pass condition. Thus, in this scenario, the tester may press the power button several additional times to change the status of the code branch to "passed".

[0036] In some implementations, the code branch annotation may include additional information, as a replacement or supplement to the exemplary information shown in the code branch annotation 206. For example, some implementations may include annotations that include optional filters that allow the code branch 208 to be enabled or disabled for various execution modes. Other implementations provide optional conditions that may cause the code branch 208 to be automatically opened or closed based on the runtime environment. Other aspects of the system 200 not explicitly described above may be the same as or similar to those Figure 1 described with reference

[0037] Figure 3 Aspects of another system 300 are illustrated that utilize code branch annotations to objectively verify the correct execution of each code branch in the code body 302. In one implementation, the code body 302 includes a plurality of code branches, each of which has been annotated by the developer with information that is the same as or similar to that Figure 2 described for the code branch annotation 206 and / or Figure 1 described for 108.

[0038] Before the code body 302 is executed, the branch tracking cursor 312 is initialized. In Figure 3 the example, the code branch tracking module 304 fills the branch tracking table 312 with various information including at least the branch ID 310. The branch ID 310 may include, for example, a numeric identifier for each code branch and / or a unique branch name that indicates the function including the branch and the conditions expected to trigger the execution of the branch (e.g., such as Figure 2The name shown (218). Additionally, the branch trace table 312 is initialized to include a field (count value 314) that tracks the number of times each branch is executed within the same runtime environment. Initially, all count values are initialized with a starting value of 0 to reflect that no branches have been executed at the time the table is created. The branch trace table 312 also includes a pass indicator 316 associated with each code branch. In one implementation, the pass indicator 316 is a "pass" or "fail" value indicating whether the pass condition identified by the annotation of the associated code branch is satisfied or not. For example, the pass condition is defined by the selected pass evaluation macro alone or in combination with one or more evaluation thresholds specified in the annotation. In some implementations, the code branch annotation may also include the pass condition in human-readable text form.

[0039] When initially populating the code branch trace table 312, the code branch trace module 304 reads the annotations within each code branch and determines whether the pass condition defined for each code branch is satisfied by the initialized (0) count value. For example, a code branch that is not expected to be executed during nominal conditions may have a pass condition that is satisfied by the initial "0" count value. The code branch trace module 304 sets the initial pass indicator to "pass" for these branches and to "fail" for all other branches.

[0040] After the branch trace table 312 is instantiated and initialized with the starting values as described above, a tester (one tester) may initiate the execution of the code body. When the code body 302 is executed during runtime, the annotation is executed when encountered. According to one implementation, the execution of each annotation requires the execution of inline code that has been generated by the compiler at compile time based on the original annotation. The execution of this inline code uses the code branch trace module 304 to update the branch trace table 312 maintained for the code body 302 to increment the count value of the code branch associated with the annotation. Additionally, the execution of each annotation calls the selected pass evaluation macro that evaluates the corresponding pass condition. Whether the pass condition is satisfied or not depends on the count value 314 of the branch code maintained and dynamically updated within the branch trace table 312 during the execution of the code body 302.

[0041] When testing the code body 302, such as when implementing changes for a subsequent version of the code, a tester can command the code branch trace reader 320 to retrieve selected information from the branch trace table 312 and present the selected information to a display. For example, the branch trace summary 322 is shown to indicate an example type and format of information that can be presented to the display during or after the execution of the code body 302. In the illustrated example, the branch trace summary 322 includes branch ID information (which includes numeric IDs (e.g.,

[001] ,

[002] ,

[003] )) and branch names 326 that indicate the expected trigger conditions (e.g., when a developer expects the branch to execute). For example, the branch name "ButtonState.Power.Down" indicates that the developer expects the branch to execute in response to a press of the power button. Similarly, the branch name "ButtonState.Power.up (Button State.Power.Up)" indicates that the developer expects the branch to execute in response to the release of the power button. As a further example, the name "ButtonState.PwrButtonDisabled (Button State.Power Button Disabled)" indicates that the developer expects the branch to execute when the system disables the power button.

[0042] In addition to the branch names 326, the branch trace summary 322 includes human-readable text 324 that indicates how the associated pass condition is met. For example, the human-readable text 324 indicates that the branch "ButtonState.Power.Down" has a pass condition that is met when the system detects 8 presses of the power button. Similarly, the human-readable text for the branch named "ButtonState.PwrButtonDisabled" indicates that the pass condition is met when the branch never executes. In one implementation, the branch names 326 and the human-readable text 324 are specified within the comments of the associated code branch.

[0043] The branch trace summary 322 also indicates the current pass indicator status for each branch. In this example, branch

[001] has a "failed" value for the pass indicator because the branch has not been executed enough times to meet the developer-defined pass condition. In this scenario, the branch trace summary 322 immediately provides the tester with meaningful information that can be crucial in helping the tester identify information such as: (1) the comprehensive identity of the code branches that may be experiencing problems (notably, the "failed" status can be triggered by non-execution or over-execution that is contrary to the developer-specified pass condition); (2) the trigger conditions that set when the developer expects the execution of each code branch to be triggered; and (3) the metrics (e.g., the pass conditions of the human-readable text) that the tester can use to troubleshoot whether the branch is actually running in accordance with the developer's original intent.

[0044] For example, a tester presented with the branch trace summary 322 can attempt to change the "failed" pass indicator status for branches

[001] and

[002] to "passed" by systematically pressing and releasing the power button 8 times. If the branches are operating correctly, the tester can expect the status to switch from "failed" to "passed" after eight presses. Since this test was designed by the original developer of the code branch, the tester can reasonably ensure that he has verified the correct execution of all the expected trigger paths for that branch. When using the above method to systematically test all the code branches in the code body 302, the tester can reasonably ensure that each branch has been tested against all the expected trigger conditions of the original developer.

[0045] Figure 4 An example computer-implemented operation 400 for objectively verifying the correct execution of each code branch in a code body using code branch annotations is illustrated. Prior to operation 400, the software author has written the code body, defined the branches in the code, and annotated each code branch to include executable annotations operable to evaluate both when the branch should execute and whether the branch executes as expected. When the body of the code is compiled, the compiler generates inline code based on the branch annotations of each branch to be executed when the code branch is executed. At program initialization, an initialization operation 402 initializes branch trace information for each of the multiple annotated code branches in the code body. According to one implementation, the branch trace information includes a branch ID field, a count value indicating the number of times the code branch has been executed, and a pass indicator status indicating whether the pass condition identified by the annotation is met or not. The initialization operation 402 fills the branch ID field with the branch ID of each annotated branch in the code body and initializes the count value of each branch to a value of 0. During this stage, a code branch trace module (e.g., Figure 3 304) reads each annotation into a table and executes a pass evaluation macro identified within the annotation to evaluate whether the initial counter value (0) of the code branch meets the pass condition provided by the annotation within the code branch. In the case where the "0" counter value does meet the pass condition defined by the annotation, the initialization operation sets the pass indicator status to "passed". Otherwise, the initialization operation 402 sets the pass indicator status of the code branch to "failed".

[0046] Subsequently, operation 404 begins to execute the functional code of the code. When the code body is executed, the operation sequence 406-416 (described below) can be executed multiple times, once for each of the annotated code branches. Determination operation 406 determines whether a branch trace annotation is encountered when branch tracing is enabled. If not, operation 404 continues to execute the functional code of the code body. When determination operation 406 determines that an annotation has been encountered when branch tracing is enabled, counter increment operation 408 increments the count value of the code branch, and execution operation 410 executes the annotation (e.g., thereby executing the inline code generated during compilation of the code body) to both evaluate the passing condition and update the branch trace table 312.

[0047] According to one implementation, the passing condition sets forth one or more rules for determining whether the current execution of the code conforms to the original intent of the code author. The annotation can, for example, include a function call to a specific passing evaluation macro, one or more thresholds for evaluating the passing condition, or (in some cases) human-readable text explaining the passing condition. Execution of the annotation invokes a passing evaluation macro that evaluates satisfaction of the passing condition based on the current counter value of the code branch and / or other information (such as information included within the code branch).

[0048] Determination operation 415 determines whether the passing condition of the branch is satisfied by the current counter value of the branch. If the passing condition is not satisfied, recording operation 414 checks the most recent passing indicator status of the branch and updates the passing indicator status to "failed" (if it has not already been set to "failed"). If the passing condition is satisfied, recording operation 416 checks the most recent passing indicator status of the branch and updates the passing indicator status to "passed" (if it has not already been set to "passed"). At any time during or after the operation of operation 400, a tester can command a read operation to retrieve the branch trace information and present the branch trace information to a display.

[0049] Figure 5 An example schematic diagram of a processing device 500 that can be suitable for implementing aspects of the disclosed technology is illustrated. Processing device 500 includes one or more processor units 502, a memory 504, a display 506, and other interfaces 508 (e.g., buttons). Memory 504 generally includes both volatile memory (e.g., RAM) and non-volatile memory (e.g., flash memory). An operating system 510 (such as the Microsoft operating system, the Microsoft Phone operating system, or a specific operating system designed for a gaming device) resides in memory 504 and is executed by the processor unit(s) 502, but it should be understood that other operating systems can be employed.

[0050] One or more applications 512 (e.g., such as Figure 1 the code body 102, the code branch tracking module 104, and the code branch tracking reader 118) are loaded into the memory 504 and executed by the processor unit 502 on the operating system 510.

[0051] The application 512 can receive input from various input local devices (not shown), such as microphones, keyboards, mice, styli, touchpads, joysticks, etc. In addition, the application 512 can receive input from such devices by communicating with one or more remote devices (such as remotely located smart devices) via a wired or wireless network using the more communication transceiver 530 and the antenna 532 to provide network connectivity (e.g., mobile phone network, )

[0052] The processing device 500 further includes a storage device 528 and a power supply 516, which is powered by one or more batteries (e.g., battery 520) and / or other power sources and provides power to other components of the processing device 500. The power supply 516 can also be connected to an external power source (not shown), which overrides or recharges the built-in battery or other power source.

[0053] In an example implementation, the code branch tracking module and / or the code branch tracking reader include hardware and / or software implemented by instructions stored in the memory 504 and / or the storage device 528 and processed by the processor unit 502. The memory 504 can be the memory of the host device or an accessory coupled to the host.

[0054] The processing device 500 may include various tangible computer readable storage media and intangible computer readable communication signals. Tangible computer readable storage may be embodied by any available medium accessible by the processing device 500 and includes both volatile and nonvolatile storage media, both removable and nonremovable storage media. Tangible computer readable storage media does not include intangible and transient communication signals, but rather includes volatile and nonvolatile, removable and nonremovable storage media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules, or other data. Tangible computer readable media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic tape cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other tangible medium that can be used to store the desired information and that is accessible by the processing device 500. In contrast to tangible computer readable storage media, intangible computer readable communication signals may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other signal transmission mechanism. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, intangible communication signals include wired media such as a wired network or direct line connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0055] Some embodiments may include an article of manufacture. The article of manufacture may include a tangible storage medium (memory device) for storing logic. Examples of storage media may include one or more types of processor-readable storage media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and the like. Examples of logic may include various software elements, such as software components, programs, applications, computer programs, applications programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, operational segments, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computing code, computer code, code segments, computer code segments, literal values, symbols, or any combination thereof. For example, in one implementation, the article of manufacture may store executable computer program instructions that, when executed by a computer, cause the computer to perform the methods and / or operations according to the described embodiments. The executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. The executable computer program instructions may be implemented according to a predefined computer language, manner, or syntax for instructing a computer to perform a particular operational segment. These instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0056] An example system disclosed herein includes a code branch tracking module stored in a memory that performs the following operations: maintaining a counter value associated with each of a plurality of code branches defined within a code body during execution of the code body; and dynamically updating and maintaining a pass indicator status for each of the code branches during execution of the code body. The counter value for each code branch represents the number of times the code branch has been executed within the current runtime environment, and the pass indicator status indicates whether a pass condition identified within the code branch is satisfied or not, the satisfaction or non-satisfaction being based on the associated counter value of the code branch.

[0057] In an example system according to any of the foregoing systems, the pass condition for each of the code branches is identified within a comment included within the code branch, the comment referencing a pass evaluation macro that is executed when the code branch is executed to evaluate the pass condition.

[0058] In another example system of any of the foregoing systems, the pass condition for a selected branch of the code branches is satisfied by execution of the selected branch, and the pass indicator status for the selected branch indicates a failure status when the code branch has not been executed.

[0059] In another example system of any of the foregoing systems, the pass condition is a metric selected by a developer that can be used to verify that an associated code branch has been executed according to the developer's original intent.

[0060] In another example system of any of the foregoing systems, the annotation for each of the plurality of code branches identifies the condition for triggering the execution of the code branch.

[0061] In another example system of any of the foregoing systems, the code branch tracking module is further configured to: re-evaluate the satisfaction of the pass condition for a selected branch of the code branches in response to incrementing a counter value for the selected branch; and update the pass indicator status for the selected branch based on the re-evaluation.

[0062] Another example system of any of the foregoing systems further includes a reader tool stored in a memory that performs the following operations: queries the code branch tracking module with a request for information related to the execution of the code body, where the requested information includes the pass indicator status for one or more of the plurality of code branches. The reader tool receives the requested information and presents the requested information on a display of the computing device.

[0063] In another example system of any of the foregoing systems, the pass condition for each of the code branches is evaluated by a pass evaluation macro that is identified in an annotation included within the code branch, and the pass evaluation macro can be executed to evaluate whether a count value for the code branch matches a threshold count value selected by the developer.

[0064] An example method disclosed herein includes: maintaining a counter value associated with each of a plurality of code branches defined within the code body during execution of the code body; and dynamically updating and maintaining a pass indicator status for each of the code branches during execution of the code body. The counter value for each code branch represents the number of times the code branch has been executed within the current runtime environment, and the pass indicator status indicates whether a pass condition identified within the code branch is satisfied or not satisfied, where satisfaction or non-satisfaction is based on the associated counter value of the code branch.

[0065] In another example method according to any of the foregoing methods, the pass condition for each code branch is identified within an annotation included within the code branch. The annotation references a pass evaluation macro that is executed when the code branch is executed to evaluate the pass condition. In another example method of any of the foregoing methods, the annotation in at least one of the plurality of code branches includes a threshold count value selected by the developer, and the threshold count value selected by the developer is used by the pass evaluation macro to evaluate the pass condition.

[0066] In another example method of any of the foregoing methods, the pass condition identified by each code branch is a metric selected by the developer that can be used to verify that the code branch has been executed according to the developer's original intent.

[0067] In another example method of any of the foregoing methods, the annotation includes a branch name identifying a condition sufficient to trigger the execution of the code branch.

[0068] In another example method of any of the foregoing methods, the annotation includes human-readable text indicating the pass condition.

[0069] In another example method of any of the foregoing methods, the method further includes: re-evaluating the satisfaction of the pass condition for the selected branch in response to incrementing a counter value for the selected branch in the code branch; and updating the pass indicator status for the selected code branch based on the re-evaluation.

[0070] Another example method of any of the foregoing methods further includes: requesting information related to the execution of the code body, including the pass indicator status for one or more of the plurality of code branches; and presenting the requested information on a display of the computing device.

[0071] In another example method of any of the foregoing methods, the pass condition for each code branch is evaluated by a pass evaluation macro that is identified in an annotation included within the code branch. The pass evaluation macro can be executed to evaluate whether a count value for the code branch matches a threshold count value selected by the developer.

[0072] Another example system disclosed herein includes a code body stored in a memory and including code branches, the code branches having annotations that identify pass conditions that can be used to verify whether the execution of the code branches conforms to the original intent of the developer who authored the code branches. The annotations can be executed to evaluate the satisfaction of the pass conditions based on the determined number of times the code branches have been executed within the current runtime environment prior to the evaluation.

[0073] In another example system of any of the foregoing systems, the system further includes a code branch tracking module stored in the memory that dynamically updates and maintains a pass indicator status for each code branch during the execution of the code body. The pass indicator status indicates whether the pass conditions identified by the annotations of each code branch are satisfied or not.

[0074] Another example system of any of the foregoing systems further includes a reader tool stored in a memory that performs the following operations: query the code branch tracing module with a request for information related to the execution of the code body, the requested information including a pass indicator status for the code branch; and present the requested information on a display of the computing device.

[0075] Another example system disclosed herein includes: means for maintaining a counter value associated with each of a plurality of code branches defined within a code body during execution of the code body; and means for dynamically updating and maintaining a pass indicator status for each of the code branches during execution of the code body. The counter value for each code branch represents the number of times that the code branch has been executed within the current runtime environment, and the pass indicator status indicates whether a pass condition identified within the code branch is met or not, the met or not being based on the associated counter value of the code branch.

[0076] In another example memory device of any of the foregoing memory devices, the memory space is a contiguous sequence of sequential physical blocks.

[0077] The implementations described herein can be implemented as logical steps in one or more computer systems. The logical operations can be implemented as: (1) a sequence of processor-implemented steps executed in one or more computer systems; and (2) interconnected machine or circuit modules within one or more computer systems. The implementation is a matter of choice depending on the performance requirements of the computer systems utilized. Accordingly, the logical operations that make up the implementations described herein may also be referred to as operations, steps, objects, or modules. Additionally, it should be understood that the logical operations can be performed in any order unless explicitly stated or the claim language inherently requires a particular order. The foregoing description, examples, and data, along with the accompanying drawings, provide a complete description of the structure and use of the exemplary implementations.

Claims

1. A system for objective code branch verification, comprising: A code branch tracking module stored in a memory, which performs the following operations: Maintaining a counter value associated with each of a plurality of code branches defined within the code body during execution of the code body, the counter value for each of the code branches representing the number of times the code branch has been executed within the current runtime environment; And Dynamically updating and maintaining a pass indicator status for each of the code branches during execution of the code body in response to execution of an annotation included in the code branch, the pass indicator status indicating whether a pass condition identified within the code branch is satisfied or not, the satisfaction or non - satisfaction being based on the associated counter value for the code branch, the annotation within each of the code branches identifying: A pass evaluation macro; and A threshold for the pass evaluation macro of the code branch provided as input to the code branch, wherein a developer can specify a threshold for each of the plurality of code branches, and the pass evaluation macro is executed when the code branch is executed to evaluate the pass condition based on a comparison between the threshold and the counter value.

2. The system according to claim 1, wherein the pass condition for a selected branch among the code branches is satisfied by execution of the selected branch, and the pass indicator status for the selected branch indicates a failure status when the code branch has not been executed.

3. The system according to claim 1, wherein the pass condition is a metric selected by a developer that can be used to verify that an associated code branch has been executed according to the developer's original intention.

4. The system according to claim 1, wherein the annotation for each of the plurality of code branches identifies a condition for triggering execution of the code branch.

5. The system according to claim 1, wherein the code branch tracking module is further configured to: Re - evaluate satisfaction of the pass condition for the selected branch in response to incrementing the counter value for the selected branch among the code branches; and Update the pass indicator status for the selected branch based on the re - evaluation.

6. The system according to claim 1, further comprising: A reader tool stored in a memory, which performs the following operations: Querying the code branch tracking module by requesting information related to execution of the code body, the requested information including the pass indicator status for one or more of the plurality of code branches; And Presenting the requested information on a display of a computing device.

7. The system according to claim 1, wherein the pass evaluation macro can be executed to evaluate whether the counter value for the code branch matches a threshold count value selected by a developer.

8. A method for objective code branch verification, comprising: Maintain a counter value during the execution of the code body in association with each of a plurality of code branches defined within the code body in response to the execution of an annotation included in a code branch, the counter value for each of the code branches representing the number of times the code branch has been executed within the current runtime environment; And During the execution of the code body, dynamically update and maintain a pass indicator status for each of the code branches, the pass indicator status indicating whether a pass condition identified within the code branch is satisfied or not, the satisfaction or non - satisfaction being based on the associated counter value for the code branch, the annotation within each of the code branches identifying: A pass evaluation macro; and A threshold for the code branch that is provided as input to the pass evaluation macro for the code branch, wherein a developer can specify a threshold for each of the plurality of code branches, and the pass evaluation macro is executed when the code branch is executed to evaluate the pass condition based on a comparison between the threshold and the counter value.

9. The method according to claim 8, wherein the pass condition identified by each code branch is a metric selected by the developer that can be used to verify that the code branch has been executed according to the developer's original intention.

10. The method according to claim 8, wherein the annotation includes a branch name that identifies a condition sufficient to trigger the execution of the code branch.

11. The method according to claim 8, wherein the annotation includes human - readable text indicating the pass condition.

12. The method according to claim 8, further comprising: Upon incrementing the counter value for a selected branch among the code branches, re - evaluate the satisfaction of the pass condition for the selected branch; And Update the pass indicator status for the selected branch based on the re - evaluation.

13. The method according to claim 8, further comprising: Request information related to the execution of the code body, the requested information including the pass indicator status for one or more of the plurality of code branches; And Present the requested information on a display of a computing device.

14. The method according to claim 8, wherein the pass evaluation macro can be executed to evaluate whether the counter value for the code branch matches a threshold count value selected by the developer.

15. A system for objective code branch verification, comprising: A code body stored in a memory and executable by a processor, the code body including code branches having annotations that identify pass conditions that can be used to verify whether the execution of the code branches conforms to the original intention of the developer who wrote the code branches, the annotations including: A pass evaluation macro; and The threshold of the code branch through the evaluation macro provided as input to the code branch, where a developer can specify a threshold for each code branch, and the evaluation macro is executed when the code branch is executed to evaluate the passing condition based on a comparison between the threshold and the determined number of times the code branch has been executed within the current runtime environment.

16. The system of claim 15, further comprising: A code branch tracking module stored in a memory, the code branch tracking module dynamically updating and maintaining a passing indicator status for each of the code branches during execution of the code body, the passing indicator status indicating whether the passing condition identified by the annotation of each code branch is satisfied or not.

17. The system of claim 16, further comprising: A reader tool stored in a memory, which performs the following operations: Querying the code branch tracking module by requesting information related to the execution of the code body, the requested information including the passing indicator status for the code branch; And Presenting the requested information on a display of the computing device.

18. A computer system comprising an apparatus for performing the method of any one of claims 8 - 14.

19. A computer-readable storage medium having instructions stored thereon, the instructions when executed causing a machine to perform the method of any one of claims 8 - 14.

Images