Code comment quality assurance

Abstract

To provide outputs, such as a rank for a particular comment, based on comment quality, thereby providing an automated, data-driven approach to comment quality evaluation.SOLUTION: Code comment quality assurance includes the steps of: calculating a complexity of a code portion; extracting, from the code portion, one or more comments associated with the code portion; converting the one or more comments into a set of text features; quantifying the set of text features; determining, for the one or more comments, alignment between the one or more comments and the associated code portion using the quantifications of the set of text features and the complexity of the code portion; and triggering a notification responsive to the determination that the one or more comments and the associated code portion are misaligned.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to methods, apparatuses, and products for code comment quality assurance.

Background Art

[0002] In complex software systems, there may be multiple code files, each containing multiple code comments. As is known to those skilled in the art, code comments may be intended to explain, represent, or highlight one or more aspects of computer code. The creator of a code comment (e.g., the creator of the computer code) may intend for the code comment to provide useful background or explanatory information to the reader of the code (e.g., another computer programmer). Thus, code comments can take the form of an explanation, illustration, or clarification of the functionality embodied by the computer code. While the computer code can be in a computer programming language (e.g., COBOL or Java (registered trademark)), the code comments can be in a natural language (e.g., plain English). While the reader expects the code comments to represent or explain the code, the code comments can be of varying quality, and in some cases, there may even be a lack of code comments in places where the reader could benefit from their presence with respect to some code. In other cases, the code comments may fail to fully represent or explain one or more aspects of the associated computer code, thereby limiting the usefulness of the code comments. Also, modern artificial intelligence systems can similarly generate code comments, but existing systems may fail to generate code comments that provide a sufficient and consistent explanation of the purpose and functionality of code segments.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Based on comment quality, provide an output, e.g., a rank for a specific comment, thereby providing an automated data-driven approach to comment quality assessment.

Means for Solving the Problem

[0004] According to embodiments of the present disclosure, various methods, apparatuses, and products for code comment quality assurance are described herein. In some aspects, code comment quality assurance includes calculating the complexity of a code portion, extracting one or more comments associated with the code portion from the code portion, converting the one or more comments into a set of text features, quantifying the set of text features, using the quantification of the set of text features and the complexity of the code portion to determine the content consistency between the one or more comments and the associated code portion, and triggering a notification in response to a determination that the content of the one or more comments and the associated code portion is inconsistent.

Brief Description of the Drawings

[0005]

Figure 1

[0006]

Figure 2

[0007]

Figure 3

[0008]

Figure 4

[0009]

Figure 5

DETAILED DESCRIPTION OF THE INVENTION

[0010] The following disclosure describes exemplary embodiments for code comment quality assurance. In some embodiments, the system described analyzes the correlation between human language and computer code. As described herein, human language or natural language can be exemplified by any spoken, written, or expressed language (e.g., English, Japanese, American Sign Language, etc.). Examples of computer code include any computer-readable code composed of computer code written in one or more programming languages, such as Java, C++, COBOL, or the like. Analyzing the correlation between human language and computer code can include analyzing the content of a portion of the computer code and the content of any associated code comments written in natural language.

[0011] Referring now to FIG. 1, an exemplary computing environment in accordance with an aspect of the present disclosure is shown. Computing environment 100 is an example of an environment for executing at least a portion of computer code involved in performing the various methods described herein, for example, including quality assurance module 107. Quality assurance module 107 may be configured to determine aspects of computer code and associated code comments, such as code complexity levels or code comment features, and to identify correlations between the computer code and the code comments. Based on the identified correlations or degree of correlation, quality assurance module 107 may predict or determine a level of content consistency between the computer code and the code comments. If the code comments are predicted to be inconsistent in content compared to the computer code, quality assurance module 107 may trigger an alert or notification that prompts, for example, a human user or an artificial intelligence system to generate code comments that are more closely content-consistent with the computer code.

[0012] In addition to block 107, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes processor set 110 (including processing circuit 120 and cache 121), communication fabric 111, volatile memory 112, persistent storage 113 (including operating system 122 and block 107 as identified above), peripheral device set 114 (including user interface (UI) device set 123, storage 124, and Internet of Things (IoT) sensor set 125), and network module 115. Remote server 104 includes remote database 130. Public cloud 105 includes gateway 140, cloud orchestration module 141, host physical machine set 142, virtual machine set 143, and container set 144.

[0013] Computer 101 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of currently known or future-developed computer or mobile device capable of executing a program, accessing a network, or querying a database, such as remote database 130. As is well understood in the field of computer technology and depending on the technology, the execution of computer-implemented methods can be distributed among multiple computers and / or among multiple locations. On the other hand, in this description of computing environment 100, for the sake of making the description as concise as possible, the detailed discussion focuses on a single computer, specifically computer 101. Although computer 101 is not shown within the cloud in FIG. 1, it may be located within the cloud. On the other hand, computer 101 does not need to exist within the cloud, except within any scope that can be affirmatively shown.

[0014] Processor set 110 includes one or more computer processors of any type currently known or future-developed. Processing circuitry 120 can be distributed among multiple packages, such as multiple conditioned integrated circuit chips. Processing circuitry 120 can implement multiple processor threads and / or multiple processor cores. Cache 121 is a memory located within a processor chip package and is typically used with respect to data or code that should be available for fast access by a thread or core executing on processor set 110. Cache memory is typically organized into multiple levels depending on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set can be located "off-chip". In some computing environments, processor set 110 can be designed to operate using qubits and execute quantum computing.

[0015] Typically, computer-readable program instructions are loaded onto computer 101 and cause a set of operational steps to be executed by a processor set 110 of computer 101, whereby the instructions so executed implement a computer-implemented method as specified in the flowchart and / or description of the computer-implemented method included herein. These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by processor set 110 to control and direct the execution of the computer-implemented method. In computing environment 100, at least some of the instructions for implementing the computer-implemented method may be stored within block 107 within persistent storage 113.

[0016] Communication fabric 111 is a signal conduction path that enables various components of computer 101 to communicate with each other. Typically, this fabric consists of switches and electrical conductive paths, such as buses, bridges, physical input / output ports, and the like. Other types of signal communication paths, such as optical fiber communication paths and / or wireless communication paths, may be used.

[0017] Volatile memory 112 is any type of volatile memory known currently or developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 112 is characterized by random access, but this is not necessary unless expressly stated. In computer 101, volatile memory 112 is located within a single package and exists within computer 101, but alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0018] The persistent storage 113 is any form of non-volatile storage for a computer, known currently or developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is directly supplied to the computer 101 and / or to the persistent storage 113. The persistent storage 113 can be read-only memory (ROM), but usually at least a portion of the persistent storage enables writing, deleting, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid state storage devices. The operating system 122 can take multiple forms, for example, various known proprietary operating systems, or an open-source Portable Operating System Interface (POSIX)-type operating system that utilizes a kernel. Usually, the code included within block 107 includes at least a portion of the computer code involved in performing the computer implemented methods described herein.

[0019] The peripheral device set 114 includes a set of peripheral devices of the computer 101. The data communication connections between the peripheral devices of the computer 101 and other components can be implemented in various ways, such as a Bluetooth (registered trademark) connection, a Near-Field Communication (NFC) connection, a connection by a cable (e.g., a Universal Serial Bus (USB) type cable), an insertion type connection (e.g., a Secure Digital (SD) card), a connection through a local area communication network, and even a connection through a wide area network, such as the Internet. In various embodiments, the UI device set 123 can include components such as a display screen, a speaker, a microphone, wearable devices (e.g., goggles and smartwatches), a keyboard, a mouse, a printer, a touchpad, a game controller, and a haptic device. The storage 124 is an external storage, such as an external hard drive, or an insertable storage, such as an SD card. The storage 124 can be persistent and / or volatile. In some embodiments, the storage 124 can take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where the computer 101 needs to have a large amount of storage (e.g., the computer 101 locally stores and manages a large database), this storage can be provided by a peripheral storage device designed to store a very large amount of data, such as a storage area network (SAN) shared by a plurality of geographically dispersed computers. The IoT sensor set 125 is composed of sensors that can be used in applications of the Internet of Things. For example, one sensor can be a thermometer, and another sensor can be a motion detector.

[0020] The network module 115 is an aggregate of computer software, hardware, and firmware that enables the computer 101 to communicate with other computers through the WAN 102. The network module 115 may include hardware, such as a modem or Wi-Fi (registered trademark) signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data via the Internet. In some embodiments, the network control function and network transfer function of the network module 115 are executed on the same physical hardware device. In other embodiments (e.g., embodiments that utilize Software-Defined Networking (SDN)), the control function and transfer function of the network module 115 are executed on physically separate devices such that the control function manages multiple different network hardware devices. Typically, computer-readable program instructions for executing a computer implementation method can be downloaded from an external computer or external storage device to the computer 101 through a network adapter card or network interface included within the network module 115.

[0021] The WAN 102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology known currently or developed in the future for communicating computer data. In some embodiments, the WAN 102 may be replaced and / or supplemented by a local area network (LAN), such as a Wi-Fi network, designed to communicate data between devices located in a local area. The WAN and / or LAN typically includes computer hardware, such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0022] The end - user device (EUD) 103 is any computer system that is used and controlled by an end - user (e.g., a customer of an enterprise operating computer 101), and can take any of the forms discussed above in relation to computer 101. The EUD 103 typically receives beneficial and useful data from the operation of computer 101. For example, in a hypothetical case where computer 101 is designed to provide a proposal to an end - user, this proposal will typically be communicated from the network module 115 of computer 101, through the WAN 102, to the EUD 103. In this way, the EUD 103 can display or otherwise present the proposal to the end - user. In some embodiments, the EUD 103 can be a client device, such as a thin - client, a thick - client, a mainframe computer, a desktop computer, etc.

[0023] The remote server 104 is any computer system that provides at least some data and / or functions as a service to computer 101. The remote server 104 can be controlled and used by the same organization that operates computer 101. The remote server 104 represents a machine that collects and stores data that is beneficial and useful for use by other computers, such as computer 101. For example, in a hypothetical case where computer 101 is designed and programmed to provide a proposal based on past data, in that case this past data can be provided from the remote database 130 of the remote server 104 to computer 101.

[0024] The public cloud 105 is any computer system available for use by multiple organizations that provides on-demand availability of computer system resources and / or other computer functions, particularly data storage (cloud storage) and computing power, without direct and active management by the user. Cloud computing typically exploits resource sharing to achieve coherence and economies of scale. The direct and active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud orchestration module 141. Typically, the computing resources provided by the public cloud 105 are implemented by virtual computing environments that run on various computers that make up the host physical machine set 142, which is the universe of physical computers within and / or available to the public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from a virtual machine set 143 and / or containers from a container set 144. It is understood that these VCEs can be stored as images and transferred as images or after instantiation of the VCE among and within various physical machine hosts. The cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of the VCE, and manages the active instantiation of VCE deployments. The gateway 140 is an aggregate of computer software, hardware, and firmware that enables the public cloud 105 to communicate through the WAN 102.

[0025] Some further explanation of a virtualized computing environment (VCE) is provided here. A VCE can be stored as an "image". A new active instance of a VCE can be instantiated from the image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of the operating system where the kernel enables the existence of multiple isolated instances of user space, called containers. Usually, these isolated instances of user space behave as actual computers from the perspective of the programs running within them. A computer program running on a normal operating system can utilize all the resources of the computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of that container and the devices assigned to that container, and this feature is known as containerization.

[0026] The private cloud 106 is similar to the public cloud 105, except that computing resources are available only for use by a single enterprise. The private cloud 106 is shown as being in communication with the WAN 102, but in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple different types of clouds (e.g., types of private clouds, community clouds, or public clouds), and is often implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is coupled by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability among the multiple constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0027] Figure 2 depicts a flowchart of an exemplary method for code comment quality assurance according to some embodiments of the present disclosure. The method of Figure 2 can be executed, for example, by the quality assurance module 107 of Figure 1. In some embodiments, the quality assurance module 107 can be implemented as a separate process or service from the application or software that implements code comment quality assurance. For example, the quality assurance module 107 can be implemented by an operating system or other software that monitors the behavior and execution of the application that implements code comment quality assurance. As another example, in some embodiments, the quality assurance module 107 can be implemented as a process or service that applies updates or patches to an application or code capable of implementing code comment quality assurance, or as a process or service that monitors or detects updates or patches to an application or code capable of code comment quality assurance.

[0028] The method of Figure 2 includes a step 202 of calculating the complexity of a code portion. The step 202 of calculating the complexity of a code portion can be performed in a plurality of ways. In one embodiment, the step of calculating the complexity of a code portion can involve a step of analyzing the code portion and a step of determining one or more complexity metrics. These complexity metrics can represent various aspects of code complexity, such as various code complexities, code readability, code maintainability, or the like. For example, well-established metrics, such as the cyclomatic complexity method, the Halstead method, the maintainability index method, or the like can be used.

[0029] Complexity metrics can represent the number of lines of code, the number of operators or operands, the number of linearly independent or linearly dependent paths through the code logic, branch complexity, data access complexity, data flow complexity, a metric for determining class inheritance (e.g., depth) or the number of child or dependent classes, the cohesion of class methods, or the like. The reader will understand that code complexity can be defined by many different ways and combinations of ways, whereby the different ways or combinations can generate one or more complexity metrics that may be associated with the recorded code portion. Also, these complexity metrics can be used to compare one code portion to another and also to analyze the quality of code comments associated with the code portion.

[0030] In one embodiment, one or more of these methods can provide a measure, display, or quantification of the complexity of a code portion in terms of a magnitude value representing the code complexity. The magnitude value can be expressed in terms of a range, grade, value, ranking, percentile value, or the like. The magnitude value can be, for example, an eigenvector including one or more eigenvalues. For example, quality assurance module 107 may be configured to receive or select a code portion, identify computer code strings within the code portion, and execute one or more code complexity measurement functions that generate a magnitude value indicative of the code complexity for the code portion. The functions may, for example, transform the code portion into a matrix, which can then be decomposed to obtain an eigenvector including one or more eigenvalues, where the eigenvector represents the magnitude of the code complexity for the code portion.

[0031] The method of FIG. 2 also includes step 204 of extracting one or more comments associated with a code portion from the code portion. Step 204 of extracting one or more comments associated with a code portion from the code portion can be performed, for example, by searching for and identifying special characters indicating code comments. Step 204 of extracting one or more comments associated with a code portion from the code portion can be performed by referring to specific sections of a code file, such as a specific number of lines that are a file header or the start or end of a code file.

[0032] Step 204 of extracting one or more comments associated with a code portion from the code portion can be performed by identifying some other feature. For example, in a code portion, code comments can have a different color or font, or some other characteristic formatting feature. Code comments can have a different indentation or other positioning feature compared to computer code. Also, although the above examples show that code comments are extracted from the same file or object that also stores computer code, in other examples, code comments can be in a separate file or object or data store compared to computer code. Thus, the step of extracting one or more comments associated with a code portion can also include the step of obtaining code comments from a separate file or file section.

[0033] The method of FIG. 2 also includes a step 206 of converting one or more comments into a set of text features. The step 206 of converting one or more comments into a set of text features may include a step of decomposing a comment string into one or more sub-components or constituents. For example, the quality assurance module 107 may be configured to decompose a comment string by parts of speech, such as nouns, verbs, adjectives, or the like. Other text features, such as word count, punctuation, spacing, formatting, or the like may be identified. Also, the number of separate comments regarding a particular code portion may be determined. Further, other characteristic aspects of the code indicated by the comments, such as code version identifiers, code author identifiers, and any other code portion metadata (e.g., whether the code portion is part of a particular software module) may be identified.

[0034] The method of FIG. 2 also includes step 208 of quantifying a set of text features. Step 208 of quantifying a set of text features may include determining a count or some other statistical measure of one or more of the subcomponents, for example, by type, such as the number of nouns in the comment, the number of verbs in the comment, etc. The language features of the code can also be extracted and quantified using natural language processing (NLP) techniques, as further described below with respect to FIG. 4. As an example of such extraction, the extraction can be performed using regular expressions. Also, natural language processing techniques can be utilized to analyze the content of code comments. Language features, such as parts of speech (nouns, verbs, adjectives), grammatical aspects (phrases, clauses, sentence structure), and other features (punctuation, indentation, special characters, references to other comments) can be extracted and analyzed. These features can be quantified in various ways (such as by count, average, or other statistical measures). The quantification of these features can be converted into specific comment quantification values. For example, the numerical values of specific features can be squared, added together, and then the square root thereof can be determined, and one or more comment quantification values regarding the above comment can be derived.

[0035] The method of FIG. 2 also includes step 212 of determining the content consistency between the one or more comments and the associated code portions for the one or more comments using the quantification of the set of text features and the complexity of the code portion. The step of determining the content consistency between the one or more comments and the associated code portions for the one or more comments can be performed by an integrated analysis of the magnitude value created for the code portion and the comment quantification value for the code comment. As an example of the integrated analysis mentioned above, the quality assurance module 107 can determine, for example, that the code portion refers to a specific number of items (e.g., data objects), such as seven items. Next, the quality assurance module can determine whether the associated code comment includes references to the seven items (e.g., the same seven items). For example, the code comment can include seven descriptors, such as nouns each referring to an item that is part of the code portion.

[0036] The integrated analysis of the magnitude value created for the code portion and the comment quantification value for the code comment can also be performed in other ways. For example, the magnitude value created for the code portion and the comment quantification value for the code comment can be regarded as XY coordinates and plotted as such on a graph (e.g., the code portion magnitude value is indicated by a value on the X-axis and the comment quantification value for the code comment is indicated by a value on the Y-axis). Each of the XY points can represent a code / comment pair that correlates. In some embodiments, a reference line such as Y = X, which indicates 100% content consistency or correlation between the magnitude value created for the code portion and the comment quantification value for the code comment, can be included on the graph. The reader will understand that the closer the points are to Y = X, the stronger the correlation between the code portion and the comment.

[0037] Also, the magnitude values created for the code portion and the comment quantification values for the code comments can be plotted as unprocessed values as in the above example, or various operations can be applied to one or both of the sets of values to facilitate analysis. For example, one or both of the sets of values can be weighted, normalized, or processed using other statistical functions or operations (e.g., taking the floor function value of the value or taking the absolute value). Next, the processed values resulting from the application of one or more of these operations can be plotted using the graphs described above.

[0038] In some embodiments, the integrated analysis can result in a determination of the quality of the comments in the code / comment pairs. For example, using the graph creation method mentioned above, the quality assurance module 107 can determine the proximity of a point (corresponding to a code / comment pair) to the Y = X line representing 100% content consistency between the code portion and its corresponding comment. Based on the determined proximity or other metrics, the quality assurance module 107 can determine that the comment has a particular content consistency, relevance, or correspondence relationship to the code portion. In one embodiment, the code comment can be assigned an objective quality scale, for example, on a scale from 1 to 10.

[0039] The method of FIG. 2 also includes step 214 of triggering a notification in response to a determination that the content of one or more comments and the associated code portions is inconsistent. Step 214 of triggering a notification in response to a determination that the content of one or more comments and the associated code portions is inconsistent can be implemented by the quality assurance module 107 providing a message to a user (or computer module) indicating the content inconsistency and / or the degree of content inconsistency between the code portion and the code comment. In another example, the notification can take the form that the code / comment pair is visually emphasized (e.g., on a graph plotting the code / comment pairs as coordinates), or can take the form of an alert or warning in a comment review dashboard, or the like.

[0040] FIG. 3 depicts a flowchart of another exemplary method for code comment quality assurance according to some embodiments of the present disclosure. The method of FIG. 3 is similar to FIG. 2 in that the method of FIG. 3 includes step 202 of calculating the complexity of a code portion, step 204 of extracting one or more comments associated with the code portion from the code portion, step 206 of converting the one or more comments into a set of text features, step 208 of quantifying the set of text features, step 212 of determining the content consistency between the one or more comments and the associated code portions using the quantification of the set of text features and the complexity of the code portion, and step 214 of triggering a notification in response to a determination that the content of the one or more comments and the associated code portions is inconsistent.

[0041] The method of FIG. 3 differs from the method of FIG. 2 in that the method of FIG. 3 also includes step 302 of parsing the code into a plurality of code portions. The step of parsing the code into a plurality of code portions may be performed, for example, by a parser module that is part of or associated with the quality assurance module 107. The parser may disassemble the code file into one or more code portions. Each code file may be analyzed lexically to convert the code string into one or more pieces that may be referred to as tokens. Lexical analysis may, for example, determine certain delimiters within the code file at which point a certain amount of the code string may be separated from other code strings to form multiple portions. For example, certain characters may be used as delimiters (e.g., the opening bracket “{” or the closing bracket “}” characters).

[0042] FIG. 4 depicts a flowchart of another exemplary method for code comment quality assurance according to some embodiments of the present disclosure. The method of FIG. 4 is similar to FIG. 2 in that the method of FIG. 4 includes step 202 of calculating the complexity of a code portion, step 204 of extracting one or more comments associated with the code portion from the code portion, step 206 of converting the one or more comments into a set of text features, step 208 of quantifying the set of text features, step 212 of determining the content consistency between the one or more comments and the associated code portion using the quantification of the set of text features and the complexity of the code portion, and step 214 of triggering a notification in response to a determination that the content of the one or more comments and the associated code portion is inconsistent.

[0043] The method of FIG. 4 is different from the method of FIG. 2 in that the method of FIG. 4 also includes step 402 of analyzing the code portion using the Halstead complexity metric. The Halstead complexity metric may represent, in some examples, a code complexity metric that represents an algorithm implemented within the code independent of code execution. Step 402 of analyzing the code portion using the Halstead complexity metric may include the step of statically calculating the Halstead complexity metric from the code portion by the quality assurance module 107.

[0044] In some embodiments, the step of calculating the Halstead complexity metric may include the step of determining one or more quantities, such as the number of distinct operators within the code portion, the number of distinct operands within the code portion, and the like. Using these determined quantities, the quality assurance module 107 may be configured to calculate one or more Halstead complexity metrics, such as the length of the program. In this scenario, the length of the program may refer not to the number of lines of code, but rather, for example, to the sum of the total number of operators and the total number of operands. Other Halstead complexity metrics may include products or ratios or combinations thereof that represent a measure of difficulty, such as the degree of difficulty in describing or understanding the program using the number of distinct operators or operands. Still other Halstead complexity metrics may include the time or effort required to describe or understand the code, the number of possible bugs within the code, and the like. In some embodiments, the quality assurance module 107 may be configured to generate one or more of the Halstead complexity metrics mentioned above or other Halstead complexity metrics for the code portion. The resulting Halstead complexity metric may be used as the code portion magnitude value mentioned above.

[0045] The method of FIG. 4 also includes step 404 of analyzing a code portion using a cyclomatic complexity metric. Step 404 of analyzing a code portion using a cyclomatic complexity metric can be performed by quality assurance module 107 by determining a control flow graph of the code portion. The control flow graph can refer to a graph data structure where the nodes of the graph correspond to indivisible groups of commands of the code portion. The graph can include directed edges between the nodes. A directed edge leading from a first node to a second node can be found, for example, where the command of the second node is executed immediately after the command of the first node. Also, quality assurance module 107 can be configured to identify the number of linearly independent paths through the nodes of the code portion, which can represent the cyclomatic complexity of the program. This number can be used as a magnitude value representing the code complexity regarding the code portion.

[0046] The method of FIG. 4 also includes step 406 of calculating a maintainability index value regarding the code portion. Step 406 of calculating a maintainability index value regarding the code portion can include steps where quality assurance module 107 determines various metrics of the code portion, for example, the lines of code within the code portion. Such metrics can indicate a level of complexity, for example, in terms of the amount of time or effort required to maintain a particular code portion (e.g., from the perspective of man-hours).

[0047] The method of FIG. 4 also includes step 408 of processing one or more comments using natural language processing (NLP) techniques. Step 408 of processing one or more comments using natural language processing (NLP) techniques can be performed by quality assurance module 107 to determine the meaning or value of the linguistic features of the code comments. For example, NLP techniques such as morphological analysis, syntactic analysis, dictionary-based semantic analysis, relational semantics, discourse analysis, and others can be used. Also, techniques such as part-of-speech tagging can be used to markup or categorize specific words within the comments.

[0048] FIG. 5 depicts a flowchart of another exemplary method for code comment quality assurance according to some embodiments of the present disclosure. The method of FIG. 5 is similar to FIG. 2 in that the method of FIG. 5 includes a step 202 of calculating the complexity of a code portion, a step 204 of extracting one or more comments associated with the code portion from the code portion, a step 206 of converting the one or more comments into a set of text features, a step 208 of quantifying the set of text features, a step 212 of determining the content consistency between the one or more comments and the associated code portion using the quantification of the set of text features and the complexity of the code portion, and a step 214 of triggering a notification in response to a determination that the content of the one or more comments and the associated code portion is inconsistent.

[0049] The method of FIG. 5 differs from the method of FIG. 2 in that the method of FIG. 5 also includes a step 502 of generating a training data set of code portion and comment pairs, where each code portion and comment pair is associated with a degree of content consistency. In one embodiment, the training data set can be created using code / comment pairs, such as those described above with respect to FIG. 2. As described above, the code / comment pairs may be assigned a value from 1 to 10 on a comment quality scale. In some embodiments, the scale can be determined, for example, by a relative magnitude ratio or other comparison measure between a code complexity magnitude value (e.g., a Halstead metric indicating code complexity) and a comment feature magnitude value.

[0050] The method of FIG. 5 also includes step 504 of training a machine learning model using a training data set to predict the content consistency between a code portion and one or more comments associated with the code portion based on the quantification of a set of text features extracted from the one or more comments and the complexity of the code portion associated with the one or more comments. In that case, the training data set can be an input to a machine learning model, for example, a linear regression model. The above model can provide an output, for example, a rank for a specific comment, based on the comment quality, thereby providing an automated data-driven approach to comment quality assessment. Also, the process can be iterative. More specifically, the execution of the machine learning model can be repeatedly considered and refined to re-weight the parameters that created a comparison scale (e.g., magnitude ratio) between the code complexity magnitude value and the comment feature magnitude value.

[0051] Next, considering these comment quality scores, the output of the machine learning model trained on the above code / comment pair can be used to determine the comment quality of other comments. For example, a code / comment pair that is not part of the training data set can be used as an input to the machine learning model, and the model can then output a rank for the comment of the code / comment pair that represents the comment quality.

[0052] The reader will understand that an artificial intelligence (AI) system, particularly one assisted by a large language model (LLM), can be used to generate code and also generate code comments, and that these AI systems can benefit from the systems and methods described above. In an exemplary implementation, the AI system can generate a code comment, and then the code comment can be input into the machine learning model mentioned above to obtain an indicator (e.g., a rank) of the quality of the comment. The quality metric can be input into the AI system as a way to improve the performance of the AI system. For example, if the comment is ranked on a scale of 1 to 10, the rank of the comment can be input into the AI system using a specific prompt that encourages the AI system to generate a comment with a particular improvement compared to, for example, what the AI system generated initially. For example, the AI system can be prompted to generate a more detailed comment.

[0053] Also, when the AI system is generating code based on other code, the above systems and methods can provide additional improvements. As one example, the AI system can receive COBOL code as input and translate or convert it into Java code. During the process of generating Java code, the AI system can also generate code comments. One or more of the generated code comments can be analyzed, for example, using the machine learning model mentioned above to determine the quality of the comment. Next, the AI system can be prompted to generate an improved comment as part of its Java code generation process. In some cases, since comments may not be generated, the AI system can be prompted to generate a code comment for a code portion when no comment was generated during Java code generation.

[0054] In certain other cases where new code is generated from existing code, it may be observed that the quality of comments in the existing code can affect the quality of the newly generated code. For example, an AI system can be configured to take as input existing code, such as existing COBOL code, and also take as input the code comments within the existing COBOL code. In such cases, a reader will understand that improved comments within the existing COBOL code can lead to improvements in the generated Java code. Also, code comments may be absent for certain code portions within the COBOL code. Thus, the AI system may be prompted to generate comments regarding the above COBOL code. For example, if comments already exist, the AI system can be prompted to generate comments regarding code portions without code comments, or more detailed code comments to replace existing comments, or additional comments that are added to existing code comments regarding code portions. Next, the generated code comments can be used by the AI system within its code generation process.

[0055] Various aspects of the present disclosure are illustrated by descriptions, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). For any flowchart, depending on the relevant technology, operations may be executed in an order different from that shown in a given flowchart. For example, again depending on the relevant technology, two operations shown in consecutive flowchart blocks can be executed in reverse order, as a single integrated step, simultaneously, or at least partially overlapping in time.

[0056] An embodiment of a computer program product (a "CPP embodiment" or "CPP") is, in the present disclosure, a term used to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage devices, which collectively include machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. By way of non-limiting example, a computer-readable storage medium can be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media are floppy disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices (such as punch cards or pits / lands formed on the major surfaces of a disk), or any suitable combination of the foregoing. A computer-readable storage medium, as the term is used in the present disclosure, shall not be construed to include storage in the form of a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, optical pulses passing through an optical fiber cable, electrical signals transmitted through a wire, and / or other transmission media. As will be understood by those skilled in the art, data is typically moved during normal operation of a storage device, for example, at some irregular points in time during access, defragmentation, or garbage collection, but the data is not transient while it is stored, and thus, the storage device is not considered to be transient for the foregoing reason.

[0057] The description of various embodiments of the present disclosure is presented for illustrative purposes and is not intended to be exhaustive or limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein are chosen to best explain the principles of the embodiments, the practical application, or the technical improvements made to the technologies found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for code comment quality assurance, comprising: Calculating the complexity of a code portion; Extracting one or more comments associated with the code portion from the code portion; Converting the one or more comments into a set of text features; Quantifying the set of text features; Determining the content consistency between the one or more comments and the associated code portion using the quantification of the set of text features and the complexity of the code portion; and Triggering a notification in response to a determination that the content of the one or more comments and the associated code portion is inconsistent A method comprising the steps of:

2. The method according to claim 1, further comprising parsing the code into a plurality of code portions.

3. The method according to claim 1, wherein the step of calculating the complexity of the code portion further comprises analyzing the code portion using a Halstead complexity measure.

4. The method according to claim 1, wherein the step of calculating the complexity of the code portion further comprises analyzing the code portion using a cyclomatic complexity measure.

5. The method according to claim 1, wherein the step of calculating the complexity of the code portion further comprises calculating a maintainability index value for the code portion.

6. The method according to claim 1, wherein the step of extracting the one or more comments associated with the code portion from the code portion further comprises processing the one or more comments using natural language processing (NLP).

7. The method according to any one of claims 1 to 6, wherein the one or more code comments include comments generated by an artificial intelligence system.

8. The method according to claim 7, wherein the code portion includes a code portion generated by the artificial intelligence system.

9. The method according to any one of claims 1 to 6, further comprising generating a training dataset of code portion and comment pairs, wherein each code portion and comment pair is associated with a degree of content consistency.

10. Training the machine learning model using the training data set to predict the content consistency between the code portion and one or more comments associated with the code portion based on the quantification of a set of text features extracted from the one or more comments and the complexity of the code portion associated with the one or more comments, the method according to claim 9, further comprising the steps of.

11. A processing device; and A memory operably coupled to the processing device, wherein the memory, when executed, causes the processing device to: Calculate the complexity of the code portion; Extract one or more comments associated with the code portion from the code portion; Convert the one or more comments into a set of text features; Quantify the set of text features; Using the quantification of the set of text features and the complexity of the code portion to determine the content consistency between the one or more comments and the one or more comments and the associated code portion; Trigger a notification in response to a determination that the content of the one or more comments and the associated code portion is inconsistent Store instructions of a computer program An apparatus comprising.

12. The memory further stores computer program instructions for parsing the code into a plurality of code portions, the apparatus according to claim 11.

13. The instructions for calculating the complexity of the code portion further include instructions for analyzing the code portion using a Halstead complexity metric, the apparatus according to claim 11 or 12.

14. The instructions for calculating the complexity of the code portion further include instructions for analyzing the code portion using a cyclomatic complexity metric, the apparatus according to claim 11 or 12.

15. The instructions for calculating the complexity of the code portion further include instructions for calculating a maintainability index value for the code portion, the apparatus according to claim 11 or 12.

16. A computer program, which when executed by a computer, causes the computer to: Calculate the complexity of the code portion; Extract one or more comments associated with the code portion from the code portion; cause the one or more comments to be converted into a set of text feature quantities; quantify the set of text feature quantities; determine the content consistency between the one or more comments and the associated code portions with respect to the one or more comments using the quantification of the set of text feature quantities and the complexity of the code portion; trigger a notification in response to a determination that the content of the one or more comments and the associated code portions is inconsistent A computer program comprising instructions of a computer program.

17. The computer program according to claim 16, further comprising computer program instructions for causing the computer to parse the code into a plurality of code portions.

18. The instructions for causing the computer to calculate the complexity of a code portion further comprise instructions for causing the computer to analyze the code portion using a Halstead complexity metric, according to the computer program of claim 16 or 17.

19. The instructions for causing the computer to calculate the complexity of a code portion further comprise instructions for causing the computer to analyze the code portion using a cyclomatic complexity metric, according to the computer program of claim 16 or 17.

20. The instructions for causing the computer to calculate the complexity of a code portion further comprise instructions for causing the computer to calculate an ease-of-maintenance indicator value for the code portion, according to the computer program of claim 16 or 17.

Images