Code annotation quality assurance

Abstract

Code annotation quality assurance includes calculating complexity of a code portion, extracting one or more annotations associated with the code portion from the code portion, converting the one or more annotations into a set of textual features, quantifying the set of textual features, and outputting the quantified set of textual features. An alignment between the one or more annotations and the associated code portion is determined for the one or more annotations using quantification of the set of textual features and a complexity of the code portion, and a notification is triggered in response to determining that the one or more annotations are not aligned with the associated code portion.

Description

Background Art

[0001] The present disclosure relates to methods, apparatuses, and products for code comment quality assurance.

[0002] In complex software systems, there may be multiple code files, and each code file contains multiple code comments. As is known to those skilled in the art, code comments can be intended to explain, represent, or highlight one or more aspects of computer code. The author of a code comment (e.g., the author of the computer code) may expect the code comment to provide useful background or explanatory information to the reader of the computer code (e.g., another computer programmer). Thus, code comments can take the form of an explanation, example, or illustration of the functionality embodied by the computer code. Although computer code can be a computer programming language (such as COBOL or Java), code comments can be in a natural language (e.g., plain English). The reader will understand that although code comments are designed to represent or explain the code, code comments may have different qualities, and for some code, there may even be a lack of code comments where the reader could benefit from the presence of the comments. In other cases, code comments may not fully represent or explain one or more aspects of the associated computer code, thus limiting the usefulness of the code comments. Additionally, modern artificial intelligence systems can also generate code comments, but existing systems may not be able to generate code comments that consistently provide a satisfactory explanation of the purpose and functionality of code segments. Summary of the Invention

[0003] According to embodiments of the present disclosure, various methods, apparatuses, and products for code comment quality assurance are described herein. In some aspects, code comment quality assurance includes calculating the complexity of a code portion, extracting one or more comments associated with the code portion from the code portion, converting the one or more comments into a set of text features, quantifying the set of text features, for the one or more comments, using the quantification of the set of text features and the complexity of the code portion to determine the alignment between the one or more comments and the associated code portion, and triggering a notification in response to determining that the one or more comments are not aligned with the associated code portion. Brief Description of the Drawings

[0004] Figure 1 A block diagram illustrating an example computing environment for code comment quality assurance in accordance with some embodiments of the present disclosure.

[0005] Figure 2 A flowchart illustrating an example method for code comment quality assurance in accordance with some embodiments of the present disclosure.

[0006] Figure 3 A flowchart illustrating another example method for code comment quality assurance in accordance with some embodiments of the present disclosure.

[0007] Figure 4 A flowchart illustrating another example method for code comment quality assurance according to some embodiments of the present disclosure.

[0008] Figure 5 A flowchart illustrating another example method for code comment quality assurance according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0009] The following disclosure describes exemplary embodiments for code comment quality assurance. In some embodiments, the described system analyzes the correlation between human language and computer code. As used herein, human language or natural language may be exemplified by any spoken, written, or expressive language (e.g., English, Japanese, American Sign Language, etc.). Examples of computer code include any computer-readable code written in one or more programming languages, such as computer code written in Java, C++, COBOL, etc. Analyzing the correlation between human language and computer code may include analyzing the content of a portion of the computer code and the content of any associated code comments written in natural language.

[0010] Now referring to Figure 1 , an example computing environment is shown in accordance with aspects of the present disclosure. Computing environment 100 includes an example of an environment for executing at least some of the computer code involved in performing the various methods described herein, such as quality assurance module 107. Quality assurance module 107 may be configured to determine aspects of the computer code and associated code comments, such as code complexity level or code comment characteristics, and identify the correlation between the computer code and the code comments. Based on the identified correlation or degree of correlation, quality assurance module 107 may predict or determine the alignment level between the computer code and the code comments. If the code comments are predicted to be misaligned with respect to the computer code, quality assurance module 107 may trigger an alert or notification prompting, for example, a human user or an artificial intelligence system to generate code comments that are more closely aligned with the computer code.

[0011] In addition to block 107, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end-user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), communication fabric 111, volatile memory 112, permanent storage device 113 (including operating system 122 and block 107, as described above), a set of peripheral devices 114 (including a set of user interface (UI) devices 123, storage device 124, and Internet of Things (IoT) sensor set 125), and network module 115. Remote server 104 includes remote database 130. Public cloud 105 includes gateway 140, cloud coordination module 141, set of host physical machines 142, set of virtual machines 143, and set of containers 144.

[0012] Computer 101 may take the form of a desktop computer, laptop computer, tablet computer, smart phone, smart watch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or to be developed in the future that is capable of running programs, accessing a network, or querying a database such as remote database 130. As is well known in the computer art and depending on the technology, the performance of computer-implemented methods may be distributed among multiple computers and / or among multiple locations. On the other hand, in this presentation of computing environment 100, the discussion focuses on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 may be located in the cloud, even though Figure 1 it is not shown in the cloud, on the other hand, computer 101 does not need to be in the cloud, except to whatever extent may be affirmatively indicated.

[0013] The set of processors 110 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed across multiple packages, such as multiple cooperating integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be made available for rapid access by threads or cores running on the set of processors 110. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the set of processors may be located "off-chip". In some computing environments, the set of processors 110 may be designed to work with qubits and perform quantum computing.

[0014] Computer-readable program instructions are typically loaded onto computer 101 so that a set of processors 110 of computer 101 execute a series of operational steps to implement a computer-implemented method such that the instructions so executed will instantiate the method specified in the flowchart and / or narrative description of the computer-implemented method included in this document. These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the set of processors 110 to control and direct the execution of the computer-implemented method. In computing environment 100, at least some of the instructions for executing the computer-implemented method may be stored in block 107 of persistent storage 113.

[0015] Communication structure 111 is a signal conduction path that allows the various components of computer 101 to communicate with each other. Typically, this structure consists of switches and conductive paths, such as switches and conductive paths that make up a bus, a bridge, a physical input / output port, etc. Other types of signal communication paths can be used, such as fiber optic communication paths and / or wireless communication paths.

[0016] Volatile memory 112 is any type of volatile memory known now or developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 112 is characterized by random access, but this is not required unless stated affirmatively. In computer 101, volatile memory 112 is located in a single package and inside computer 101, however, alternatively or additionally, volatile memory can be distributed in multiple packages and / or be located external to computer 101.

[0017] Permanent memory 113 is any form of non-volatile memory for a computer known now or developed in the future. The non-volatility of this memory means that the stored data is retained regardless of whether power is supplied to computer 101 and / or directly to permanent memory 113. Permanent memory 113 can be read-only memory (ROM), but typically at least a portion of the permanent memory allows for the writing, deletion, and re-writing of data. Some common forms of persistent storage include disk and solid-state storage devices. Operating system 122 can take several forms, such as various known proprietary operating systems or open-source portable operating system interface type operating systems that employ a kernel. The code included in block 107 typically includes at least some of the computer code involved in executing the computer-implemented method described herein.

[0018] The peripheral device set 114 includes the peripheral device set of the computer 101. Data communication connections between the peripheral devices and other components of the computer 101 can be implemented in various ways, such as Bluetooth connections, near field communication (NFC) connections, connections made by cables (such as Universal Serial Bus (USB) type cables), plug-in connections (e.g., Secure Digital (SD) cards), connections made through local communication networks, and even connections made through wide area networks such as the Internet. In various embodiments, the UI device set 123 can include components such as display screens, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and tactile devices. The storage device 124 is an external storage device, such as an external hard disk drive, or a plug-in storage device, such as an SD card. The storage device 124 can be permanent and / or volatile. In some embodiments, the storage device 124 can take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where the computer 101 needs to have a large amount of storage (e.g., in the case where the computer 101 locally stores and manages a large database), the storage can be provided by a peripheral storage device (such as a storage area network (SAN) shared by multiple geographically distributed computers) designed to store a very large amount of data. The IoT sensor set 125 consists of sensors that can be used in Internet of Things applications. For example, one sensor can be a thermometer, and another sensor can be a motion detector.

[0019] The network module 115 is a collection of computer software, hardware, and firmware that allows the computer 101 to communicate with other computers via the WAN 102. The network module 115 can include hardware such as a modem or a Wi-Fi signal transceiver, software for packetizing and / or depacketizing data transmitted over a communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control function and the network forwarding function of the network module 115 are executed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control function and the forwarding function of the network module 115 are executed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing computer-implemented methods can generally be downloaded to the computer 101 from an external computer or an external storage device through a network adapter card or a network interface included in the network module 115.

[0020] WAN 102 is any wide area network (e.g., the Internet) that is capable of transmitting computer data over non-local distances via any technology, now known or later developed, for transmitting computer data. In some embodiments, WAN 102 may be replaced and / or supplemented by a local area network (LAN) that is designed to transmit data between devices located in a local area (e.g., a Wi-Fi network). A WAN and / or a LAN typically includes computer hardware such as copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0021] The end user device (EUD) 103 is any computer system that is used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and may take any form discussed above in connection with computer 101. The EUD 103 typically receives useful and beneficial data from the operation of computer 101. For example, in the hypothetical case where computer 101 is designed to provide recommendations to an end user, the recommendation will typically be transmitted from the network module 115 of computer 101 to the EUD 103 via WAN 102. In this way, the EUD 103 can display or otherwise present the recommendation to the end user. In some embodiments, the EUD 103 may be a client device such as a thin client, a thick client, a mainframe computer, a desktop computer, etc.

[0022] The remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. The remote server 104 may be controlled and used by the same entity that operates computer 101. The remote server 104 represents a machine that collects and stores useful and beneficial data used by other computers such as computer 101. For example, in the hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, the historical data may be provided to computer 101 from the remote database 130 of the remote server 104.

[0023] A public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages the sharing of resources to achieve consistency and economy of scale. The direct and active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers of a set of host physical machines 142, which is the universe of physical computers in and / or available for the public cloud 105. The virtual computing environment (VCE) typically takes the form of virtual machines from a set of virtual machines 143 and / or containers from a set of containers 144. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts either as images or after instantiation of the VCE. The cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of the VCE, and manages the active instantiations of the VCE deployment. The gateway 140 is a collection of computer software, hardware, and firmware that allows the public cloud 105 to communicate via the WAN 102.

[0024] Some further explanations of the virtualized computing environment (VCE) will now be provided. A VCE can be stored as an "image". New active instances of the VCE can be instantiated from this image. Two common types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user space instances, called containers. From the perspective of the programs running within them, these isolated user space instances typically appear as actual computers. A computer program running on a normal operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU capabilities, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of the container and the devices allocated to the container, which is a feature known as containerization.

[0025] The private cloud 106 is similar to the public cloud 105, except that the computing resources are only available for use by a single enterprise. Although the private cloud 106 is depicted as communicating with the WAN 102, in other embodiments, the private cloud can be completely disconnected from the Internet and only accessible through a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types) that are typically implemented by different vendors. Each of the multiple clouds remains an independent and discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable coordination, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, the public cloud 105 and the private cloud 106 are both part of the larger hybrid cloud.

[0026] Figure 2 A flowchart illustrating an example method for code comment quality assurance in accordance with some embodiments of the present disclosure is presented. Figure 2 The method can be performed, for example, by Figure 1 the quality assurance module 107. In some embodiments, the quality assurance module 107 can be implemented as a process or service separate from the application or software that implements code comment quality assurance. For example, the quality assurance module 107 can be implemented by the operating system or other software that monitors the behavior and execution of the application that implements code comment quality assurance. As another example, in some embodiments, the quality assurance module 107 can be implemented as a process or service that updates or patches an application or code application that can implement code comment quality assurance, or as a process or service that monitors or detects updates or patches to an application or code that can implement code comment quality assurance.

[0027] Figure 2 The method includes computing 202 the complexity of a code section. Computing 202 the complexity of a code section can be performed in several ways. In one embodiment, computing the complexity of a code section can include analyzing the code section and determining one or more metrics of complexity. These complexity metrics can represent various aspects of code complexity, such as various code complexities, code readability, code maintainability, etc. For example, well-established metrics such as the cyclomatic complexity method, the Halstead method, the maintainability index method, etc. can be used.

[0028] Complexity metrics can represent the number of lines of code, the number of operators or operands, the number of linearly independent or dependent paths through the code logic, branch complexity, data access complexity, data flow complexity, a metric that determines class inheritance (e.g., depth) or the number of subclasses or dependent classes, the cohesion of class methods, etc. The reader will understand that code complexity can be defined by many different methods and combinations of methods, whereby different methods or combinations will produce one or more complexity metrics, which can be recorded and associated with a code portion. Additionally, these complexity metrics can be used to compare one code portion to another and also to analyze the quality of code comments associated with the code portion.

[0029] In one embodiment, one or more of these methods can provide a measure, representation, or quantification of the complexity of a code portion in terms of a magnitude value representing code complexity. The magnitude value can be represented by a range, rank, value, ranking, percentage value, etc. The magnitude value can be, for example, a feature vector including one or more eigenvalue values. For example, the quality assurance module 107 can be configured to receive or select a code portion, identify computer code text within the code portion, and perform one or more code complexity measurement functions that produce a magnitude value indicative of the code complexity of the code portion. For example, these functions can transform the code portion into a matrix, which can then be decomposed to obtain a feature vector including one or more eigenvalue values, where the feature vector represents the magnitude of the code complexity of the code portion.

[0030] Figure 2 The method further includes extracting 204 one or more annotations associated with the code portion from the code portion. Extracting 204 one or more annotations associated with the code portion from the code portion can be performed, for example, by searching for and identifying special characters that represent code comments. Extracting 204 one or more annotations associated with the code portion from the code portion can be performed by looking at specific portions of the code file, such as the file header or a specific number of lines at the beginning or end of the code file, etc.

[0031] Extracting 204 one or more annotations associated with the code portion from the code portion can be performed by identifying some other feature. For example, within the code portion, the code comments can be a different color or font, or have some other distinguishable formatting feature. The code comments can have a different indentation or other placement feature compared to the computer code. Additionally, although the above examples indicate that the code comments are also extracted from the same file or object storing the computer code, in other examples, the code comments can be in a separate file or object or data store compared to the computer code. Thus, extracting one or more annotations associated with the code portion can also include obtaining the code comments from a separate file or portion of a file.

[0032] Figure 2 The method further includes converting one or more annotations 206 into a set of text features. Converting one or more annotations into a set of text features 206 may include breaking down the annotation text into one or more sub-components or constituent parts. For example, the quality assurance module 107 may be configured to break down the annotation text by parts of speech, such as by nouns, verbs, adjectives, etc. Other text features may be identified, such as word count, punctuation, spacing, formatting, etc. Additionally, multiple different annotations for a particular code section may be determined. Further, other differentiating aspects of the code indicated by the annotations may be identified, such as code version identifiers, code author identifiers, and any other code section metadata (such as whether the code section is part of a particular software module).

[0033] Figure 2 The method further includes quantifying 208 the set of text features. Quantifying 208 the set of text features may include, for example, determining a count of one or more sub-parts by type or some other statistical metric, such as the number of nouns in an annotation, the number of verbs in an annotation, etc. Linguistic features of the code may also be extracted and quantified using natural language processing (NLP) techniques, as will be further described with reference to Figure 4 As an example of such extraction, regular expressions may be used to perform the extraction. Additionally, natural language processing techniques may be employed to parse the content of the code annotations. Linguistic features may be extracted and analyzed, such as parts of speech (nouns, verbs, adjectives), grammatical aspects (phrases, clauses, sentence structure), and other features (punctuation, indentation, special characters, references to other annotations). These features may be quantified in various ways (e.g., as a count, an average, or some other statistical measurement). The quantification of these features may be converted into a specific annotation quantification value. For example, the quantity of a certain feature may be squared, summed, and then square-rooted to derive one or more annotation quantification values for the annotation.

[0034] Figure 2The method also includes, for one or more annotations, using the quantification of a set of text features and the complexity of the code portion to determine an alignment between the one or more annotations and the associated code portion. For one or more annotations, determining the alignment between the one or more annotations and the associated code portion can be performed by an integrated analysis of the magnitude values created for the code portion and the annotation quantification values of the code annotations. As an example of the above integrated analysis, the quality assurance module 107 can determine, for example, that the code portion references a specific number of items (e.g., data objects), such as 7 items. The quality assurance module can then determine whether the associated code annotation includes a reference to the 7 items (e.g., the same 7 items). For example, the code annotation can include 7 descriptors such as nouns, each descriptor referencing an item that is part of the code portion.

[0035] The integrated analysis of the magnitude values created for the code portion and the annotation quantification values of the code annotations can also be performed in other ways. For example, the magnitude values created for the code portion and the annotation quantification values of the code annotations can be considered as X-Y coordinates and plotted on a graph as such (e.g., the code portion magnitude values represented by the values on the X-axis and the annotation quantification values of the code annotations represented by the values on the Y-axis). Each of the X-Y points can represent a related code-annotation pair. In some embodiments, a reference line can be included on the graph, such as Y = X, representing 100% alignment or correlation between the magnitude values created for the code portion and the annotation quantification values of the code annotations. The reader will understand that the closer the points are to Y = X, the greater the correlation between the code portion and the annotation can be said to be.

[0036] In addition, as in the above example, the magnitude values created for the code portion and the annotation quantification values of the code annotations can be plotted as raw values, or different operations can be applied to one or both sets of values to facilitate the analysis. For example, one or both of these sets of values can be weighted, normalized, or processed using other statistical functions or operations (e.g., taking the floor value of the values, or taking the absolute value). Then, the processed values resulting from applying one or more of these operations can be plotted using the above graph.

[0037] In some embodiments, the integrated analysis can result in the determination of the quality of the annotation in the code-annotation pair. For example, using the above plotting method, the quality assurance module 107 can determine the proximity of the points (corresponding to the code-annotation pairs) to the Y = X line, which represents 100% alignment between the code portion and its corresponding annotation. Based on the determined proximity, or via other measures, the quality assurance module 107 can determine that the annotation has a specific alignment, correlation, or correspondence with the code portion. In one embodiment, the code annotation can be assigned an objective quality metric, such as on a scale of 1 to 10.

[0038] Figure 2 The method further includes triggering a 212 notification in response to determining that one or more annotations and the associated code portions are misaligned. Triggering the 212 notification in response to determining that one or more annotations and the associated code portions are misaligned can be performed by the quality assurance module 107 providing a message to the user (or computer module) indicating the misalignment and / or degree of misalignment between the code portion and the code annotation. In another example, the notification can take the form of the code-annotation pairs being visually highlighted (e.g., on a graph where the code-annotation pairs are plotted as coordinates), or can take the form of a warning or alarm in an annotation review dashboard, etc.

[0039] Figure 3 A flowchart illustrating another example method for code annotation quality assurance in accordance with some embodiments of the present disclosure is presented. Figure 3 The method of Figure 2 is similar to Figure 3 in that the method of

[0040] Figure 3 includes calculating 202 the complexity of a code portion, extracting 204 one or more annotations associated with the code portion from the code portion, converting 206 the one or more annotations into a set of text features, quantifying 208 the set of text features, determining 212, for the one or more annotations, the alignment between the one or more annotations and the associated code portion using the quantification of the set of text features and the complexity of the code portion, and triggering 214 a notification in response to determining that the one or more annotations and the associated code portion are misaligned. Figure 2 The method of Figure 3 differs from the method of

[0041] Figure 4 in that the method of Figure 4 also includes parsing 302 the code into a plurality of code portions. For example, parsing the code into a plurality of code portions can be performed by a parser module that is part of or associated with the quality assurance module 107. The parser can break down a code file into one or more code portions. Each code file can be lexically analyzed to convert the code text into one or more segments that can be referred to as tokens. Lexical analysis can, for example, identify certain breakpoints within the code file at which some code text can be separated from other code text to form a portion. For example, certain characters can be used as delimiters (e.g., the curly brace '{' or '}' characters). Figure 2 is similar to Figure 4The method includes calculating the complexity of the 202 code portion, extracting 204 one or more comments associated with the code portion from the code portion, converting 206 the one or more comments into a set of text features, quantifying 208 the set of text features, for the one or more comments, using the quantification of the set of text features and the complexity of the code portion to determine 212 the alignment between the one or more comments and the associated code portion, and in response to determining that the one or more comments are not aligned with the associated code portion, triggering 214 a notification.

[0042] Figure 4 The method of Figure 2 differs from the method of Figure 4 in that the method of

[0043] also includes using a Halstead complexity metric to analyze 402 the code portion. In some examples, the Halstead complexity metric can represent a measure of code complexity, which implements an algorithm in code independent of code execution. Using the Halstead complexity metric to analyze 402 the code portion can include statically calculating, by the quality assurance module 107, the Halstead complexity metric from the code portion.

[0044] Figure 4The method also includes using cyclomatic complexity metrics to analyze the 404 code section. Analyzing the 404 code section using cyclomatic complexity metrics can be performed by the quality assurance module 107 that determines the control flow graph of the code section. The control flow graph can refer to a graph data structure where the nodes of the graph correspond to an indivisible set of commands of the code section. The graph can include directed edges between the nodes. A directed edge from a first node to a second node can be found, where, for example, the command of the second node will be executed immediately after the command of the first node. Additionally, the quality assurance module 107 can be configured to identify multiple linearly independent paths through the nodes of the code section, which can represent the loop complexity of the program. This number can be used as a magnitude value representing the code complexity of the code section.

[0045] Figure 4 The method also includes calculating a maintainability index value for the 406 code section. Calculating the maintainability index value for the 406 code section can include the quality assurance module 107 determining various metrics of the code section, such as the lines of code in the code section. Such metrics can indicate a complexity level, for example, in terms of the time or effort (e.g., in man-hours) required to maintain a certain code section.

[0046] Figure 4 The method also includes using natural language processing (NLP) techniques to process 408 one or more comments. Processing 408 one or more comments using natural language processing (NLP) techniques can be performed by the quality assurance module 107 to determine the meaning or value of the linguistic features of the code comments. For example, NLP techniques such as morphological analysis, syntactic analysis, lexical semantic analysis, relational semantics, stylistic analysis, etc. can be used. Additionally, techniques such as part-of-speech tagging can be used to tag or classify specific words in the comments.

[0047] Figure 5 A flowchart illustrating another example method for code comment quality assurance according to some embodiments of the present disclosure is presented. Figure 5 The method of Figure 2 is similar to Figure 5 in that

[0048] Figure 5 The method of Figure 2 differs from the method ofFigure 5 The method also includes generating a training data set of 502 code sections and comment pairs, where each code section and comment pair is associated with an alignment degree. In one embodiment, a training data set can be created having code-comment pairs such as those described above with reference to Figure 2 the code-comment pairs described above. As described above, a comment quality metric can be assigned to the code-comment pairs, such as a value from 1 to 10 on a quality scale. In some embodiments, the metric can be determined by, for example, a relative magnitude ratio or other comparative metric between a code complexity magnitude value (e.g., a Halstead metric indicating code complexity) and a comment feature magnitude value.

[0049] Figure 5 The method also includes using the training data set to train a 504 machine learning model to predict the alignment between a code section and one or more comments associated with the code section based on a quantification of a set of text features extracted from one or more comments and the complexity of the code section associated with the one or more comments. The training data set can then be an input to the machine learning model (such as a linear regression model). The model can provide an output such as a rank of a particular comment based on the comment quality, thus providing an automated, data-driven method for comment quality assessment. Additionally, the process can be iterative. More specifically, the performance of the machine learning model can be iteratively reviewed and refined in order to re-weight the parameters that create a comparative metric (e.g., a magnitude ratio) between the code complexity magnitude value and the comment feature magnitude value.

[0050] Given these comment quality scores, the output of the machine learning model trained on the code-comment pairs can then be used to determine the comment quality of other comments. For example, a code-comment pair that is not part of the training data set can be used as an input to the machine learning model, where the model can then output a rank of the comment of the code-comment pair, which represents the comment quality.

[0051] The reader will understand that artificial intelligence (AI) systems, particularly those supported by large language models (LLMs), can be used to generate code as well as generate code comments, and these AI systems can benefit from the systems and methods described above. In an example implementation, an AI system can generate a code comment and then input the code comment into the machine learning model described above to obtain an indicator (e.g., a rank) of the quality of the comment. The quality metric can be an input to the AI system as a way to improve the performance of the AI system. For example, in the case of a rank of a comment on a scale of 1-10, a particular prompt can be used to prompt the AI system to generate, for example, a comment with certain improvements compared to the comment initially generated by the AI system, and the rank of the comment can be input into the AI system. For example, the AI system can be prompted to generate a more verbose comment.

[0052] In addition, in cases where an AI system generates code based on other code, the above-described systems and methods can provide additional improvements. As an example, the AI system can take COBOL code as input and translate or convert it into Java code. During the Java code generation process, the AI system can also generate code comments. One or more of the generated code comments can be analyzed using, for example, the above-described machine learning model to determine the quality of the comments. The AI system can then be prompted to generate improved comments as part of its Java code generation process. In some cases, comments may not be generated, so the AI system can be prompted to generate code comments for code sections where comments were not generated during Java code generation.

[0053] In certain other cases of generating new code from existing code, it can be observed that the quality of the comments in the existing code may affect the quality of the new code being generated. For example, the AI system can be configured to take existing code such as existing COBOL code as input and also take the code comments in the existing COBOL code as input. The reader will understand that in such cases, improved comments in the existing COBOL code can lead to improvements in the Java code being generated. Additionally, for some code sections in the COBOL code, code comments may be missing. Thus, the AI system can be prompted to generate comments for the COBOL code. For example, in cases where comments already exist, the AI system can be prompted to generate comments for code sections that do not have code comments, or more detailed code comments in place of existing comments, or additional comments in addition to the existing code comments for the code sections. The AI system can then use the generated code comments in its code generation process.

[0054] Aspects of the present disclosure are described by narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). With respect to any flowchart, depending on the technology involved, operations can be performed in an order different from the order shown in a given flowchart. For example, again depending on the technology involved, two operations shown in consecutive flowchart blocks can be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0055] A computer program product embodiment (“CPP embodiment” or “CPP”) is a term used in this disclosure to describe any collection of one or more storage media (also referred to as “media”) collectively included in a set of one or more storage devices, the set of one or more storage devices collectively including machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A “storage device” is any tangible device that can hold and store instructions used by a computer processor. By way of non-limiting example, computer-readable storage media can be electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices such as punched cards or pits / lands formed in the main surface of a disc, or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, should not be construed as storing in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, optical pulses propagating through an optical fiber cable, or electrical signals transmitted through wires and / or other transmission media. As will be understood by those skilled in the art, data is typically moved at certain incidental points in time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but this does not render the storage device transitory because the data is not transitory when it is stored.

[0056] The description of the various embodiments of this disclosure has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen to best explain the principles of the embodiments, the practical application, or the technical improvement present in the marketplace, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for code comment quality assurance, comprising: Calculate the complexity of the code section; extracting, from the code portion, one or more annotations associated with the code portion; converting the one or more annotations into a set of text features; quantifying the text feature set; for the one or more annotations, determining an alignment between the one or more annotations and the associated code portion using the quantification of the set of textual features and the complexity of the code portion; as well as In response to determining that the one or more annotations are not aligned with the associated code portion, a notification is triggered.

2. The method of claim 1, further comprising: Parse the code into multiple code sections.

3. The method of claim 1, wherein: Calculating the complexity of the code portion further includes analyzing the code portion using a Halstead complexity metric.

4. The method of claim 1, wherein: Calculating the complexity of the code portion further includes analyzing the code portion using a cyclomatic complexity metric.

5. The method of claim 1, wherein: Calculating the complexity of the code portion further includes: calculating a maintainability index value of the code portion.

6. The method of claim 1, wherein: Extracting one or more annotations associated with the code portion from the code portion further includes processing the one or more annotations using natural language processing (NLP).

7. The method of claim 1, wherein: The one or more code annotations include annotations generated by an artificial intelligence system.

8. The method of claim 7, wherein: The code portion includes a code portion generated by the artificial intelligence system.

9. The method of claim 1, further comprising: A training data set of code portion and annotation pairs is generated, wherein each code portion and annotation pair is associated with an alignment.

10. The method of claim 9, further comprising: The training data set is used to train a machine learning model to predict an alignment between a code portion and the one or more annotations associated with the code portion based on a quantification of a set of text features extracted from the one or more annotations and a complexity of the code portion associated with the one or more annotations.

11. An apparatus comprising: Processing equipment; as well as A memory operatively coupled to the processing device, wherein the memory stores computer program instructions which, when executed, cause the processing device to perform the method of any one of claims 1-10.

12. A computer program product, comprising computer program instructions, wherein the computer program instructions are executable by a processor to cause the processor to perform the method according to any one of claims 1 to 10.

Images