A system for assisting in the execution of software engineering tasks

Abstract

A system for training a machine learning model to assist in the performance of software engineering tasks is described. The system obtains data from data sources associated with the software engineering tasks. The system links the data by linking each issue report describing any one of the software engineering tasks with source code associated with that one of the software engineering tasks. The system transforms the data to be compatible with a data format used to train a machine learning model to assist in the performance of the software engineering tasks. The system trains the machine learning model using the transformed data to assist in the performance of the software engineering tasks by making predictions of source code changes associated with the software engineering tasks.

Description

[Background technology]

[0001] Software engineering is the process of building functional software from requirements and / or specifications, which are human-centric and typically take the form of natural language documents and supporting data. The built software is machine-centric and typically takes the form of textual source code and supporting data. Software developers often use software tools to assist them in these complex tasks of creating requirements and specification documents and building source code.

[0002] Requirements and specification documents need to be interpreted and refined in a process that involves understanding ideas at various levels of abstraction, from vague to precise, and organizing these ideas into a coherent software design. Software engineers need to understand technical natural language documents and align their implied needs with the capabilities of the computing platform underlying the source code.

[0003] Software tools for requirements and specification management and design / architecture are a broad category of products that include process-focused management systems that facilitate the communication and cataloging of requirements and specifications, modeling tools that allow the visualization of potential software designs of various fidelity, and project management systems that are often used to store requirements and specifications and track their progress toward implementation. These early-stage software tools are similar in that they facilitate relatively narrow tasks and do not even attempt to fully automate these tasks.

[0004] Software tools for building software form an even larger category of products. Most of these tools are implemented in code editors, which provide assistance with many of the following tasks: Code editors make source code easier to read by organizing and highlighting it and facilitating source code navigation through hyperlinks. Source code can include readme files and other types of text files, such as boilerplate license headers, that are commonly associated with source code files. These software building tools edit the code to make its syntax style consistent and add dependency source code constructs, such as import statements. Code editors generate boilerplate source code from fixed templates and reduce typing by completing partially typed words or lines of source code. While software building tools streamline the process of writing source code, they do not automate writing source code or help software developers write source code that is better connected to requirements or specifications. [Brief explanation of the drawings]

[0005] [Figure 1A] 1 illustrates example lines of source code that may be used to assist in performing a software engineering task, according to one embodiment. [Figure 1B] 1 illustrates example lines of source code that may be used to assist in performing a software engineering task, according to one embodiment. [Figure 2] 1 illustrates a block diagram of an exemplary system for assisting in the performance of software engineering tasks, according to one embodiment. [Figure 3] 1 depicts a flowchart illustrating a method for training a machine learning model to assist in performing a software engineering task, according to one embodiment. [Figure 4A]1 depicts a flowchart illustrating a method for using a machine learning model to assist in performing a software engineering task, according to one embodiment. [Figure 4B] 1 depicts a flowchart illustrating a method for using a machine learning model to assist in performing a software engineering task, according to one embodiment. [Figure 5A] 1 depicts a flowchart illustrating a method for assisting in writing an issue report describing a software engineering task and issue information, according to one embodiment. [Figure 5B] 1 depicts a flowchart illustrating a method for assisting in writing an issue report describing a software engineering task and issue information, according to one embodiment. [Figure 6A] 1 depicts a flowchart illustrating a method for assisting in the automation of software engineering tasks, according to one embodiment. [Figure 6B] 1 depicts a flowchart illustrating a method for assisting in the automation of software engineering tasks, according to one embodiment. [Figure 7A] 1 depicts a flowchart illustrating a method for assisting in writing source code for a software engineering task, according to one embodiment. [Figure 7B] 1 depicts a flowchart illustrating a method for assisting in writing source code for a software engineering task, according to one embodiment. [Figure 8A] 1 depicts a flowchart illustrating a method for assisting in identifying unexpected portions of a source code file for a software engineering task, according to one embodiment. [Figure 8B] 1 depicts a flowchart illustrating a method for assisting in identifying unexpected portions of a source code file for a software engineering task, according to one embodiment. [Figure 9]FIG. 1 is a block diagram illustrating an exemplary hardware device in which the present subject matter may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0006] 1A and 1B show exemplary lines of source code that may be used to assist in the performance of a software engineering task, according to one embodiment. Source code line 102 shows exemplary source code for performing standard artificial intelligence (AI)-driven code completion, which interprets comments and generates entire lines of source code, as depicted by FIG. 1A. Source code line 104 shows exemplary source code for performing more sophisticated AI-driven code completion, which generates entire function bodies from function names and lines of documentation contained in comments, as depicted by FIG. 1A, similar to existing state-of-the-art code completion-style tools.

[0007] Embodiments of the present disclosure provide a system that includes higher-level tools not similar to others on the market. For example, the system integrates more directly with software issue trackers, allowing developers to experiment more purely with natural language input. This system differs significantly from software issue trackers that generate issue reports by allowing specialization of behavior through textual descriptions of software engineering task requirements and specifications. Recognizing that change is central to modern software development, this system extends standard code completion idioms with issue reports and machine learning models that can suggest source code changes, and a toolchain that can present these changes in a streamlined manner.

[0008] The issue report 106 describes the software engineering task, and the system can automatically read it from issue tracking software used by software developers to compile issue reports, requirements, and specifications, as depicted by FIG. 1A. The text of the issue report has a strong influence on what source code is generated, as this ability is entirely derived from the system's de novo use of project history. The source code suggestions or predictions are focused on the specific software engineering task. Source code line 108 indicates a specific prediction of the source code, which is based on the software engineering task issue report 106 and appears within the context of the existing source code, as depicted by FIG. 1A.

[0009] At times, the machine learning model is confident in the scope of the source code change—not only what needs to be changed, but also where the change will end up. Source code line 110 shows the predicted source code change presented as a boxed inline popup, fully isolated to a single line, as depicted by Figure 1B.

[0010] When confident, the machine learning model can suggest source code changes in locations other than where the software developer is currently working. Source code line 112 shows the complete source code change prediction, eight lines below the current cursor, as depicted by Figure 1B. In some cases, the predicted source code change may be hundreds of lines away or in an entirely different source code file. Being able to localize, rather than simply predict, the source code change prediction is key to achieving the ambitious challenge of having artificial intelligence solve issues on its own.

[0011] In addition to predicting new lines of source code, the system can help software developers adapt and modify existing source code. Source code line 114 shows the machine learning model having high confidence in predicting portions of source code that do not match corresponding portions of existing source code, and thus rendering the corresponding portions of the existing source code as highlighted in bold and underlined, as depicted by FIG. 1B. A software developer notices such clear signals in the corresponding portions of the existing source code, selects the highlighted portions of the source code, examines a comparison between the existing portions of the source code and the predicted portions of the source code, recognizes the errors, and corrects the errors.

[0012] Embodiments of the present disclosure provide a system and agent that automates generalized software engineering tasks, broadly encompassing tasks involving interpreting issue reports, requirements, and specifications and implementing the interpreted issue reports, requirements, and specifications as source code. The system teaches and learns from various sources of software engineering data. Previously performed and recorded tasks are standardized, abstracted, and incorporated into one or more machine learning models. The agent then acts as an interface between the software developer and the one or more machine learning models, leveraging the knowledge learned from past tasks to streamline and automate current and future tasks.

[0013] Embodiments herein provide a system for training a machine learning model to assist in the performance of software engineering tasks. The system obtains data from data sources associated with the software engineering tasks. The system links this data by linking each issue report describing any one of the software engineering tasks with source code associated with that one of the software engineering tasks. The system transforms the data to be compatible with a data format used to train a machine learning model to assist in the performance of the software engineering tasks. The system trains the machine learning model using the transformed data to assist in the performance of the software engineering tasks by making predictions of source code changes associated with the software engineering tasks.

[0014] For example, the system may acquire data from large-scale open software engineering data through a dynamic data pipeline, and the scale of learned knowledge may grow to a collective software engineering intelligence knowledge that captures the current state-of-the-art across the entire software engineering field. The system links the acquired data by linking source code commits in a version control system to issue reports in an issue tracker that the commits resolve, and by linking the issue reports to more general documents referenced by the issue reports. The system converts the linked data into a common, unified format, such as representing commits in text format as encoded differences between before and after states. The system uses the converted data to train a machine learning model, which enables it to make diverse predictions about source code changes for software engineering tasks. The more links between past issue reports and past source code changes used to train the machine learning model, the more current source code changes the machine learning model can predict for a current issue report, and the more accurate the predictions will be.

[0015] Embodiments herein provide a system that uses machine learning models to assist in the execution of software engineering tasks. The system receives a request from a software developer's issue tracker or code editor to begin work on a software engineering task and outputs an issue report describing the software engineering task and / or source code associated with the software engineering task to the issue tracker and / or code editor. The system saves updates to the issue report or source code changes associated with the software engineering task received from the issue tracker and / or code editor. The system receives a request from the issue tracker or code editor for predicted completion of the software engineering task, obtains context data establishing the context of the software engineering task, and converts the context data to be compatible with a data format used to train a machine learning model to assist in the execution of the software engineering task. The trained machine learning model uses the converted context data to predict any number of completions of the software engineering task. The system enables a software developer to complete the software engineering task by outputting the predicted completion of the software engineering task to the issue tracker and / or code editor.

[0016] For example, the system outputs a requested issue report and / or source code for a software engineering task to a code editor or issue tracker of a software developer named Sofia to fix a body validation error response, and then the system saves Sofia's updates, which include Sofia's source code changes. In response to Sofia's request for predicted completion of the software engineering task to fix the body validation error response, the system obtains context data establishing the context of the software engineering task, such as Sofia's source code changes. The system then converts the context data, including Sofia's source code changes, to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task. The trained machine learning model uses the converted context data to predict one set of source code changes for the software engineering task with 95% prediction confidence and another set of source code changes for the software engineering task with 90% prediction confidence. The system outputs a compact representation of the set of predicted source code changes to Sofia, which can use its issue tracker to expand the selected compact representation and provide Sofia with the predicted source code changes within the context of the existing source code, e.g., the predicted source code change at line 651 of lines 108 of the source code depicted by FIG. 1A, which Sofia can accept to be committed. Upon enabling Sofia to complete the software engineering task, the machine learning model can provide multiple alternative sets of predicted source code changes with sufficiently high prediction confidence to be reasonable alternatives for completing the software engineering task, from which Sofia can select one of the sets of predicted source code changes to be committed.

[0017] Embodiments herein provide a system for assisting in writing an issue report describing a software engineering task and issue information. The system receives a request from an issue tracker associated with a stakeholder of the software engineering task to initiate an open issue report describing the software engineering task and issue information and intended for a software developer, and assigns the software engineering task to the software developer. The system receives the stakeholder's request to predict completion of the open issue report, obtains context data establishing a context for the software engineering task, and transforms the context data to be compatible with a data format used to train a machine learning model to assist with the software engineering task. The trained machine learning model uses the transformed context data to predict completion of the open issue report describing the software engineering task and issue information. The system enables the software developer to complete the software engineering task by outputting to the software developer's issue tracker an accepted completion of the open issue report based on the predicted completion of the open issue report describing the software engineering task and issue information.

[0018] For example, the system may initiate a requested issue report for a software engineering task to fix a body validation error response to an issue tracker for Stacy Holder, a stakeholder in the software engineering task, and then respond to Stacy's next request by assigning the software engineering task to a software developer named Sofia. In response to Stacy's next request to predict completion of Stacy's open issue report, the system retrieves context data establishing the context of the software engineering task and converts the context data, including the open issue report, to be compatible with a data format used to train a machine learning model to assist in the execution of the software engineering task. The trained machine learning model uses the converted context data to predict completion of the open issue report for the software engineering task to fix a body validation error response, correcting the description of the software engineering task to require removal of a variable name when the variable name is not embedded, instead of an incorrect description of the software engineering task that requires removal of a variable name when the variable name is not omitted. The system enables Sofia to complete its assigned software engineering tasks by outputting corrected statements that, in addition to the machine learning model being trained to predict source code changes for software engineering tasks, the machine learning model has been trained on sufficiently diverse issue reports to be able to predict completion of incomplete issue reports, which may require the removal and / or replacement of portions of the incomplete issue reports, such as removing / replacing the word "omitted" that was erroneously included in the software engineering task.

[0019] Embodiments herein provide a system for assisting in the automation of software engineering tasks. The system receives a request from a software developer's issue tracker to review an issue report describing a software engineering task and outputs the issue report to the issue tracker. The system receives the issue tracker's request for predicted source code changes for the software engineering task, obtains context data establishing a context for the software engineering task, and transforms the context data to be compatible with a data format used to train a machine learning model to assist in the execution of the software engineering task. The trained machine learning model uses the transformed context data to predict source code changes for the software engineering task. The system outputs the predicted source code changes for the software engineering task to the issue tracker. The system commits source code changes based on the predicted source code changes to source code associated with the software engineering task that are accepted by the issue tracker.

[0020] For example, the system outputs a requested issue report describing a software engineering task to fix a body validation error response to a software developer's issue tracker named Sofia. In response to Sofia's request to predict source code changes for the software engineering task to fix the body validation error response, the system obtains context data establishing the context of the software engineering task, such as Sofia's updates to the issue report for the software engineering task. The system then converts the context data, including Sofia's updated issue report, to be compatible with a data format used to train a machine learning model to assist in the execution of the software engineering task. The trained machine learning model uses the converted context data to predict source code changes, such as new source code at line 651 of the lines of source code 108 depicted by FIG. 1A. The system outputs the predicted source code changes for the software engineering task to Sofia, who commits any source code changes that Sofia accepts. Even if Sofia only works on an issue report describing Sofia's software engineering task without generating any source code changes for the software engineering task and then requests predicted source code changes for the software engineering task, the system can use the updated issue report to predict all source code changes required to complete the software engineering task, thereby automating the generation of the source code changes.

[0021] Embodiments herein provide a system for assisting in writing source code for a software engineering task. The system saves source code changes received from a software developer's code editor to a location in the source code associated with the software engineering task. The system receives a request from the code editor for predicted source code changes at the source code location, obtains context data establishing a context for the software engineering task, and transforms the context data to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task. The trained machine learning model uses the transformed context data to predict source code changes at the source code location. The system outputs the predicted source code changes at the source code location to the software developer's code editor. The system commits source code changes based on the predicted source code changes at the source code location to the source code associated with the software engineering task, as accepted by the software developer.

[0022] For example, the system stores a portion of the source code changes generated by a code editor of a software developer named Sofia that are required for a software engineering task of correcting a body validation error response. In response to Sofia's request for predicted source code changes at line 701 of the source code where Sofia is working on the software engineering task of correcting a body validation error response, the system retrieves context data that establishes the context of the software engineering task, such as Sofia's source code changes. The system then converts the context data, including Sofia's source code changes, to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task. The trained machine learning model uses the converted context data to predict the source code changes at line 701 of the source code to complete the localized software engineering task of correcting a body validation error response, as shown by line 704 of the lines of source code 110 depicted by FIG. 1B. The system outputs the predicted source code changes at source code line 704 to Sofia's code editor, allowing Sofia to complete its localized software engineering task, commit the predicted source code changes that Sofia has accepted, and move on to any other location in the source code where Sofia wishes to focus on completing the software engineering task of fixing the body validation error response.

[0023] Embodiments herein provide a system for assisting in identifying unpredicted portions of a source code file for a software engineering task. The system receives source code changes from a code editor associated with a software developer and saves the source code changes to a source code file associated with the software engineering task. The system receives a request from the code editor to predict source code for the source code file, obtains context data establishing a context for the software engineering task, and transforms the context data to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task. The trained machine learning model uses the transformed context data to predict source code for the source code file, and portions of the source code file correspond to portions of the predicted source code. The system identifies, via the code editor, each portion of the source code file that is determined to differ from the corresponding portion of the predicted source code. The system commits any different portions of the predicted source code requested and accepted by the code editor to the source code file.

[0024] For example, the system stores source code changes generated by a software developer's code editor, Sofia, in a source code file for a software engineering task that corrects a body validation error response. In response to Sofia's request to predict source code for a source code file for a software engineering task that corrects a body validation error response, the system obtains context data that establishes a context for the software engineering task, such as Sofia's source code changes in the source code file. The system then converts the context data, including Sofia's source code changes, to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task. The trained machine learning model uses the converted context data to predict source code for the source code file, where each portion of the source code file corresponds to a predicted portion of the source code. Using her code editor, Sofia reviews the bold-highlighted and underlined portion of line 701 ("body", field_alias) of source code 114 depicted by FIG. 1B, selects the bold-highlighted portion of line 701 that displays the predicted portion of the source code (loc) that differs from the bold-highlighted portion of line 701, and accepts this predicted portion of the source code to replace the bold-highlighted portion of line 701. If Sofia's source code changes begin to fix the body validation error response, many of the highlighted portions of the source code file may identify predicted source code necessary to complete Sofia's software engineering task, while if Sofia's source code changes complete Sofia's task, some of the highlighted portions of the source code file may identify predicted source code that corrects Sofia's possible mistakes and / or makes Sofia's source code more efficient.

[0025] 2 illustrates a block diagram of a system 200 for assisting in the performance of software engineering tasks, according to one embodiment. As shown in FIG. 2, system 200 may exemplify a cloud computing environment in which data, applications, services, and other resources are stored and distributed through a shared data center and appear as a single point of access for end users. System 200 may also represent any other type of distributed computer network environment in which a server controls the storage and allocation of resources and services for different client users.

[0026] In one embodiment, system 200 represents a cloud computing system including a first client 202, a second client 204, and a first server 206 and a second server 208, which may be provided by a hosting company. Clients 202-204 and servers 206-208 communicate via network 210.

[0027] Client 202, which may be referred to as software developer client 202, includes word-processor-like software for editing source code, such as a version control system, which may be referred to as code editor 212, which incorporates coding assistant 214, which may be implemented as a plug-in that extends the native functionality of code editor 212 by focusing on helping a user write the most appropriate source code to accomplish a given task. Similarly, client 202 includes word-processor-like software, which may be referred to as issue tracker 216, for writing issue reports and natural language descriptions of project requirements and specifications for software engineering tasks, which incorporates writing assistant 218, which may be implemented as a plug-in that extends the native functionality of issue tracker 216 by focusing on helping a user write the most appropriate natural language text to accomplish a given task. Similarly, the client 204 , which may be referred to as a stakeholder client 204 of a software engineering task, may include a code editor 220 that may incorporate a coding assistant 222 and an issue tracker 224 that may incorporate a writing assistant 226 .

[0028] Server 206, which may be referred to as training server 206, may include a dynamic data pipeline 228, one or more machine learning models 230, an acquisition component 232, a linking component 234, a transformation component 236, and a training component 238. Server 208, which may be referred to as production server 208, may include a dynamic data pipeline 240, one or more machine learning models 242, and an agent 244. Machine learning models 230 and 242 may be based on any of a variety of models, such as a gradient boosting classifier, a k-nearest neighbor classifier, a neural network, a random forest, a support vector machine, a naive Bayes classifier, and a logistic regression model. Assistants 214, 218, 222, and 226 may be provided by training server 206 and / or production server 208 to assist code editor 212, issue tracker 216, code editor 220, and issue tracker 224 in interacting with components 228-238 residing on training server 206 and / or system components 240-244 residing on production server 208. System 200 may include any number of clients 202-204, any number of servers 206-208, any number of networks 210, and any number of components 212-244 depicted in FIG. 2 as residing on either the clients 202-204 or the servers 206-208.

[0029] Each of the clients 202-204 and the servers 206-208 may be substantially similar to the system 900 depicted in Figure 9 and described below. Although Figure 2 depicts system components 228-238 residing entirely on the first server 206 and system components 240-244 residing entirely on the second server 208, the system components 228-244 may be residing entirely on the first server 206, entirely on the second server 208, entirely on the clients 202-204, entirely on another server not depicted in Figure 2, or any combination of being partially residing on the servers 206-208, partially residing on the clients 202-204, and partially residing on other servers.

[0030] Software engineers, software developers, and software project stakeholders typically keep digital records of their work in a variety of stores, including issue trackers, project documentation, text communications, and version control systems. Issue trackers were originally designed to track software defects, but today they are used to file and track the progress of all types of software engineering work. Issue trackers contain numbered, natural-language issue reports that can be assigned as work items to personnel, such as software developers. These issue reports have meta-labels, such as priority or subsystem, and they have commenting and editing capabilities that allow them to be updated as the work progresses.

[0031] Project documentation consists of documents written by the software developers themselves or other software project stakeholders. These documents may contain information about project requirements, specifications, schedules, progress of work, and / or descriptions of the software being built. Text communication occurs between team members, both software developers and other software project stakeholders, and can be synchronous or asynchronous, and in person (one-on-one) or in groups. Prime examples include email and chat platforms.

[0032] A version control system is a temporary database that holds an authoritative copy of a software engineering project's source code. Version control systems are notable for storing the entire history of all the data they contain. The history is divided into atomic units, commonly called commits, which contain a set of file-level changes associated with a text description. This history allows software developers to trace the origin of source code changes and retrieve any recorded version of the source code. In addition to the software source code, version control systems often also contain configuration, supporting data, and even some form of documentation.

[0033] An embodiment of system 200 learns by processing disparate software engineering data from different sources into a common stream and using the stream of data to train machine learning models 230 to assist in the performance of software engineering tasks. Figure 3 is a flowchart illustrating a method for training machine learning models 230 to assist in the performance of software engineering tasks, according to one embodiment. Flowchart 300 depicts method acts illustrated as flowchart blocks of particular steps involved within and / or between clients 202-204 and / or servers 206-208 of Figure 2.

[0034] The system acquires data from multiple data sources associated with the software engineering task (block 302). The system collects various types and amounts of data. For example, without limitation, this may include the acquisition component 232 acquiring all data through the dynamic data pipeline 228 into a central database.

[0035] Data can be information that has been transformed into a form that is efficient for movement or processing. A data source can be a place where information originates. A software engineering task can be a quantity of work to design, develop, test, and / or maintain a computer program.

[0036] Each source of data is connected to a central location. Local data sources are accessed by reading files, and network sources are accessed via network file transfers or more standardized application programming interfaces (APIs). The acquisition component 232 can copy the data completely to the central database, or the acquisition component 232 can connect the data so that it is accessible on demand.

[0037] The acquisition component 232 may acquire data from data sources, including data sources associated with multiple software engineering projects associated with a single enterprise. For example, the training server 206 can scale up data collection and subsequent learning procedures from information on a single project to data collection and training for multiple projects. Within an organization, this process can foster institutional memory, allowing work and knowledge learned from past projects across the organization to be transferred to new and current projects. A software engineering project can be a carefully planned endeavor to achieve a specific goal in the design, development, testing, and / or maintenance of a computer program. A single enterprise can be a business establishment or a company.

[0038] The acquisition component 232 can acquire data from data sources, including open source software projects. For example, the training server 206 can further expand the same data collection and learning procedures by using large-scale open software engineering data sources, such as those published by open source software projects. As the training server 206 scales, the knowledge learned grows from institutional memory to a body of collective software engineering intelligence, capturing the current state-of-the-art of the entire software engineering field. An open source software project can be a carefully planned endeavor to achieve a specific goal in the design, development, testing, and / or maintenance of a computer program, with code made freely available for possible modification and redistribution.

[0039] The training server 206 encodes the project history as text and trains the project history within a large-scale language model. The training server 206 can scale this process infinitely to achieve any level of accuracy, subject to computational cost and data availability. Project history models follow the same scaling laws as other large-scale language modeling tasks: scaling them up with more data and more computing power makes them more accurate, because the more resources you put into them, the more accurate they become. As the training server 206 diversifies its data from more and more different projects, accuracy increases to breakthrough levels.

[0040] After obtaining the data, the system links the data by linking each issue report describing one of the software engineering tasks with the source code associated with that one of the software engineering tasks (block 304). The system links the data to enable prediction of source code data based on the issue report data. By way of example, and not limitation, this may include the linking component 234 linking the data together to establish locality. An issue report may be a text account describing a problem. Source code may be a text list of commands to be compiled or assembled into an executable computer program. Additionally, source code may include readme files and other types of text files, such as boilerplate license headers, that are commonly associated with source code files.

[0041] Many data entries are more related to each other than to other data entries; for example, documentation about subsystem A is more related to the source code of subsystem A than this documentation is to any other source code. Link component 234 can link source code commits from a version control system, such as code editor 212, to issue reports in issue tracker 216 that the commits resolve, and link issue reports to more general documents that the issue reports reference. Training server 206 can expand these links and store them in a central database.

[0042] Following linking of the data, the system converts the data to be compatible with the data format used to train the machine learning model (block 306). The system unifies the various data formats of the various types of data. In embodiments, this may include the conversion component 236 converting the data into a common, unified format that is compatible with the learning methodology being used.

[0043] Compatibility may be the ability to be used with specified software without special adaptation or modification. Data format may be the structural organization of information. Machine learning model may be the application of artificial intelligence to dynamic data, providing a system with the ability to automatically learn and improve from experience without being explicitly programmed.

[0044] Each data source is primarily text-based, but the structure and content of that text varies. Transformations vary depending on the input data and training method, but an example transformation may include the transformation component 236 ingesting source code commits, which may generally be considered before and after states, from a version control system and representing the commits in text format as encoded differences between the two states. Another example of a transformation may include the training server 206 filtering irrelevant and unhelpful portions of individual data items, such as boilerplate license headers from source code files. The transformation component 236 may transform data separately, thereby generating a processed dataset, or the transformation component 236 may transform data on-demand as part of a pipeline.

[0045] After transforming the data, the system trains a machine learning model using the transformed data to aid in the performance of the software engineering task by making predictions of source code changes for the software engineering task (block 308). The system trains the machine learning model to predict source code for a current issue based on the history of source code created for the issue. For example, without limitation, this may include the training component 238 applying a learning method to train the machine learning model 230. The training component 238 can feed the complete pipeline to the learning method, which trains the machine learning model 230 using data that can make various predictions about that data and similar data, such as source code changes for the software engineering task. Examples of learning methods include rule-based engines and machine learning methods such as deep neural networks.

[0046] Transformed data may be information converted from one form to another. A prediction may be a forecast. A source code change may be an alteration to a text list of commands that is compiled or assembled into an executable computer program.

[0047] The training component 238 trains each instance of the machine learning model 230 on the data contributed by the open source developers strictly before the deadline date, and then performs evaluations / experiments on all data strictly after that date. In layman's terms, the training component 238 evaluates the ability of the machine learning model 230 to perform the "new" work when trained on the "old" work.

[0048] Software engineering is a discipline that is constantly being updated. Broadly speaking, best practices evolve, new programming languages ​​emerge, and libraries and other components rise and fall in popularity. More urgent updates occur when previously unknown security vulnerabilities are discovered that must be addressed in current software and prevented in future source code. The training server 206 can adapt the steps of acquiring, linking, and transforming data for a typical training procedure by adding new source data, thereby refreshing the database with new input. These same steps of acquiring, linking, and transforming data also serve to remove existing data, allowing bad or outdated software engineering knowledge to be forgotten.

[0049] Thus, after initially training the machine learning model, the system iteratively obtains additional data from the data source, links the additional data, transforms the additional data, and trains the machine learning model with the transformed additional data (block 310). The system can update the training of the machine learning model. By way of example, and not limitation, this may include the training server 206 applying a learning procedure using the updated data to create an updated machine learning model 230. Iteratively training the machine learning model 230 with the transformed additional data may include initializing the machine learning model 230 and then training the initialized machine learning model 230 using only the transformed additional data, training the machine learning model 230 using both the transformed additional data and previously transformed data, or training the machine learning model 230 incrementally using the transformed additional data. For example, the training server 206 can train the machine learning model 230 entirely from scratch using the additional data, creating an entirely new machine learning model 230 from the additional data, or incrementally training the machine learning model 230 using the additional data.

[0050] Transformed additional data can be supplemental information converted from one form to another. Incrementally can be periodic increases, additions, or steps. An initialized machine learning model can be the application of artificial intelligence to dynamic data, providing a system with the ability to automatically learn and improve from experience without being explicitly programmed and placed in a starting state.

[0051] The training server 206 adds, deletes, and updates data as annotated data in the database, which data is identifiable by the learning procedure. If supported, the training of the machine learning model 230 can then resume its process using the current machine learning model 230 as input, focusing only on updating the data. Such focus can be facilitated by the training server 206 allowing the learning procedure to access only updated data, or the training server 206 can continue to access all data but adjust the weighting of the updated data accordingly.

[0052] A standard learning procedure can enable the machine learning model 230 to learn knowledge of an individual project or team, while a scaled learning procedure can enable the machine learning model 230 to learn general software engineering knowledge. Often, both types of machine learning models 230 are beneficial. Thus, the machine learning models 230 can include a group of machine learning models 230 trained on data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge, or machine learning models 230 trained on both data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge. For example, the downstream agent 244 can utilize these machine learning models 230 adapted to use two or more of the machine learning models 242, such as by using one focused machine learning model 242 for a team-specific task and one broad machine learning model 242 for general knowledge. However, such a procedure can also be facilitated by the learning procedure itself, such as by generating just one hybrid machine learning model 242 trained on both team-specific tasks and broad general knowledge.

[0053] Assuming the training server 206 has performed the above-described large-scale adaptation and generated the general machine learning model 242, the training server 206 can perform the data acquisition, linking, and transformation steps only for the possibly private and confidential data of one project, team, or individual. Thus, training the machine learning model 230 may involve learning with data associated with general software engineering knowledge and then learning with data associated with a single company, which is weighted lower than the data associated with the general software engineering knowledge. For example, the training server 206 can adapt by applying a learning method to updated data based on using the general machine learning model 230 as an initialization point and then training the specific machine learning model 230 with specialized data. To facilitate blending knowledge captured in the machine learning model 230 without overwriting it, the training server 206 can adapt the learning procedure to lower the weight of the new specialized data. The general software engineering knowledge may be a broad understanding of the design, development, testing, and / or maintenance of computer programs. The more links between past issue reports and past source code changes used to train the machine learning model, the more current source code changes the machine learning model can predict for the current issue report, and the more accurate the predictions will be.

[0054] System 200 is built around a project history machine learning model 242 that smoothly integrates assistance with a software developer's existing workflow. System 200 exposes machine learning model 242 through tools that resemble but functionally encompass code completion-style tools, such as code editors 212 and 220. Such code completion-style tools have the advantage of allowing software developers to move very smoothly between assisted and unassisted development. When such a tool does something unexpected, there is no shocking moment that interrupts the flow; the software developer simply ignores the assistance and continues typing.

[0055] 3 depicts blocks 302-310 occurring in a particular order, blocks 302-310 may occur in another order. In other implementations, each of blocks 302-310 may be performed in combination with other blocks and / or some blocks may be divided into different sets of blocks.

[0056] To facilitate and automate software engineering tasks, system 200 connects machine learning model 242 to software developers and their data through agent 244. The role of agent 244 is to collect real-time data about the software engineering task at hand, send the real-time data to machine learning model 242 to inquire how to complete or assist with the software engineering task, and present that automation or assistance to the human software developer.

[0057] A general form of agent 244 includes a dynamic data pipeline 240 that mirrors the dynamic data pipeline 228 used to build machine learning models 242. Agent 244 uses dynamic data pipeline 240 to establish the context of the current software engineering task. Agent 244 applies this context with machine learning models 242 to generate some form of fully or partially automated work. That work is then presented by agent 244 through the instantiation of various specific tools.

[0058] The machine learning model 242 generated by the learning procedure captures a wide range of software engineering knowledge. The machine learning model 242 learns about issue reports, requirements, specifications, and designs from natural language sources, and about software construction via programming languages ​​from structured source code sources. Linked data sources enable the machine learning model 230 to connect these different types of knowledge.

[0059] The steps of acquiring, linking, and transforming data for a general learning procedure describe the connecting, centralizing, linking, and normalizing of software engineering data as it is consumed by the machine learning model 230 during the learning procedure. Use of such machine learning models 242 by agents 244 requires extracting or querying knowledge from the machine learning models 242, which requires a similar, but localized, data extraction procedure. The software engineering work performed or facilitated by agents 244 is performed in the context of a specific project and software engineering task. A subset of the project's linked data is relevant to that software engineering task and must be acquired when supporting that software engineering task. For example, a software engineering task assigned through issue tracking software may include the text of the issue report as dynamic data; a software engineering task discussed in a particular thread in a chat or email system may include the text of that thread; and a task related to one particular subsystem of a software project may include the programming language and source code of that subsystem.

[0060] The collection of such data is performed in dynamic data pipeline 240. The collection or acquisition mechanism may be similar to that used in instantiating a typical learning procedure, but must accommodate additional constraints. Typically, a single software engineering task at hand will have much less data than is consumed during training, and current data must be updated and collected in real time. For example, if a participating project stakeholder makes a comment clarifying the original issue report regarding future interactions with machine learning model 242, dynamic data pipeline 240 can include this clarification.

[0061] If the software developer seeks further clarification about the software engineering task from a colleague through a chat system, future model queries by agent 244 may include the text of this interaction. If the software developer performs part of the software engineering task by writing and saving some amount of source code, further interactions between agent 244 and machine learning model 242 will include this partially finished work. If the software developer deletes some amount of source code to partially complete the software engineering task, continuing interactions will include the fact that this source code was deleted.

[0062] The dynamic data pipeline 240 is responsible for maintaining constant data connections to each of these sources. Exemplary forms of these connections include application programming interfaces (APIs), database connections, and local or network file system read operations.

[0063] Agent 244 uses dynamic data pipeline 240 to establish the context of a given software engineering task, which is presented as a query to machine learning model 242. Training server 206 may, and likely and ideally would, expose machine learning model 230 to similar or similarly adaptable contexts, allowing machine learning model 242 to use the embedded knowledge to make predictions about how the software engineering task should be performed.

[0064] 4 is a flowchart illustrating a method for using machine learning models to assist in the performance of software engineering tasks, according to one embodiment. Flowchart 400 depicts method acts illustrated as flowchart blocks of particular steps involved within and / or between clients 202-204 and / or servers 206-208 of FIG. 2.

[0065] After a system, such as an issue tracking system, assigns a software engineering task to a software developer, the software developer logs on to issue tracker 216 on client 202 and begins reading the task description. The system receives a request from a code editor or issue tracker associated with the software developer to begin work on the software engineering task (block 402). The system stores the source code and issue report for the software engineering task. For example, without limitation, this may include agent 244 receiving a request from code editor 212 or issue tracker 216 on software developer Sofia's client 202 to begin work on a software engineering task to correct a body validation error response by writing source code or editing an issue report and to remove variable names if they are not embedded.

[0066] A request may be a message asking for information or a resource. A software developer may be someone who designs, creates, and / or maintains applications that allow users to perform specific tasks on a computer. A code editor may be a tool used to write software. An issue tracker may be a tool that helps manage and address issues.

[0067] After receiving a request from the software developer to begin work on the software engineering task, the system outputs an issue report describing the software engineering task and / or source code for the software engineering task to an issue tracker and / or code editor associated with the software developer (block 404). The system provides the source code and issue report to the software developer. By way of example, and not limitation, this may include agent 244 outputting the issue report and / or source code to Sofia for a software engineering task that fixes body validation error responses and removes variable names if they are not embedded.

[0068] Following optional output of the issue report to the software developer, the system optionally allows the software developer to clarify the issue report describing the software engineering task via an issue tracker associated with a stakeholder of the software engineering task and / or update the issue report to describe a strategy for completing the software engineering task (block 406). The system enables updates to the issue report describing the software engineering task. In an embodiment, this may include agent 244 enabling Sofia to communicate with issue tracker 224 on client 204 of Stacy Holder, a stakeholder of the software engineering task, to correct body validation error responses and remove variable names if they are not embedded. Sofia and Stacy agree on a high, but not the highest, priority for the software engineering task to correct body validation error responses and remove variable names if they are not embedded.

[0069] A stakeholder can be anyone who has a connection or interest in something, especially a business. A strategy can be a plan of action or policy designed to achieve a primary or overall objective.

[0070] After optionally clarifying the issue report, the system stores any updates to the issue report describing the software engineering task and / or any source code for the software engineering task received from the issue tracker and / or code editor associated with the software developer (block 408). The system stores any updates to the software engineering task's source code and the issue report. For example, without limitation, this may include the agent 244 storing an update to Sofia in the dynamic data pipeline 232, including a high, but not highest, priority for the software engineering task that fixes body validation error responses and removes variable names if they are not embedded, and a limited amount of Sofia source code changes. The update may be replacing an older version with a newer version.

[0071] After outputting the issue report and / or source code of the software engineering task to the software developer, the system receives an implicit or explicit request from the software developer's issue tracker or code editor to predict completion of the software engineering task (block 410). The system is asked to predict the source code of the software engineering task. By way of example, and not limitation, this may include the agent 244 receiving an explicit request from Sofia's issue tracker 216 or code editor 212 for predicted completion of the software engineering task to correct body validation error responses and remove variable names if they are not embedded.

[0072] An explicit request can be a message that explicitly asks for information or a resource. An implicit request can be a message that implies a request for information or a resource. Completion can be the act of ending an action, activity, or process.

[0073] In addition to receiving explicit requests to predict completion of a software engineering task, agent 244 can identify implicit requests to predict completion of a software engineering task. For example, Sofia launches its issue tracker 216 and begins editing an issue report for the software engineering task that fixes body validation error responses and removes variable names if they are not embedded, and / or launches its code editor 212 and begins writing source code for the software engineering task that fixes body validation error responses and removes variable names if they are not embedded, and then stops and waits when agent 244 notifies Sofia that if no editing activity continues while a displayed timer counts down without interruption, machine learning model 242 will predict completion of the software engineering task that fixes body validation error responses and removes variable names if they are not embedded, based on the issue report and source code for the software engineering task.

[0074] After receiving a request to predict the completion of a software engineering task, the system retrieves context data establishing the context of the software engineering task (block 412). The system retrieves the context data of the software engineering task to accurately predict the software engineering task. In an embodiment, this may include agent 244 retrieving context data establishing the context of the software engineering task from dynamic data pipeline 232, such as a description for a high, but not highest priority software engineering task that corrects body validation error responses and removes variable names if they are not embedded, and updates to issue reports, as well as Sofia's source code changes from dynamic data pipeline 240. The context may be the circumstances that form the setting for the behavior. The context data may be information about the circumstances that form the setting for the behavior, converted into a form that is efficient for movement or processing.

[0075] After obtaining the context data establishing the context of the software engineering task, the system converts the context data to be compatible with a data format used to train a machine learning model to assist in the execution of the software engineering task (block 414). The system converts the context data of the software engineering task to be in a format used to predict the software engineering task. For example, without limitation, this may include the agent 244 converting the obtained issue reports and source code changes to be compatible with a data format used to train the machine learning model 230 to learn to predict the completion of the software engineering task.

[0076] After the system converts the format of various types of context data for the software engineering task, the machine learning model uses the converted context data to predict completion of the software engineering task (block 416). The system uses the converted data to predict source code for the software engineering task. By way of example, and not limitation, this may include the machine learning model 242 using the converted context data to predict a first set of source code changes with 95% prediction confidence for the software engineering task of correcting body validation error responses and removing variable names when they are not embedded, and a second set of source code changes with 90% prediction confidence for the software engineering task of correcting body validation error responses and removing variable names when they are not embedded. The converted context data may be information about the circumstances that form the setting for an action, converted from one format to another format that is more efficient for transport or processing.

[0077] Predicting zero completions for a software engineering task may indicate that the machine learning model is not useful for the software engineering task or that there is a lack of predicted completions for the software engineering task with a predicted confidence above a threshold. For example, the machine learning model 242 may predict no completions for a software engineering task that modifies a body validation error response and removes variable names when they are not embedded if none of the initially determined completions have a predicted confidence above a 50% threshold.

[0078] A lack can be an absence. A predicted completion can be a prediction of an action to end an action, activity, or process. A prediction confidence can be a probability that a prediction is accurate. A threshold can be a signal strength or value that will result in a certain response or a particular effect.

[0079] Predicting multiple completions of the software engineering task may indicate that the multiple predicted completions of the software engineering task have corresponding prediction confidences above a threshold, and each of the multiple completions of the software engineering task having a prediction confidence above the threshold may be interpreted as a reasonable alternative.

[0080] Following the prediction of the completion of the software engineering task, the system enables the software developer to complete the software engineering task by outputting the predicted completion of the software engineering task to a code editor or issue tracker associated with the software developer (block 418). The system outputs the predicted completion of the software engineering task. In an embodiment, this may include the agent 244 outputting two different sets of predicted completions of the software engineering task within or alongside the issue tracker 216 that correct body validation error responses and remove variable names if they are not embedded, each predicted completion depicted in difference form, which is a compact representation of the possibly many source code changes that the machine learning model 242 analyzed as being capable of completing the software engineering task and has a prediction confidence above a threshold.

[0081] In addition to enabling Sofia to complete the software engineering task, the machine learning model 242 can provide multiple alternative sets of predicted source code changes with a sufficiently high prediction confidence to be reasonable alternatives for completing the software engineering task, from which Sofia can select one of the sets of predicted source code changes to be committed. Because the set of predicted source code changes is a low-level implementation of the completion of the software engineering task and the agent 244 is outputting the predicted source code changes to Sofia's high-level issue tracker 216, the predicted source code changes can be depicted as a compact representation of all of the sets of predicted source code changes for each predicted completion of the software engineering task. As Sofia uses its high-level issue tracker 216 to review all compact representations of the set of predicted source code changes for every predicted completion of the software engineering task, by selecting an individual representation, agent 244 expands the selected representation and provides Sofia with a complete section of the selected source code change within the context of the existing source code, such as the predicted source code change at line 651 of the lines of source code 108 depicted by FIG. 1A.

[0082] After predicting source code changes for multiple completions of the software engineering task, the machine learning model optionally predicts other source code changes for another completion of the software engineering task based on modifications to issue reports describing the software engineering task and / or source code changes associated with the software engineering task received from an issue tracker and / or code editor associated with the software developer (block 420). For example, without limitation, this may include: in response to Sofia modifying an issue report for a software engineering task that modifies a body validation error response and removes variable names where they are not embedded, the machine learning model 242 specifying that Sofia completed source code changes to remove variable names where they are not embedded by selecting a predicted completion of the software engineering task, and using such modifications as context data for predicting other source code changes for another completion of the software engineering task that only modifies the body validation error response without removing embedded variable names. The modifications may be mutations.

[0083] After optionally predicting other source code changes for another completion of the software engineering task, the system optionally outputs the other predicted source code changes for another completion of the software engineering task to a source code editor and / or issue tracker associated with the software developer (block 422). The system responds to the software engineering task issue report and / or source code modifications by outputting new predictions for the software engineering task. For example, without limitation, this may include agent 244 outputting to Sofia the other predicted source code changes for another completion of the software engineering task that only modify body validation error responses without removing embedded variable names.

[0084] Following the output of all predicted source code changes, the system optionally commits source code changes based on the predicted source code changes associated with one of the software engineering task completions to the source code associated with the software engineering task as accepted by a code editor and / or issue tracker associated with the software developer (block 424). The system commits the predicted source code as accepted by the software developer. By way of example, and not limitation, this may include agent 244 committing the predicted source code changes for the completions accepted by Sofia to the software engineering task's source code control repository and closing the software engineering task on Sofia's behalf. The predicted source code changes may be predictions of modifications to a text list of commands to be compiled or assembled into an executable computer program.

[0085] The source code changes may include source code to be added to, removed from, and / or replaced in the source code associated with the software engineering task. In addition to enabling Sofia to complete the software engineering task, the machine learning model 242 may provide multiple alternative sets of predicted source code changes with sufficiently high prediction confidence to be reasonable alternatives for completing the software engineering task, from which Sofia can select one of the sets of predicted source code changes to be committed.

[0086] 4 depicts blocks 402-424 occurring in a particular order, blocks 402-424 may occur in another order. In other implementations, each of blocks 402-424 may be performed in combination with other blocks and / or some blocks may be divided into different sets of blocks.

[0087] Agent 244, through one or more software tools, presents predictions or suggestions from machine learning model 242. The software developer's user experience need not be fully specified within this general architecture, and its ideal form may depend on many factors, including, but not limited to, the software developer's individual preferences, the particular type of task being performed within this general framework, or the effectiveness and / or accuracy of a particular instantiation of machine learning model 242.

[0088] In its general form, the machine learning model 242 can read many types of software engineering data and make predictions about their ideal form. While most instantiations focus on making predictions about what source code should be written, thereby automating the work of software developers, some instantiations can work directly with natural language issue reports and / or project requirements and / or specifications. The work of software developers is greatly facilitated by clear and precise natural language issue reports and / or project requirements and / or specifications. A natural language-focused instantiation of the tool would be presented directly within an issue tracking system, such as a writing assistant 218 and / or 226, which could assist project stakeholders in this process.

[0089] 5 is a flowchart illustrating a method for assisting in writing an issue report describing a software engineering task and issue information, according to one embodiment. Flowchart 500 depicts method acts illustrated as flowchart blocks of particular steps involved within and / or between clients 202-206 and / or servers 206-208 of FIG. 2.

[0090] The system receives a request from an issue tracker associated with a stakeholder of the software engineering task to describe the software engineering task and issue information and initiate an open issue report for a software developer (block 502). The system stores the source code and issue report for the software engineering task. For example, without limitation, this may include an agent 244 receiving a request from a stakeholder named Stacy Holder using the issue tracker 224 to create a new issue report for the software engineering task that fixes a body validation error response, and beginning to fill in a text description of the work Stacy Holder wants a software developer named Sofia to perform. The open issue report may be an open text account describing the problem. The issue information may be facts identified or learned about the problem.

[0091] After initiating an open issue report describing the software engineering task, the system assigns the software engineering task to a software developer (block 504). The system provides the source code and the issue report to the software developer. By way of example, and not limitation, this may include agent 244 assigning Sofia the software engineering task of correcting a body validation error response.

[0092] After assigning the software engineering task, the system receives a request from an issue tracker associated with the stakeholder to predict completion of an open issue report describing the software engineering task and issue information (block 506). The system receives the request to predict completion of the open issue report. In an embodiment, this may include agent 244 responding to an intentional button press on issue tracker 224 by receiving Stacy's request to predict completion of the open issue report for the software engineering task that corrects the body validation error response.

[0093] In addition to receiving explicit requests to predict completion of an open issue report, agent 244 can identify implicit requests to predict completion of an open issue report. For example, Stacy launches its issue tracker 224, starts an open issue report describing a software engineering task, and then stops and waits when agent 244 informs Stacy that, if no continued editing activity occurs while a displayed timer counts down uninterrupted, the machine learning model 242 will predict completion of the open issue report based on any edits Stacy makes to the open issue report.

[0094] Upon receiving a request to predict completion of an open issue report, the system obtains context data establishing a context for the software engineering task (block 508). The system obtains the context data for the software engineering task to make a prediction regarding the software engineering task. For example, without limitation, this may include agent 244 obtaining the open issue report for the software engineering task of correcting the body validation error response and the already-existing source code for the body validation error response from dynamic data pipeline 240 as context data for the software engineering task of correcting the body validation error response.

[0095] After obtaining the context data establishing the context of the software engineering task, the system transforms the context data to be compatible with a data format used to train a machine learning model to assist with the software engineering task (block 510). The system transforms the context data to enable predictions regarding the software engineering task. By way of example, and without limitation, this may include agent 244 transforming the context data of the software engineering task that modifies body validation error responses and the source code of the software engineering task that modifies body validation error responses to be compatible with a data format used to train machine learning model 230 to predict completion of open issue reports that describe the software engineering task.

[0096] After converting the context data for the software engineering task into a compatible format used to train the machine learning model, the machine learning model uses the converted context data to predict completion of an open issue report that describes the software engineering task and issue information (block 512). The system makes predictions regarding the software engineering task. In an embodiment, this may include the machine learning model 242 using the converted context data to predict completion of an open issue report for the software engineering task that modifies the body validation error response, correcting the description of the software engineering task to require deletion of variable names when variable names are not embedded, instead of an incorrect description of the software engineering task that requires deletion of variable names when variable names are not omitted.

[0097] After using the transformed context data to predict the completion of the open issue report describing the software engineering task, the system enables the software developer to complete the software engineering task by outputting an accepted completion of the open issue report to an issue tracker associated with the software developer based on the predicted completion of the open issue report describing the software engineering task and issue information (block 514). The system enables the software developer to complete the software engineering task. For example, without limitation, this may include enabling the agent 244 to more easily and quickly complete the assigned software engineering task by outputting an expanded description provided by the predicted completion of the open issue report of the software engineering task that corrects a body validation error response. The predicted completion of the open issue report corrects the description of the software engineering task to require the deletion of a variable name when the variable name is not embedded, instead of an incorrect description of the software engineering task that requires the deletion of a variable name when the variable name is not omitted. The accepted completion may be a selected action that terminates an action, activity, or process.

[0098] Outputting an accepted completion of an open issue report may include identifying at least one of a deletion or replacement of a portion of the open issue report. For example, if agent 244 outputs a predicted completion of an open issue report for a software engineering task that corrects a body validation error response, an incorrect description of the software engineering task that erroneously stated that a variable name required deletion when the variable name was not omitted is revised with a strikethrough identifying the proposed deletion of the word “omitted” and the proposed replacement with the word “embedded.” A deletion may refer to removing data from a computer. A replacement may refer to replacing one entity with another. A part may be a portion of something, such as an object, that is combined with other parts to form a whole. In addition to the machine learning model 242 being trained to predict source code changes for software engineering tasks, the machine learning model is trained with a sufficient variety of issue reports to be able to predict completion of an open issue report that may require the deletion and / or replacement of a portion of the open issue report, such as removing / replacing the word “omitted” that was erroneously included in the software engineering task.

[0099] The accepted completion of the open issue report may be based on at least one edit to the predicted completion of the open issue report from an issue tracker associated with the stakeholder, describing the software engineering task and issue information, where the issue information may include whether the issue is reproducible, the requirements for reproducing the issue, and / or the current priority associated with the issue. For example, Stacy reviews the predicted completion of the open issue report and notices that the open issue report is still missing references to several items that are important to Sofia, such as whether the issue is reliably reproducible, the steps to reproduce the issue, and how high the issue's priority is currently. Therefore, Stacy accepts the predicted completion of the open issue report and edits the generated text to specify that the issue is reproducible, the steps necessary to reproduce the issue, and a high, but not highest, priority for the software engineering task of fixing the body validation error response.

[0100] An edit can be a change to text. An issue can be a problem. Reproducible can be the ability to be shown, done, or made again. Current priority can be a current fact or condition that is considered or treated as more important than others.

[0101] In addition to enabling the software developer to complete the software engineering task, the system optionally enables the software developer, via an issue tracker associated with the stakeholder, to clarify an open issue report describing the software engineering task and / or update the open issue report to describe a strategy for completing the software engineering task (block 516). The system enables clarification and updating of the open issue report. By way of example, and without limitation, this may include enabling agent 244 to record this clarification detail as part of the open issue report, with Sofia contacting Stacy via their respective issue trackers 216 and 224 to clarify exactly how to perform the second step required to reproduce an issue that occasionally occurs in other circumstances related to a body validation error response.

[0102] After predicting completion of the open issue report, the machine learning model optionally predicts another completion of the open issue report describing the software engineering task and issue information based on a modification of the predicted completion of the open issue report describing the software engineering task and issue information, as received from an issue tracker associated with the software developer (block 518). The system can revise the prediction of completion of the open issue report. In an embodiment, this may include the machine learning model 242 revising the predicted completion of the open issue report to require the software engineering task only to modify the body validation error response without removing embedded variable names, because Sofia modifies the open issue report to indicate that Sofia completed removing variable names if the variable names were not embedded in the body validation error response.

[0103] After optionally predicting another completion for the open issue report, the system may output another completion for the open issue report describing the software engineering task and issue information to an issue tracker associated with the software developer (block 520). The system outputs a revised prediction of completion for the open issue report. For example, without limitation, this may include the agent 244 outputting to Sofia's issue tracker 216 a new prediction of completion for the open issue report that only requires the software engineering task of correcting the body validation error response without removing embedded variable names.

[0104] 5 depicts blocks 502-520 occurring in a particular order, blocks 502-520 may occur in another order. In other implementations, each of blocks 502-520 may be performed in combination with other blocks and / or some blocks may be divided into different sets of blocks.

[0105] The most common tool instantiation of this framework is to enable software developers to automatically perform common software engineering tasks given natural language descriptions and surrounding software-related data. This software developer interface may be presented alongside or within the system to which the software developer is assigned work. For example, many software developers receive tasks through issue tracking systems, which often also serve project management duties. In this instantiation, the software developer user experience may appear as the method described in the flowchart of Figure 6.

[0106] 6 is a flowchart illustrating a method for facilitating the automation of software engineering tasks, according to one embodiment. Flowchart 600 depicts method acts illustrated as flowchart blocks for particular steps involved within and / or between clients 202-204 and / or servers 206-208 of FIG. 2.

[0107] After a system, such as an issue tracking system, assigns a software engineering task to a software developer, the software developer logs on to issue tracker 216 on client 202 and begins reading an issue report describing the software engineering task. The system receives a request from the issue tracker associated with the software developer to review the issue report describing the software engineering task (block 602). The system stores the source code and issue report for the software engineering task. For example, but not by way of limitation, this may include agent 244 receiving a request from issue tracker 216 on client 202 of software developer Sofia to review an issue report describing a software engineering task that fixes body validation error responses and removes variable names when they are not embedded.

[0108] After receiving a request from the software developer's issue tracker to review the issue report, the system outputs an issue report describing software engineering tasks to the issue tracker associated with the software developer (block 604). The system provides the source code and the issue report to the software developer. By way of example, and without limitation, this may include the agent 244 outputting an issue report to Sofia's issue tracker 216 describing software engineering tasks to fix body validation error responses and remove variable names if they are not embedded.

[0109] After outputting the issue report to the software developer, the system optionally allows the software developer to clarify the issue report describing the software engineering task via an issue tracker associated with a stakeholder of the software engineering task and / or update the issue report to describe a strategy for completing the software engineering task (block 606). The system enables updates to the issue report describing the software engineering task. In an embodiment, this may include enabling agent 244 to communicate with issue tracker 216 on client 204 of Stacy Holder, a stakeholder of the software engineering task, for Sofia to clarify an issue report for the software engineering task of correcting body validation error responses and removing variable names when they are not embedded. Sofia and Stacey agree on a high, but not the highest, priority for the software engineering task of correcting body validation error responses and removing variable names when they are not embedded.

[0110] After optionally clarifying the issue report, the system optionally saves any updates to the issue report describing the software engineering task received from the issue tracker associated with the software developer (block 608). Saves any updates to the source code and issue report of the software engineering task. For example, without limitation, this may include agent 244 saving to dynamic data pipeline 240 Sofia's updates for a high, but not highest, priority software engineering task that fixes body validation error responses and removes variable names when they are not embedded.

[0111] After outputting an issue report describing the software engineering task to the software developer's issue tracker, the system receives an implicit or explicit request from the issue tracker associated with the software developer to predict source code changes for the software engineering task (block 610). The system is asked to predict the source code for the software engineering task. By way of example, and not limitation, this may include the agent 244 receiving an explicit request from a button pressed on Sofia's issue tracker 216 for predicted source code changes for the software engineering task to correct body validation error responses and remove variable names if they are not embedded.

[0112] In addition to receiving explicit requests to predict source code changes for software engineering tasks, agent 244 can identify implicit requests to predict source code changes for software engineering tasks. For example, Sofia launches its issue tracker 216 and begins updating an issue report describing a software engineering task that fixes body validation error responses and removes variable names if they are not embedded, and then stops and waits when agent 244 notifies Sofia that if no editing activity continues while a displayed timer counts down without interruption, machine learning model 242 will predict source code changes based on all of the modifications Sofia made to the issue report describing the software engineering task that fixes body validation error responses and removes variable names if they are not embedded.

[0113] After receiving a request to predict source code changes for a software engineering task, the system obtains context data that establishes the context of the software engineering task (block 612). The system obtains the context data for the software engineering task to make accurate predictions about the software engineering task. In an embodiment, this may include agent 244 obtaining the context data that establishes the context of the software engineering task from dynamic data pipeline 240, such as a description of the software engineering task that corrects body validation error responses and removes variable names when they are not embedded, and updates to issue reports, for a high, but not highest, priority of the software engineering task that corrects body validation error responses and removes variable names when they are not embedded.

[0114] After obtaining the context data establishing the context of the software engineering task, the system converts the context data to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task (block 614). The system converts the context data of the software engineering task to be in a format used to make predictions about the software engineering task. For example, without limitation, this may include agent 244 converting a obtained issue report of the software engineering task to be compatible with a data format used to train machine learning model 230 to learn to predict source code changes for the software engineering task.

[0115] Once the system has transformed various types of context data for the software engineering task, the machine learning model uses the transformed context data to predict source code changes for the software engineering task (block 616). The system uses the transformed issue reports to predict source code changes for the software engineering task. By way of example, and not limitation, this may include the machine learning model 242 using the transformed context data to predict source code changes, such as new source code at line 651 of the lines of source code 108 depicted by FIG. 1A. The source code changes may include a) new source code to be added to the source code associated with the software engineering task, b) some existing source code to be removed from the source code associated with the software engineering task, and / or c) new source code that replaces a portion of the source code associated with the software engineering task.

[0116] After predicting the source code changes for the software engineering task, the system outputs the predicted source code changes for the software engineering task to an issue tracker associated with the software developer (block 618). The system outputs the predicted source code changes for the software engineering task. In an embodiment, this may include the agent 244 outputting the predicted source code changes for the software engineering task to Sofia's issue tracker 216, including the new source code at line 651 of the lines of source code 108 depicted by FIG. 1A.

[0117] These predicted or proposed source code changes may be presented within or alongside issue tracker 216 in the form of a differential that is a compact representation of the possibly many source code changes that machine learning model 242 has analyzed to complete the software engineering task. The predicted source code changes are a low-level implementation for completing the software engineering task, and agent 244 outputs the predicted source code changes to Sofia's high-level issue tracker 216, so the predicted source code is depicted as a compact representation of the predicted source code changes for the software engineering task. When Sofia uses its high-level issue tracker 216 to review the compact representation of the predicted source code changes for the software engineering task, selection of an individual representation causes agent 244 to expand the selected representation and provide Sofia with a complete section of the predicted source code within the context of the existing source code, such as the predicted source code change at line 651 of the lines of source code 108 depicted by FIG. 1A.

[0118] After predicting source code changes for the software engineering task, the machine learning model optionally predicts other source code changes for the software engineering task based on modifications to the issue report describing the software engineering task and / or the source code for the software engineering task received from the issue tracker associated with the software developer (block 620). The system responds to the modifications to the issue report and / or source code for the software engineering task by making a new prediction for the software engineering task. In an embodiment, this may include the machine learning model 242, in response to Sofia modifying the issue report for the software engineering task to modify the body validation error response and remove variable names where they are not embedded, specifying that Sofia has completed source code modifications to remove variable names where they are not embedded, and using such modifications as context data for predicting other source code changes for another completion of the software engineering task that only modifies the body validation error response without removing embedded variable names.

[0119] After optionally predicting other source code changes for the software engineering task, the system optionally outputs the other predicted source code changes for the software engineering task to an issue tracker associated with the software developer (block 622). The system responds to the issue report and / or source code modifications for the software engineering task by outputting a new prediction for the software engineering task. By way of example, and not limitation, this may include the agent 244 outputting other source code changes for another completion of the software engineering task that only fixes the body validation error response without removing embedded variable names.

[0120] After outputting all predicted source code changes for the software engineering task, the system commits source code changes based on the predicted source code changes to the source code associated with the software engineering task as accepted by the issue tracker (block 624). The system commits the predicted source code as accepted by the software developer. In an embodiment, this may include agent 244 committing the predicted source code changes accepted by Sofia to remove embedded variable names to the source code control repository for the software engineering task and closing the software engineering task on Sofia's behalf. Even if Sofia works only on the issue report describing Sofia's software engineering task without generating any source code changes for the software engineering task and then requests predicted source code changes for the software engineering task, machine learning model 242 can use the updated issue report to predict all source code changes required to complete the software engineering task, thereby automating the generation of source code changes.

[0121] 6 depicts blocks 602-624 occurring in a particular order, blocks 602-624 may occur in another order. In other implementations, each of blocks 602-624 may be performed in combination with other blocks and / or some blocks may be divided into different sets of blocks.

[0122] A software developer may prefer closer oversight of automated tools. A further instantiation may reside within a software developer's code editor 212, which is software like a word processor that a software developer uses to edit source code. Such tools may be implemented within the code editor 212 as plug-ins that extend the native functionality of the code editor 212. This code editor-focused tool focuses on assisting the software developer in writing the most appropriate source code to accomplish a given task.

[0123] 7 is a flowchart illustrating a method for assisting in writing code for a software engineering task, according to one embodiment. Flowchart 700 illustrates method acts illustrated as flowchart blocks for particular steps involved within and / or between clients 202-206 and / or servers 206-208 of FIG. 2.

[0124] After a system, such as an issue tracking system, assigns a software engineering task to a software developer, the software developer logs on to issue tracker 216 on client 202 and begins reading the task description. The system optionally receives a request from a code editor associated with the software developer to begin work on a location in the source code associated with the software engineering task (block 702). The system stores the source code and issue reports for the software engineering task. For example, without limitation, this may include agent 244 receiving a request from code editor 212 on client 202 of software developer Sofia to determine a location in the software source code where Sofia would like to begin work on a software engineering task that fixes body validation error responses and removes variable names when they are not embedded. The location may be a region.

[0125] After optionally receiving a request from the software developer's code editor to begin work at the source code location associated with the software engineering task, the system optionally outputs the source code at the location in the source code associated with the software engineering task to the code editor associated with the software developer (block 704). The system provides the source code and an issue report to the software developer. By way of example, and not limitation, this may include the agent 244 outputting a portion of the source code for the source code location starting at line 701 where Sofia positioned her cursor in her code editor 212.

[0126] After optionally outputting the source code of the software engineering task to the software developer's code editor, the system optionally enables the software developer to clarify an issue report describing the software engineering task via an issue tracker associated with a stakeholder of the software engineering task and / or update the issue report to describe a strategy for completing the software engineering task (block 706). The system enables updates to the issue report describing the software engineering task. In an embodiment, this may include agent 244 enabling Sofia to communicate with issue tracker 216 on client 204 of Stacy Holder, a stakeholder of the software engineering task, to fix body validation error responses and remove variable names if they are not embedded. Sofia and Stacy agree on a high, but not the highest, priority for the software engineering task to fix body validation error responses and remove variable names if they are not embedded.

[0127] After optionally clarifying the issue report describing the software engineering task, the system saves the source code changes received from the code editor associated with the software developer to a location in the source code associated with the software engineering task (block 708). The system saves any updates to the source code and issue report for the software engineering task. For example, without limitation, this may include agent 244 saving Sofia's source code changes to line 651 in the source code where Sofia was working on the same software engineering task, which are part of the source code changes needed to fix the body validation error response and remove variable names if they are not embedded.

[0128] After saving the source code changes to the location in the source code, the system receives an implicit or explicit request from the code editor for predicted source code changes at the source code location associated with the software engineering task (block 710). The system is requested to predict the source code of the software engineering task. By way of example, and not limitation, this may include the agent 244 receiving an explicit request from a button pressed on Sofia's code editor 212 for predicted source code changes for the software engineering task at line 701 in the source code where Sofia's cursor is currently located. The location in the source code may be the same or different from the source code location; for example, after Sofia saves the source code changes at one location, Sofia may stay in the same location or may move to a new location where Sofia requested the predicted source code changes.

[0129] In addition to receiving explicit requests to predict source code changes at source code locations, agent 244 can identify implicit requests to predict source code changes at source code locations. For example, Sofia uses her code editor 212 to write source code changes at various source code locations, including line 651, places her cursor on line 701 of the same source code, and then stops and waits when agent 244 informs Sofia that if no coding activity continues while a displayed timer counts down without interruption, the machine learning model 242 will predict a source code change for the software engineering task at line 701 where Sofia's cursor is located. A source code location can be a region of a textual list of commands to be compiled or assembled into an executable computer program.

[0130] After receiving a request to predict source code changes at source code locations associated with a software engineering task, the system retrieves context data that establishes the context of the software engineering task (block 712). The system retrieves the context data for the software engineering task in order to make accurate predictions about the software engineering task. In an embodiment, this may include agent 244 retrieving context data from dynamic data pipeline 240 that establishes the context of the software engineering task from dynamic data pipeline 232, such as Sofia's source code changes at various locations in the source code that are partial completion of source code changes required for the software engineering task to correct body validation error responses and remove variable names if they are not embedded.

[0131] After obtaining the context data establishing the context of the software engineering task, the system converts the context data to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task (block 714). The system converts the context data of the software engineering task to be in a format used to make predictions about the software engineering task. For example, without limitation, this may include agent 244 converting source code changes at various locations within the source code to be compatible with a data format used to train machine learning model 230 to predict source code changes for the software engineering task.

[0132] After the system transforms various types of context data for the software engineering task, the machine learning model uses the transformed context data to predict source code changes at source code locations associated with the software engineering task (block 716). The system uses the transformed data to predict source code for the software engineering task. By way of example, and not limitation, this may include the machine learning model 242 using the transformed context data to predict source code changes at line 701 of the lines of source code 108 depicted by FIG. 1A to complete a localized software engineering task of correcting a body validation error response and removing a variable name if it is not embedded. The source code changes may include a) source code added to a source code location, b) source code removed from a source code location, c) source code replaced at a source code location, and / or d) any type of source code change for another source code location. The type may be a category or classification.

[0133] After predicting the source code changes at the source code locations, the system outputs the predicted source code changes at the source code locations to a code editor (block 718). The system outputs the predicted source code changes at the source code locations. In an embodiment, this may include agent 244 outputting the predicted source code changes at line 704 of the lines of source code 110 depicted by FIG. 1B to Sofia's code editor 212, thereby allowing Sofia to complete its localized software engineering task.

[0134] After predicting and outputting source code changes at source code locations associated with the software engineering task, the machine learning model optionally predicts other source code changes at other source code locations associated with the software engineering task based on modifications to the predicted source code changes at the source code locations and / or issue reports received from a code editor and / or issue tracker associated with the software developer (block 720). The system responds to the issue reports and / or source code modifications of the software engineering task by making new predictions for the software engineering task. For example, without limitation, this may include the machine learning model 242 specifying that Sofia has completed source code modifications to remove variable names where they are not embedded in response to Sofia modifying the issue report of the software engineering task to modify the body validation error response and remove variable names where they are not embedded, and using such modifications as context data for predicting new source code changes at various source code locations associated with the software engineering task that only modify the body validation error response without removing embedded variable names.

[0135] After optionally predicting other source code changes at other source code locations associated with the software engineering task, the system optionally outputs the other predicted source code changes at other source code locations for the software engineering task to the code editor (block 722). The system responds to the issue report and / or source code modifications for the software engineering task by outputting a new prediction for the software engineering task. By way of example, and not limitation, this may include the agent 244 outputting to Sofia slightly different source code changes for the localized software engineering task that only correct body validation error responses without removing embedded variable names.

[0136] Following output of the predicted source code changes, the system commits source code changes based on any predicted source code changes at any source code locations as accepted by the code editor (block 724). The system commits the predicted source code as accepted by the software developer. In an embodiment, this may include agent 244 committing the predicted source code changes accepted by Sofia to remove the embedded variable names to the source code control repository of the software engineering task and closing the software engineering task on Sofia's behalf.

[0137] Predicting source code changes, outputting the predicted source code changes, and committing source code changes based on any predicted source code changes at the source code locations may include predicting additional source code changes, outputting the additional source code changes, and committing source code changes based on any additional source code changes at another source code location associated with the software engineering task. For example, in addition to predicting a source code change at line 704 of the lines of source code 110 depicted by FIG. 1B, machine learning model 242 also predicts a source code change at line 676 of the lines of source code 112 depicted by FIG. 1A to complete the localized software engineering task of correcting a body validation error response.

[0138] After committing the source code changes at the source code location, the system has the option to iteratively obtain context data of source code changes at other source code locations, transform the context data of source code changes at other source code locations, predict source code changes at other source code locations, output predicted source code changes at other source code locations, and commit source code changes at other source code locations until the software developer completes the software engineering task (block 726). The system enables prediction of source code changes at other locations until the software developer completes the software engineering task. For example, but not limited to, this may include agent 244 continuously repeating the above process, allowing Sofia to move to any other locations in the source code where Sofia intends to focus on completing each of the localized software engineering tasks until Sofia completes the software engineering task of correcting body verification error responses at all required source code locations. Agent 244 then commits the source code changes to the source code control repository for the software engineering task and closes the software engineering task on behalf of the software developer.

[0139] 7 depicts blocks 702-726 occurring in a particular order, blocks 702-726 may occur in another order. In other implementations, each of blocks 702-726 may be performed in combination with other blocks and / or some blocks may be divided into different sets of blocks.

[0140] The agent 244 can help a software developer find anomalies in source code immediately after the software developer writes the source code. These anomalies may be bugs or other problems, such as inefficiently written code, and finding these bugs and / or problems early can help the software developer complete software engineering tasks more effectively. Large-scale versions of the machine learning model 242 encompass a wide range of knowledge, and with sufficient data, it is rare for a software engineering task or source code base to be truly unique. As a result, smaller portions of the software developer's source code that differ from the source code predicted by the machine learning model 242 may be unique, surprising, or unexpected and therefore worthy of further review by the software developer. The software developer may discover that such anomalies are errors or inefficiently written source code.

[0141] 8 is a flowchart illustrating a method for assisting in identifying unexpected portions of source code files in a software engineering task, according to one embodiment. Flowchart 800 depicts method acts illustrated as flowchart blocks for particular steps involved within and / or between clients 202-204 and / or servers 206-208 of FIG. 2.

[0142] After a system, such as an issue tracking system, assigns a software engineering task to a software developer, the software developer logs on to issue tracker 216 on client 202 and begins reading the task description. Next, the system optionally receives a request from a code editor associated with the software developer to begin work on a source code file associated with the software engineering task (block 802). The system stores the source code and issue report for the software engineering task. For example, without limitation, this may include agent 244 receiving a request from code editor 212 on software developer Sofia's client 202 to begin writing source code for the software engineering task's source code file that corrects body validation error responses and removes variable names if they are not embedded. A source code file may be an object in a computer system for storing a text list of commands to be compiled or assembled into an executable computer program.

[0143] After optionally receiving a request for a source code file, the system optionally outputs the source code file associated with the software engineering task to a code editor associated with the software developer (block 804). The system provides the source code and issue report to the software developer. By way of example, and not limitation, this may include agent 244 sending the source code file to Sofia, which attempts to address Sofia's software engineering task of correcting body validation error responses and removing variable names if they are not embedded.

[0144] After optionally outputting the source code files of the software engineering task to the software developer's code editor, the system optionally enables the software developer to clarify an issue report describing the software engineering task via an issue tracker associated with a stakeholder of the software engineering task and / or update the issue report to describe a strategy for completing the software engineering task (block 806). The system enables updates to the issue report describing the software engineering task. In an embodiment, this may include agent 244 enabling Sofia to communicate with issue tracker 216 on client 204 of Stacy Holder, a stakeholder of the software engineering task, to fix body validation error responses and remove variable names if they are not embedded. Sofia and Stacy agree on a high, but not the highest, priority for the software engineering task to fix body validation error responses and remove variable names if they are not embedded.

[0145] After optionally clarifying the issue report describing the software engineering task, the system saves the source code changes received from the code editor associated with the software developer to the source code file associated with the software engineering task (block 808). Any updates to the software engineering task's source code and issue report are saved. For example, without limitation, this may include agent 244 saving Sofia's source code changes to the software engineering task's source code file that correct body validation error responses and remove variable names if they are not embedded.

[0146] After saving the source code changes to the source code file, the system receives an implicit or explicit request from the code editor to predict source code for the source code file (block 810). The system is asked to predict source code for a software engineering task. By way of example, and not limitation, this may include the agent 244 receiving an explicit request from a button pressed on the Sofia code editor 212 to predict source code for the source code file for a software engineering task that corrects body validation error responses and removes variable names if they are not embedded.

[0147] In addition to receiving explicit requests to predict source code for a source code file, agent 244 can identify implicit requests to predict source code for a source code file. For example, Sofia uses its code editor 212 to write source code changes to a source code file, and then stops and waits when agent 244 informs Sofia that if no coding activity continues while a displayed timer counts down uninterrupted, machine learning model 242 will predict source code for the source code file.

[0148] After receiving a request for source code predictions for source code files of a software engineering task, the system obtains context data establishing the context of the software engineering task (block 812). The system obtains the context data for the software engineering task to accurately predict for the software engineering task. In an embodiment, this may include agent 244 obtaining context data establishing the context of the software engineering task from dynamic data pipeline 240, such as Sofia's source code changes in the source code files.

[0149] After obtaining the context data establishing the context of the software engineering task, the system converts the context data to be compatible with a data format used to train a machine learning model to assist in performing the software engineering task (block 814). The system converts the context data of the software engineering task to be in a format used to make predictions about the software engineering task. For example, without limitation, this may include agent 244 converting the retrieved source code changes of source code files to be compatible with a data format used to train machine learning model 230 to learn to predict source code for source code files associated with the software engineering task.

[0150] After the system transforms various types of context data for the software engineering task, the machine learning model uses the transformed context data to predict source code for a source code file associated with the software engineering task, where a portion of the source code file corresponds to the predicted portion of the source code (block 816). The system uses the transformed data to predict the source code for the software engineering task. By way of example, and not limitation, this may include the machine learning model 242 using the transformed context data to predict source code for the source code file of the software engineering task that modifies body validation error responses and removes variable names if they are not embedded. A predicted portion of the source code, for example, a prediction of "else:# pragma: no cover," matches a portion of the source code file, such as "else:# pragma: no cover" on line 702 of the source code file. A portion may be a portion of the whole. The predicted source code may be a prediction of a text list of commands to be compiled or assembled into an executable computer program.

[0151] The predicted source code may include a) source code to be added to a source code file, b) source code to be removed from a source code file, c) source code to be replaced within a source code file, and / or d) any type of source code from any other source code file. A portion of a source code file may be a line of source code, a word of source code, and / or a single text character of source code. A line may be a horizontal string of text. A word may be a single, distinct, meaningful text element. A single text character may be a printed or written symbol or letter.

[0152] Following prediction of the source code for the source code file, the system identifies, via a code editor, each portion of the source code file that is determined to differ from the corresponding portion of the predicted source code (block 818). The system identifies locations where the predicted source code differs from the source code file. In an embodiment, this may include agent 244 using bold highlighting and underlining to render a portion of line 701 ("body", field_alias) of the lines of source code 114 depicted by FIG. 1B because this portion of line 701 differs from the corresponding portion of the predicted source code (loc) with a prediction confidence score of 95%, above a 75% probability threshold.

[0153] Each portion of the source code file determined to differ from the corresponding portion of the predicted source code is conditionally identified based on a determination of whether the corresponding portion of the predicted source code has a predicted confidence score that meets a threshold. For example, for each portion of the predicted source code that differs from the corresponding portion of the source code file and has a predicted confidence score below a 75% probability threshold, agent 244 highlights the corresponding portion of the source code file on the user interface of Sofia code editor 212 with only an underline and without bold highlighting. For portions of the predicted source code that do not differ from the corresponding portion of the source code file, agent 244 does not render anything differently than previously rendered on the user interface of Sofia code editor 212.

[0154] After identifying portions of the source code file that differ from the predicted source code portions, the machine learning model optionally predicts other source code of the source code file where the portion of the source code file corresponds to the predicted other source code portion based on modifications to the predicted source code and / or accepted portion of the issue report received from a code editor and / or issue tracker associated with the software developer (block 820). The system responds to the modification of the issue report and / or source code of the software engineering task by making a new prediction for the software engineering task. For example, without limitation, this may include the machine learning model 242 specifying that Sofia completed source code modifications to remove variable names where they are not embedded in response to Sofia modifying the body validation error response and removing variable names where they are not embedded, and using such modifications as context data for predicting other source code of the source code file that only modify the body validation error response without removing embedded variable names. The accepted portion may be part of a whole approved for a purpose.

[0155] After optionally predicting other source code for the source code file, the system optionally outputs the other predicted source code for the source code file to the code editor (block 822). The system responds to issue reports and / or source code modifications for the software engineering task by outputting new predictions for the software engineering task. By way of example, and not limitation, this may include agent 244 outputting to Sofia slightly revised source code for the software engineering task's source code file that only corrects body validation error responses without removing embedded variable names.

[0156] After outputting the predicted source code for the source code file to the code editor, the system commits any different portions of the predicted source code requested and accepted by the code editor to the source code file (block 824). The system commits the predicted source code as accepted by the software developer. In an embodiment, this may include agent 244, in response to Sofia using code editor 212, reviewing the bold-highlighted portion of line 701 (“body”, field_alias), selecting the bold-highlighted portion of line 701, thereby displaying a predicted portion of the source code (loc) that differs from the bold-highlighted portion of line 701, and choosing to accept the predicted portion of the source code to replace the bold-highlighted portion of line 701 among the lines of source code 114 depicted by FIG. 1B. Agent 244 commits Sofia's accepted portions of the predicted source code to the source code control repository for the software engineering task and closes the software engineering task correcting the body verification error response on Sofia's behalf. If Sofia's source code changes undertake to fix a body verification error response, many highlighted portions of the source code file may identify predicted source code needed to complete Sofia's software engineering task, whereas if Sofia's source code changes complete the task, fewer highlighted portions of the source code file may identify predicted source code that corrects Sofia's possible mistakes and / or makes Sofia's source code more efficient. A different portion may be a part of a whole that is distinguishable from a corresponding portion.

[0157] 8 depicts blocks 802-824 occurring in a particular order, blocks 802-824 may occur in another order. In other implementations, each of blocks 802-824 may be performed in combination with other blocks and / or some blocks may be divided into different sets of blocks.

[0158] Software engineering is a complex process that can be facilitated by tools. Typical systems include focused, vertical tools that assist, to varying degrees, in narrow parts of the process. The present system 200 encompasses prior art by learning the entire software development process end-to-end with a generalized model. An associated agent 244 and one or more associated tool instantiations use machine learning models 242 and the software developer's own engineering data to assist and automate a highly generalized characterization of software development tasks. Embodiments of this system 200 provide varying degrees of assistance and automation, most commonly by being able to perform an entire software engineering task assigned to a software developer by reading an issue report that uses natural language to describe the software engineering task.

[0159] An exemplary hardware device in which the subject matter may be implemented is described. Those skilled in the art will appreciate that the elements illustrated in Figure 9 may vary depending on the system implementation. Referring to Figure 9, an exemplary system for implementing the subject matter disclosed herein includes a hardware device 900 including a processing unit 902, memory 904, storage 906, a data input module 908, a display adapter 910, a communication interface 912, and a bus 914 coupling elements 904-912 to the processing unit 902.

[0160] The bus 914 may include any type of bus architecture. Examples include a memory bus, a peripheral bus, a local bus, etc. The processing unit 902 is an instruction execution machine, apparatus, or device and may include a microprocessor, a digital signal processor, a graphics processing unit, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. The processing unit 902 may be configured to execute program instructions stored in the memory 904 and / or storage 906 and / or received via the data input module 908.

[0161] Memory 904 may include read-only memory (ROM) 916 and random access memory (RAM) 918. Memory 904 may be configured to store program instructions and data during operation of device 900. In various embodiments, memory 904 may include any of a variety of memory technologies, such as, for example, static random access memory (SRAM) or dynamic RAM (DRAM), including variants such as dual data rate synchronous DRAM (DDR SDRAM), error correcting code synchronous DRAM (ECC SDRAM), or RAMBUS DRAM (RDRAM).

[0162] The memory 904 may also include non-volatile memory technologies such as non-volatile flash RAM (NVRAM) or ROM. It is contemplated that in some embodiments the memory 904 may include a combination of the foregoing technologies, as well as other technologies not specifically mentioned. When the subject matter is implemented in a computer system, a basic input / output system (BIOS) 920, containing the basic routines that help to transfer information between elements within the computer system, such as during start-up, is stored in the ROM 916.

[0163] Storage 906 may include flash memory data storage devices for reading from and writing to flash memory, hard disk drives for reading from and writing to hard disks, magnetic disk drives for reading from or writing to removable magnetic disks, and / or optical disk drives for reading from or writing to removable optical disks, such as CD-ROMs, DVDs, or other optical media. These drives and their associated computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the hardware device 900.

[0164] It should be noted that the methods described herein may be embodied in executable instructions stored on a computer-readable medium for use by or in connection with an instruction-executing machine, apparatus, or device, such as a computer-based or processor-containing machine, apparatus, or device. Those skilled in the art will appreciate that in some embodiments, other types of computer-readable media capable of storing data accessible by a computer may be used; for example, magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, RAM, ROM, etc. may also be used in exemplary operating environments. As used herein, a "computer-readable medium" may include one or more of any suitable medium for storing executable instructions of a computer program in one or more of electronic, magnetic, optical, and electromagnetic form, and an instruction-executing machine, system, apparatus, or device may read (or fetch) the instructions from the computer-readable medium and execute the instructions to perform the described methods. A non-exhaustive list of conventional exemplary computer-readable media includes portable computer diskettes; RAM; ROM, erasable programmable read-only memory (EPROM or flash memory); optical storage devices, including portable compact discs (CDs), portable digital video discs (DVDs), high definition DVDs (HD-DVD™), Blu-ray discs; and the like.

[0165] A number of program modules may be stored in storage 906, ROM 916, or RAM 918, including an operating system 922, one or more application programs 924, program data 926, and other program modules 928. A user can enter commands and information into the hardware device 900 through a data input module 908. The data input module 908 may include mechanisms such as a keyboard, a touch screen, a pointing device, or the like.

[0166] Other external input devices (not shown) are connected to the hardware device 900 via the external data input interface 930. By way of example, and without limitation, the external input devices may include a microphone, a joystick, a gamepad, a satellite dish, a scanner, etc. In some embodiments, the external input devices may include video or audio input devices, such as a video camera, a still camera, etc. The data input module 908 may be configured to receive input from one or more users of the device 900 and deliver such input to the processing unit 902 and / or the memory 904 via the bus 914.

[0167] A display 932 is also connected to bus 914 via display adapter 910. The display 932 may be configured to display the output of device 900 to one or more users. In some embodiments, a given device, such as a touchscreen, may function as both the data input module 908 and the display 932. An external display device may also be connected to bus 914 via an external display interface 934. Other peripheral output devices, not shown, such as speakers and printers, may be connected to hardware device 900.

[0168] Hardware device 900 can operate in a networked environment using logical connections to one or more remote nodes (not shown) via communication interface 912. A remote node may be another computer, a server, a router, a peer device, or other common network node and typically includes many or all of the elements described above with respect to hardware device 900. Communication interface 912 can interface with wireless and / or wired networks.

[0169] Examples of wireless networks include, for example, a Bluetooth network, a wireless personal area network, a wireless 802.11 local area network (LAN), and / or a wireless telephony network (e.g., a cellular, PCS, or GSM network). Examples of wired networks include, for example, a LAN, a fiber optic network, a wired personal area network, a telephony network, and / or a wide area network (WAN). Such networking environments are commonplace in intranets, the Internet, offices, enterprise-wide computer networks, and the like. In some embodiments, communication interface 912 may include logic configured to support direct memory access (DMA) transfers between memory 904 and other devices.

[0170] In a networked environment, program modules depicted relative to hardware device 900, or portions thereof, may be stored in a remote storage device, such as on a server. It will be appreciated that other hardware and / or software for establishing communications links between hardware device 900 and other devices may be used.

[0171] 9 is just one possible implementation, and that other configurations are possible. It should also be understood that the various system components (and means) defined by the claims, described below, and illustrated in the various block diagrams represent logical components configured to perform the functions described herein. For example, one or more of these system components (and means) may be realized in whole or in part by at least some of the components illustrated in the configuration of hardware device 900.

[0172] Furthermore, at least one of these components is implemented, at least in part, as electronic hardware components, and thus constitutes a machine, while other components may be implemented in software, hardware, or a combination of software and hardware. More specifically, at least one component defined by the claims is implemented, at least in part, as electronic hardware components, such as an instruction-executing machine (e.g., a processor-based or processor-including machine), and / or as a specialized circuit or circuit configuration (e.g., discrete logic gates interconnected to perform a specialized function), such as that shown in FIG. 9.

[0173] Other components may be implemented in software, hardware, or a combination of software and hardware. Moreover, some or all of these other components may be combined, some may be omitted entirely, and additional components may be added while still achieving the functionality described herein. Thus, the subject matter described herein may be embodied in many different variations, and all such variations are contemplated to be within the scope of what is claimed.

[0174] In the above description, the subject matter is described with reference to acts and symbolic representations of operations that are performed by one or more devices, unless otherwise indicated. As such, it is understood that such acts and operations, sometimes referred to as being computer-executed, include the manipulation by a processing unit of data in a structured format. This manipulation transforms the data or maintains the data in locations within the computer's memory system, thereby reconfiguring or otherwise altering the operation of the device in a manner well understood by those skilled in the art. The data structures in which the data is maintained are physical locations of memory that have particular characteristics defined by the format of the data. However, while the subject matter is described in a context, it is not intended to be limiting, as those skilled in the art will recognize that various acts and operations described below can also be implemented in hardware.

[0175] To facilitate understanding of the above-described subject matter, many aspects are described in terms of a sequence of actions. At least one of these aspects, as defined by the claims, is performed by electronic hardware components. For example, it will be recognized that various actions may be performed by specialized circuits or circuit configurations, by program instructions being executed by one or more processors, or by a combination of both. The description of any sequence of actions herein is not intended to imply that the particular order described must be followed to perform that sequence. All methods described herein may be performed in any suitable order unless otherwise indicated herein or clearly contradicted by context.

[0176] While one or more embodiments have been described in terms of specific embodiments by way of example, it is to be understood that the one or more embodiments are not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar structures that would be apparent to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar structures.

[0177] [Embodiment] (1) A system for training a machine learning model to assist in the performance of a software engineering task, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions; Including, The instructions, when executed, cause the one or more processors to: Acquire data from multiple data sources associated with a software engineering task; linking the data by linking each issue report describing one of the software engineering tasks with source code associated with the one of the software engineering tasks; converting the data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; and training the machine learning model using the transformed data to assist in performing a software engineering task by predicting source code changes associated with the software engineering task. (2) The system of embodiment 1, wherein the instructions further cause the one or more processors to iteratively acquire additional data from the plurality of data sources, link the acquired additional data, transform the additional data, and train the machine learning model using the transformed additional data. (3) The system of embodiment 2, wherein iteratively training the machine learning model using the transformed additional data includes one of initializing the machine learning model and then training the initialized machine learning model using only the transformed additional data, training the machine learning model using both the transformed additional data and the data that was previously transformed, or training the machine learning model using the transformed additional data in stages. (4) The system of embodiment 1, wherein the data is obtained from multiple data sources, including open source software projects. (5) The system of embodiment 1, wherein the data is obtained from multiple data sources, including data sources associated with multiple software engineering projects associated with a single company.

[0178] (6) The system of embodiment 1, wherein the machine learning models include one of a group including the machine learning models trained on data associated with multiple software engineering projects associated with a single company and the machine learning models trained on data associated with general software engineering knowledge, or the machine learning models trained on both data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge. (7) The system of embodiment 6, wherein training the machine learning model includes training with data associated with general software engineering knowledge and then training with data associated with the single company that is weighted lower than the data associated with general software engineering knowledge. (8) A computer-implemented method for training a machine learning model to assist in performing a software engineering task, comprising: acquiring data from a plurality of data sources associated with a software engineering task; linking the data by linking each issue report describing one of the software engineering tasks with source code associated with the one of the software engineering tasks; converting the data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; training the machine learning model using the transformed data to assist in performing a software engineering task by predicting source code changes associated with the software engineering task; and A method comprising: (9) The method of embodiment 8, wherein the computer-implemented method further includes iteratively obtaining additional data from the plurality of data sources, linking the additional data, transforming the additional data, and training the machine learning model using the transformed additional data. (10) The method of embodiment 9, wherein iteratively training the machine learning model using the transformed additional data includes one of initializing the machine learning model and then training the initialized machine learning model using only the transformed additional data, training the machine learning model using both the transformed additional data and the data that was previously transformed, or training the machine learning model using the transformed additional data in a stepwise manner.

[0179] (11) The method of embodiment 8, wherein the data is obtained from multiple data sources, including open source software projects. (12) The method of embodiment 8, wherein the data is obtained from multiple data sources, including data sources associated with multiple software engineering projects associated with a single company. (13) The method of embodiment 8, wherein the machine learning models include one of a group including the machine learning models trained with data associated with multiple software engineering projects associated with a single company and the machine learning models trained with data associated with general software engineering knowledge, or the machine learning models trained with both data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge. (14) The method of embodiment 13, wherein training the machine learning model includes training with data associated with general software engineering knowledge and then training with data associated with the single company that is weighted lower than the data associated with general software engineering knowledge. (15) A computer program product, a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: Acquire data from multiple data sources associated with a software engineering task; linking the data by linking each issue report describing one of the software engineering tasks with source code associated with the one of the software engineering tasks; converting the data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; training the machine learning model using the transformed data to assist in performing a software engineering task by predicting source code changes associated with the software engineering task; a computer program product comprising instructions for:

[0180] (16) The computer program product of claim 15, wherein the program code further includes instructions for iteratively obtaining additional data from the plurality of data sources, linking the additional data, transforming the additional data, and training the machine learning model using the transformed additional data. (17) The computer program product of embodiment 16, wherein iteratively training the machine learning model with the transformed additional data includes one of initializing the machine learning model and then training the initialized machine learning model using only the transformed additional data, training the machine learning model using both the transformed additional data and the data that was previously transformed, or training the machine learning model using the transformed additional data in a stepwise manner. (18) The computer program product of embodiment 15, wherein the data is obtained from multiple data sources, including open source software projects. (19) The computer program product of embodiment 15, wherein the data is obtained from multiple data sources, including data sources associated with multiple software engineering projects associated with a single company. (20) The machine learning models include one of a group including the machine learning models trained with data associated with multiple software engineering projects associated with a single company and the machine learning models trained with data associated with general software engineering knowledge, or the machine learning models trained with both data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge; 16. The computer program product of claim 15, wherein training the machine learning model includes training with data associated with general software engineering knowledge and then training with data associated with the single company that is weighted lower than the data associated with general software engineering knowledge.

[0181] (21) A system that uses a machine learning model to assist in the execution of a software engineering task, one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: In response to receiving a request from one of an issue tracker or a code editor associated with a software developer to begin work on a software engineering task, outputting at least one of an issue report describing the software engineering task or source code associated with the software engineering task to the issue tracker and / or the code editor; storing at least one of updates to the issue report describing the software engineering task or source code changes associated with the software engineering task received from at least one of the issue tracker or the code editor; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from one of the issue tracker or the code editor for an expected completion of the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicts a number of completions of the software engineering tasks by using the transformed context data; a system for enabling the software developer to complete the software engineering task by outputting the predicted completion of the software engineering task to at least one of the issue tracker and the code editor; (22) A computer-implemented method for using a machine learning model to assist in the performance of a software engineering task, comprising: In response to receiving a request from one of an issue tracker or a code editor associated with a software developer to begin work on a software engineering task, outputting at least one of an issue report describing the software engineering task or source code associated with the software engineering task to the issue tracker and / or the code editor; storing at least one of updates to the issue report describing the software engineering task or source code changes associated with the software engineering task received from at least one of the issue tracker or the code editor; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from one of the issue tracker or the code editor for an expected completion of the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; predicting a number of completions of the software engineering tasks by using the transformed context data with the machine learning model; outputting the predicted completion of the software engineering task to at least one of the issue tracker and the code editor to enable the software developer to complete the software engineering task; 11. A computer-implemented method comprising: (23) A computer program product, a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: In response to receiving a request from one of an issue tracker or a code editor associated with a software developer to begin work on a software engineering task, outputting at least one of an issue report describing the software engineering task or source code associated with the software engineering task to the issue tracker and / or the code editor; storing at least one of updates to the issue report describing the software engineering task or source code changes associated with the software engineering task received from at least one of the issue tracker or the code editor; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from one of the issue tracker or the code editor for an expected completion of the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicts a number of completions of the software engineering tasks by using the transformed context data; outputting the predicted completion of the software engineering task to at least one of the issue tracker and the code editor, thereby enabling the software developer to complete the software engineering task; a computer program product comprising instructions for: (24) A system for assisting in writing an issue report that describes a software engineering task and issue information, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: assigning the software engineering task to a software developer in response to receiving a request from an issue tracker associated with a stakeholder of the software engineering task describing the software engineering task and the issue information and initiating an open issue report directed to the software developer; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker for an expected completion of the open issue report describing the software engineering task and the issue information; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict the completion of the open issue report describing the software engineering task and the issue information; A system that enables a software developer to complete the software engineering task by outputting to an issue tracker associated with the software developer an accepted completion of the uncompleted issue report based on the predicted completion of the uncompleted issue report describing the software engineering task and the issue information. (25) A computer-implemented method for assisting in writing an issue report describing a software engineering task and issue information, comprising: assigning the software engineering task to a software developer in response to receiving a request from an issue tracker associated with a stakeholder of the software engineering task describing the software engineering task and the issue information and initiating an open issue report directed to the software developer; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker for an expected completion of the open issue report describing the software engineering task and the issue information; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict the completion of the open issue report describing the software engineering task and the issue information; enabling the software developer to complete the software engineering task by outputting to an issue tracker associated with the software developer an accepted completion of the open issue report based on the predicted completion of the open issue report describing the software engineering task and the issue information; 11. A computer-implemented method comprising:

[0182] (26) A computer program product, a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: assigning the software engineering task to a software developer in response to receiving a request from an issue tracker associated with a stakeholder of the software engineering task describing the software engineering task and the issue information and initiating an open issue report directed to the software developer; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker for an expected completion of the open issue report describing the software engineering task and the issue information; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict the completion of the open issue report describing the software engineering task and the issue information; enabling the software developer to complete the software engineering task by outputting to an issue tracker associated with the software developer an accepted completion of the open issue report based on the predicted completion of the open issue report describing the software engineering task and the issue information; a computer program product comprising instructions for: (27) A system for assisting in the automation of software engineering tasks, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: In response to receiving a request from an issue tracker associated with a software developer to review an issue report describing a software engineering task, outputting the issue report to the issue tracker; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker to predict source code changes for the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes for the software engineering task; outputting the predicted source code changes for the software engineering task to the issue tracker; The system commits source code changes based on any predicted source code changes as accepted by the issue tracker to source code associated with the software engineering task. (28) A computer-implemented method for assisting in the automation of software engineering tasks, comprising: In response to receiving a request from an issue tracker associated with a software developer to review the issue report describing the software engineering task, outputting the issue report to the issue tracker; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker to predict source code changes for the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; The machine learning model uses the transformed context data to predict source code changes for the software engineering task; and outputting the predicted source code changes for the software engineering task to the issue tracker; committing source code changes based on any predicted source code changes as accepted by the issue tracker to source code associated with the software engineering task; 11. A computer-implemented method comprising: (29) A computer program product, a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: In response to receiving a request from an issue tracker associated with a software developer to review an issue report describing a software engineering task, outputting the issue report to the issue tracker; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker to predict source code changes for the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes for the software engineering task; outputting the predicted source code changes for the software engineering task to the issue tracker; committing source code changes based on any predicted source code changes as accepted by the issue tracker to the source code associated with the software engineering task; a computer program product comprising instructions for: (30) A system for assisting in writing source code for a software engineering task, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: storing source code changes received from a code editor associated with a software developer in a location within the source code associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor for predicted source code changes at source code locations associated with the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes at the source code locations; outputting the predicted source code changes at the source code locations to the code editor; A system for committing source code changes based on any predicted source code changes at any source code location as accepted by the code editor.

[0183] (31) A computer-implemented method for assisting in writing source code for a software engineering task, comprising: saving source code changes received from a code editor associated with the software developer to a location within the source code associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor for predicted source code changes at source code locations associated with the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes at the source code locations; and outputting the predicted source code changes at the source code locations to the code editor; committing source code changes based on any predicted source code changes at any source code locations as accepted by the code editor; 11. A computer-implemented method comprising: (32) A computer program product, a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: storing source code changes received from a code editor associated with a software developer in a location within the source code associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor for predicted source code changes at source code locations associated with the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes at the source code locations; outputting the predicted source code changes at the source code locations to the code editor; committing source code changes based on any predicted source code changes at any source code locations as accepted by the code editor; a computer program product comprising instructions for: (33) A system for assisting in identifying unexpected portions of a source code file in a software engineering task, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: saving source code changes received from a code editor associated with the software developer to a source code file associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor to predict source code for the source code file; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicting the source code of the source code file by using the transformed context data, wherein a portion of the source code file corresponds to a portion of the predicted source code; and Identifying, via the code editor, each portion of the source code file that is determined to differ from the corresponding portion of the expected source code; committing any different portions of the predicted source code that are requested and accepted by the code editor to the source code file; The system. (34) A computer-implemented method for assisting in a software engineering task of identifying unexpected portions of a source code file, comprising: saving source code changes received from a code editor associated with the software developer to a source code file associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor to predict source code for the source code file; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicting the source code of the source code file by using the transformed context data, wherein a portion of the source code file corresponds to a portion of the predicted source code; and Identifying, via the code editor, each portion of the source code file that is determined to differ from the corresponding portion of the expected source code; committing any different portions of the predicted source code that are requested and accepted by the code editor to the source code file; 11. A computer-implemented method comprising: (35) A computer program product, a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: saving source code changes received from a code editor associated with the software developer to a source code file associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor to predict source code for the source code file; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicting the source code of the source code file by using the transformed context data, wherein a portion of the source code file corresponds to a portion of the predicted source code; and Identifying, via the code editor, each portion of the source code file that is determined to differ from the corresponding portion of the expected source code; committing any different portions of the predicted source code that are requested and accepted by the code editor to the source code file; 20. A computer program product comprising instructions for:

Claims

1. 1. A system for training a machine learning model to assist in performing a software engineering task, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions; Including, The instructions, when executed, cause the one or more processors to: Acquire data from multiple data sources associated with a software engineering task; linking the data by linking each issue report describing any one of the software engineering tasks with source code associated with the any one of the software engineering tasks; converting the data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; and training the machine learning model using the transformed data to assist in performing a software engineering task by predicting source code changes associated with the software engineering task.

2. 2. The system of claim 1, wherein the instructions further cause the one or more processors to iteratively obtain additional data from the multiple data sources, link the obtained additional data, transform the additional data, and train the machine learning model using the transformed additional data.

3. 3. The system of claim 2, wherein iteratively training the machine learning model with the transformed additional data includes one of initializing the machine learning model and then training the initialized machine learning model using only the transformed additional data, training the machine learning model using both the transformed additional data and the data that was previously transformed, or training the machine learning model using the transformed additional data in a stepwise manner.

4. The system of claim 1 , wherein the data is obtained from multiple data sources, including open source software projects.

5. The system of claim 1 , wherein the data is obtained from multiple data sources, including data sources associated with multiple software engineering projects associated with a single company.

6. 2. The system of claim 1, wherein the machine learning models include one of a group including the machine learning models trained with data associated with multiple software engineering projects associated with a single company and the machine learning models trained with data associated with general software engineering knowledge, or the machine learning models trained with both data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge.

7. 7. The system of claim 6, wherein training the machine learning model includes training with data associated with general software engineering knowledge and then training with data associated with the single company that is weighted lower than the data associated with general software engineering knowledge.

8. 1. A computer-implemented method for training a machine learning model to assist in performing a software engineering task, comprising: acquiring data from a plurality of data sources associated with a software engineering task; linking the data by linking each issue report describing one of the software engineering tasks with source code associated with the one of the software engineering tasks; converting the data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; training the machine learning model using the transformed data to assist in performing a software engineering task by predicting source code changes associated with the software engineering task; and A method comprising:

9. 10. The method of claim 8, wherein the computer-implemented method further comprises iteratively obtaining additional data from the multiple data sources, linking the additional data, transforming the additional data, and training the machine learning model with the transformed additional data.

10. 10. The method of claim 9, wherein iteratively training the machine learning model with the transformed additional data comprises one of: initializing the machine learning model and then training the initialized machine learning model using only the transformed additional data; training the machine learning model using both the transformed additional data and the data that was previously transformed; or training the machine learning model using the transformed additional data in a stepwise manner.

11. The method of claim 8 , wherein the data is obtained from multiple data sources, including open source software projects.

12. The method of claim 8 , wherein the data is obtained from multiple data sources, including data sources associated with multiple software engineering projects associated with a single enterprise.

13. 9. The method of claim 8, wherein the machine learning models include one of a group including the machine learning models trained with data associated with multiple software engineering projects associated with a single company and the machine learning models trained with data associated with general software engineering knowledge, or the machine learning models trained with both data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge.

14. 14. The method of claim 13, wherein training the machine learning model includes training with data associated with general software engineering knowledge and then training with data associated with the single company that is weighted lower than the data associated with general software engineering knowledge.

15. 1. A computer program product comprising: a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: Acquire data from multiple data sources associated with a software engineering task; linking the data by linking each issue report describing any one of the software engineering tasks with source code associated with the any one of the software engineering tasks; converting the data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; training the machine learning model using the transformed data to assist in performing a software engineering task by predicting source code changes associated with the software engineering task; a computer program product comprising instructions for:

16. 16. The computer program product of claim 15, wherein the program code further comprises instructions for iteratively obtaining additional data from the multiple data sources, linking the additional data, transforming the additional data, and training the machine learning model with the transformed additional data.

17. 17. The computer program product of claim 16, wherein iteratively training the machine learning model with the transformed additional data comprises one of: initializing the machine learning model and then training the initialized machine learning model using only the transformed additional data; training the machine learning model using both the transformed additional data and the data that was previously transformed; or training the machine learning model using the transformed additional data in a stepwise manner.

18. 16. The computer program product of claim 15, wherein the data is obtained from multiple data sources including open source software projects.

19. 16. The computer program product of claim 15, wherein the data is obtained from multiple data sources, including data sources associated with multiple software engineering projects associated with a single company.

20. the machine learning models include one of a group including the machine learning models trained with data associated with multiple software engineering projects associated with a single company and the machine learning models trained with data associated with general software engineering knowledge, or the machine learning models trained with both data associated with multiple software engineering projects associated with a single company and data associated with general software engineering knowledge; 16. The computer program product of claim 15, wherein training the machine learning model comprises training with data associated with general software engineering knowledge and then training with data associated with the single company that is weighted less than the data associated with general software engineering knowledge.

21. 1. A system that uses machine learning models to assist in performing software engineering tasks, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: In response to receiving a request from one of an issue tracker or a code editor associated with a software developer to begin work on a software engineering task, outputting at least one of an issue report describing the software engineering task or source code associated with the software engineering task to the issue tracker and / or the code editor; storing at least one of updates to the issue report describing the software engineering task or source code changes associated with the software engineering task received from at least one of the issue tracker or the code editor; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from one of the issue tracker or the code editor for an expected completion of the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicts a number of completions of the software engineering tasks by using the transformed context data; a system for enabling the software developer to complete the software engineering task by outputting the predicted completion of the software engineering task to at least one of the issue tracker and the code editor.

22. 1. A computer-implemented method for using a machine learning model to assist in performing a software engineering task, comprising: In response to receiving a request from one of an issue tracker or a code editor associated with a software developer to begin work on a software engineering task, outputting at least one of an issue report describing the software engineering task or source code associated with the software engineering task to the issue tracker and / or the code editor; storing at least one of updates to the issue report describing the software engineering task or source code changes associated with the software engineering task received from at least one of the issue tracker or the code editor; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from one of the issue tracker or the code editor for an expected completion of the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; predicting a number of completions of the software engineering tasks by using the transformed context data with the machine learning model; outputting the predicted completion of the software engineering task to at least one of the issue tracker and the code editor to enable the software developer to complete the software engineering task; 11. A computer-implemented method comprising:

23. 1. A computer program product comprising: a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: In response to receiving a request from one of an issue tracker or a code editor associated with a software developer to begin work on a software engineering task, outputting at least one of an issue report describing the software engineering task or source code associated with the software engineering task to the issue tracker and / or the code editor; storing at least one of updates to the issue report describing the software engineering task or source code changes associated with the software engineering task received from at least one of the issue tracker or the code editor; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from one of the issue tracker or the code editor for an expected completion of the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicts a number of completions of the software engineering tasks by using the transformed context data; outputting the predicted completion of the software engineering task to at least one of the issue tracker and the code editor, thereby enabling the software developer to complete the software engineering task; a computer program product comprising instructions for:

24. 1. A system for assisting in writing an issue report describing a software engineering task and issue information, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: assigning the software engineering task to a software developer in response to receiving a request from an issue tracker associated with a stakeholder of the software engineering task describing the software engineering task and the issue information and initiating an open issue report directed to the software developer; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker for an expected completion of the open issue report describing the software engineering task and the issue information; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict the completion of the open issue report describing the software engineering task and the issue information; A system that enables a software developer to complete the software engineering task by outputting to an issue tracker associated with the software developer an accepted completion of the uncompleted issue report based on the predicted completion of the uncompleted issue report describing the software engineering task and the issue information.

25. 1. A computer-implemented method for assisting in writing an issue report describing a software engineering task and issue information, comprising: assigning the software engineering task to a software developer in response to receiving a request from an issue tracker associated with a stakeholder of the software engineering task describing the software engineering task and the issue information and initiating an open issue report directed to the software developer; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker for an expected completion of the open issue report describing the software engineering task and the issue information; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model using the transformed context data to predict the completion of the open issue report describing the software engineering task and the issue information; enabling the software developer to complete the software engineering task by outputting to an issue tracker associated with the software developer an accepted completion of the open issue report based on the predicted completion of the open issue report describing the software engineering task and the issue information; 11. A computer-implemented method comprising:

26. 1. A computer program product comprising: a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: assigning the software engineering task to a software developer in response to receiving a request from an issue tracker associated with a stakeholder of the software engineering task describing the software engineering task and the issue information and initiating an open issue report directed to the software developer; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker for an expected completion of the open issue report describing the software engineering task and the issue information; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict the completion of the open issue report describing the software engineering task and the issue information; enabling the software developer to complete the software engineering task by outputting to an issue tracker associated with the software developer an accepted completion of the open issue report based on the predicted completion of the open issue report describing the software engineering task and the issue information; a computer program product comprising instructions for:

27. 1. A system for assisting in the automation of software engineering tasks, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: In response to receiving a request from an issue tracker associated with a software developer to review an issue report describing a software engineering task, outputting the issue report to the issue tracker; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker to predict source code changes for the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes for the software engineering task; outputting the predicted source code changes for the software engineering task to the issue tracker; The system commits source code changes based on any predicted source code changes as accepted by the issue tracker to the source code associated with the software engineering task.

28. 1. A computer-implemented method for assisting in the automation of software engineering tasks, comprising: In response to receiving a request from an issue tracker associated with a software developer to review the issue report describing the software engineering task, outputting the issue report to the issue tracker; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker to predict source code changes for the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; The machine learning model uses the transformed context data to predict source code changes for the software engineering task; and outputting the predicted source code changes for the software engineering task to the issue tracker; committing source code changes based on any predicted source code changes as accepted by the issue tracker to source code associated with the software engineering task; 11. A computer-implemented method comprising:

29. 1. A computer program product comprising: a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: In response to receiving a request from an issue tracker associated with a software developer to review an issue report describing a software engineering task, outputting the issue report to the issue tracker; acquiring context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the issue tracker to predict source code changes for the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes for the software engineering task; outputting the predicted source code changes for the software engineering task to the issue tracker; committing source code changes based on any predicted source code changes as accepted by the issue tracker to the source code associated with the software engineering task; a computer program product comprising instructions for:

30. 1. A system for assisting in writing source code for a software engineering task, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: storing the source code changes received from the code editor associated with the software developer in a location within the source code associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor for predicted source code changes at source code locations associated with the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes at the source code locations; outputting the predicted source code changes at the source code locations to the code editor; A system for committing source code changes based on any predicted source code changes at any source code location as accepted by the code editor.

31. 1. A computer-implemented method for assisting in writing source code for a software engineering task, comprising: saving source code changes received from a code editor associated with the software developer to a location within the source code associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor for predicted source code changes at source code locations associated with the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes at the source code locations; outputting the predicted source code changes at the source code locations to the code editor; committing source code changes based on any predicted source code changes at any source code locations as accepted by the code editor; 11. A computer-implemented method comprising:

32. 1. A computer program product comprising: a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: storing the source code changes received from the code editor associated with the software developer in a location within the source code associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor for predicted source code changes at source code locations associated with the software engineering task; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model uses the transformed context data to predict source code changes at the source code locations; outputting the predicted source code changes at the source code locations to the code editor; committing source code changes based on any predicted source code changes at any source code locations as accepted by the code editor; a computer program product comprising instructions for:

33. 1. A system for assisting in a software engineering task of identifying unexpected portions of a source code file, comprising: one or more processors; a non-transitory computer-readable medium storing a plurality of instructions, the plurality of instructions, when executed, causing the one or more processors to: saving source code changes received from a code editor associated with the software developer to a source code file associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor to predict source code for the source code file; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicting the source code of the source code file by using the transformed context data, wherein a portion of the source code file corresponds to a portion of the predicted source code; and Identifying, via the code editor, each portion of the source code file that is determined to differ from the corresponding portion of the predicted source code; committing any different portions of the predicted source code that are requested and accepted by the code editor to the source code file; The system.

34. 1. A computer-implemented method for assisting in a software engineering task of identifying unexpected portions of a source code file, comprising: saving source code changes received from a code editor associated with the software developer to a source code file associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor to predict source code for the source code file; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicting the source code of the source code file by using the transformed context data, wherein a portion of the source code file corresponds to a portion of the predicted source code; and Identifying, via the code editor, each portion of the source code file that is determined to differ from the corresponding portion of the predicted source code; committing any different portions of the predicted source code that are requested and accepted by the code editor to the source code file; 11. A computer-implemented method comprising:

35. 1. A computer program product comprising: a non-transitory computer-readable medium having computer-readable program code embodied therein for execution by one or more processors, the program code comprising: saving source code changes received from a code editor associated with the software developer to a source code file associated with the software engineering task; obtaining context data establishing a context for the software engineering task in response to receiving one of an implicit request or an explicit request from the code editor to predict source code for the source code file; converting the context data to be compatible with a data format used to train a machine learning model to assist in performing a software engineering task; the machine learning model predicting the source code of the source code file by using the transformed context data, wherein a portion of the source code file corresponds to a portion of the predicted source code; and Identifying, via the code editor, each portion of the source code file that is determined to differ from the corresponding portion of the predicted source code; committing any different portions of the predicted source code that are requested and accepted by the code editor to the source code file; 20. A computer program product comprising instructions for:

Images