Dynamic automation of pipeline artifact selection
An AI-driven system generates dependency graphs and uses machine learning to optimize software delivery pipelines by identifying and encoding changes, improving efficiency and reliability in microservice environments.
Patent Information
- Application Number
- JP2022568536
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-05-19
- Filing Date
- 2021-05-18
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-05-18
AI Technical Summary
Existing solutions struggle to efficiently identify and process changes in dynamic software delivery pipelines, particularly in environments with microservices, leading to complex and slow redeployment processes.
An AI platform that generates dependency graphs and uses machine learning to capture relationships between application and pipeline artifacts, detecting changes and optimizing the pipeline by encoding mapped impacts, thereby automating selective optimization.
This approach enhances pipeline efficiency and reliability by optimizing redeployment processes, ensuring only affected artifacts are updated, reducing complexity and deployment time.
Smart Images

Figure 0007590088000001 
Figure 0007590088000002 
Figure 0007590088000003
Abstract
Description
[Technical field]
[0001] The present invention relates to continuous integration / continuous delivery (CI / CD) pipelines for software development and operations (DevOps). More specifically, the present invention relates to automatically and selectively optimizing CI / CD pipeline artifacts based on changes identified by learning dependencies between application components and CI / CD pipeline components. [Background technology]
[0002] Machine learning (ML) is a subset of artificial intelligence (AI) that utilizes algorithms to learn from data and generate predictions based on this data. AI refers to the intelligence that maximizes the chances of success in a given topic when a machine can make informed decisions. More specifically, AI can learn from a data set to solve problems and give appropriate suggestions. Cognitive computing is a blend of computer science and cognitive science. Cognitive computing utilizes self-learning algorithms that use data minimization, visual recognition, and natural language processing to solve problems and optimize human processes.
[0003] At the heart of AI and related reasoning lies the concept of similarity. The process of understanding natural language and objects requires reasoning in terms of relationships, which can be challenging. Structures, including static and dynamic structures, define deterministic outputs or actions given deterministic inputs. More specifically, the deterministic outputs or actions are based on explicit or implicit relationships within the structures. This configuration may be satisfactory under certain circumstances and conditions. However, it should be appreciated that, especially in the context of a pipeline for delivering software, dynamic structures are inherently subject to change, and outputs or actions may change accordingly. Existing solutions for efficiently identifying changes in objects and processing contents, as well as structures, are very challenging at a practical level. Summary of the Invention
[0004] Embodiments include systems, computer program products, and methods for automatically and selectively optimizing a CI / CD pipeline by learning dependencies between application artifacts and pipeline artifacts.
[0005] In one aspect, a computer system includes a processing unit operatively coupled to a memory and an artificial intelligence (AI) platform operatively coupled to the processing unit. The AI platform supports a continuous integration and deployment (CI / CD) pipeline for software development and operations (DevOps). The AI platform includes tools in the form of a graph manager, a machine learning (ML) manager, and a monitor. The graph manager is operable to generate one or more dependency graphs based on two or more application artifacts. The ML manager is operatively coupled to the graph manager and utilizes an ML model to capture relationships between the dependency graph(s) and one or more pipeline artifacts. The monitor, shown operatively coupled herein to the ML manager, detects changes to the application artifact(s) and, in response to the detected changes, the ML manager is operable to utilize the captured relationships to identify impacts of the detected changes on the pipeline artifact(s) and, in response to the identified impacts, selectively optimize the pipeline. The optimization is the automatic encoding of mapped changes into the pipeline that correspond to the identified impacts. A processing unit executes the optimized pipeline.
[0006] In another aspect, a computer program product is provided for supporting a continuous integration and continuous deployment (CI / CD) pipeline for software development and operations (DevOps). The computer program product includes a computer readable storage medium having program code embodied therein. The program code is executable by a processor and is arranged to generate one or more dependency graphs based on two or more application artifacts. The program code utilizes a machine learning (ML) model to capture relationships between the dependency graph(s) and one or more pipeline artifacts. The program code detects changes to the application artifact(s) and, in response to the detected changes, the program code utilizes the captured relationships to identify an impact of the detected changes on the pipeline artifact(s) and, in response to the identified impact, selectively optimizes the pipeline. The optimization includes automatically encoding the mapped changes corresponding to the identified impact into the pipeline. The optimized pipeline is then subject to execution.
[0007] In yet another aspect, a method is provided for supporting a continuous integration and deployment (CI / CD) pipeline for software development and operations (DevOps). The method includes generating one or more dependency graphs based on two or more application artifacts. A machine learning (ML) model is utilized to capture relationships between the dependency graph(s) and one or more pipeline artifacts. The pipeline is selectively optimized in response to the identified impacts, including automatically encoding mapped changes into the pipeline corresponding to the identified impacts, and the optimized CI / CD pipeline is executed.
[0008] These and other features and advantages will become apparent from the following detailed description of the presently preferred embodiment(s), which take into account the accompanying drawings. [Brief description of the drawings]
[0009] References in this specification to the drawings form part of this specification. Features shown in the drawings are intended as illustrations of only some embodiments and not all embodiments, unless expressly indicated otherwise.
[0010] [Figure 1] 1 shows a schematic diagram of a computer system for supporting and enabling automatic selection and optimization of a CI / CD pipeline. [Diagram 2] FIG. 2 shows a block diagram illustrating AI platform tools and their associated application program interfaces (APIs) as shown and described in FIG. [Diagram 3] 1 shows a flowchart illustrating a process for performing impact analysis to determine relationships between application artifacts and pipeline artifacts. [Figure 4] 1 shows a flowchart illustrating a process of automatic and selective optimization of a CI / CD pipeline. [Diagram 5] FIG. 5 shows a block diagram illustrating an example of a computer system / server of a cloud-based support system for implementing the systems and processes described above with respect to FIGS. 1-4. [Figure 6] 1 shows a block diagram illustrating a cloud computing environment. [Figure 7] 1 shows a block diagram illustrating a set of functional abstraction model layers provided by a cloud computing environment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] It will be readily understood that the components of the present embodiments, as generally described and illustrated in the Figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the present apparatus, systems, methods, and computer program products as illustrated in the Figures are not intended to limit the scope of the present embodiments as claimed, but are merely representative of selected embodiments.
[0012] Throughout this specification, references to "selected embodiments," "one embodiment," or "one embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment. Thus, appearances of the phrases "selected embodiments," "in one embodiment," or "in one embodiment" in various places throughout this specification do not necessarily refer to the same embodiment.
[0013] The exemplary embodiments will be best understood by reference to the drawings, in which like parts are designated with like numerals throughout. The following description is intended to illustrate, by way of example only, certain selected embodiments of devices, systems and processes consistent with the embodiments claimed herein.
[0014] DevOps (a portmanteau of "development" and "operations") is a software development methodology that emphasizes communication, collaboration, integration, automation, and measurement of collaboration between software developers and other information technology (IT) professionals. DevOps recognizes the interdependence of software development, quality assurance, and IT operations, and aims to help organizations rapidly produce software products and services, resulting in faster development and deployment cycles while improving reliability and security. Continuous integration (CI) is a software development process in which developers integrate code to identify integration issues. More specifically, CI serves to reduce code development effort, resulting in higher quality software and predictable delivery schedules. Thus, continuous integration is a DevOps practice in which software code being developed is integrated with existing software code. Continuous delivery (CD) automates the delivery of applications to selected infrastructure environments. This ensures the automatic push of code changes to different environments such as development, testing, and production. In DevOps, CI serves as a prerequisite for the test-deploy and release phases of CD.
[0015] A continuous integration / continuous delivery (CI / CD) pipeline, hereafter referred to as a pipeline, is an automated set of processes that are utilized as part of or integrated into DevOps. A pipeline is composed of several stages. In one embodiment, the stages may include Build, Unit Test, Deploy Development, Integration Tests, Compliance Checks, and Deploy Product. Each stage may consist of one or more actions or choices. For example, a test stage may employ a simple tester, an advanced tester, or a compliance scanner, or all of these. In one embodiment, one or more stages of a pipeline may require only a selection of less than all of the available actions to avoid duplication and inefficiencies. A pipeline automates the build, test, and deployment stages.
[0016] Microservices or microservices architecture refers to a computing environment in which applications are structured as a set of modular components or services based on functional definitions, each corresponding to one functional definition, each running in its own process and communicating through lightweight mechanisms. In a microservices architecture, data is stored outside the service, and therefore the service is stateless. The services or components are usually called "atomic services". Each atomic service is a lightweight component for independently executing modular services. For example, an atomic service can receive and combine keywords, process an algorithm, or make decisions based on the results of algorithm processing. Each module supports a specific task and uses a limited interface, such as an application programming interface (API), to communicate with other services. A microservices architecture supports and enables scalability in a hybrid network.
[0017] The number of microservices used in pipelines is growing exponentially as microservices provide more benefits when split in cloud native environments. Although microservices have several advantages regarding development and deployment, they also have their own challenges. Managing microservice repositories and pipelines becomes more difficult as the number of applications increases. Pipelines are utilized to automate the deployment of microservices. Some companies operate their services with hundreds or even thousands of microservices running, so the pipelines become more robust over time and continue to grow larger. Every time there is any change such as code, configuration, variable changes, network port changes, etc., the microservices need to be redeployed. Redeployment follows the same process as the process of preparing the microservices, and with too many processes, the process becomes more and more complicated and slow. As shown and explained herein, an optimal solution is provided to support and enable the pipeline to proceed efficiently with the redeployment process while maintaining a level of reliability.
[0018] Application artifacts are known in the art as tangible by-products produced during software development. Artifacts act as resources for sharing files between stages of a pipeline or between different pipelines. Examples of application artifacts include, but are not limited to, application source code, test code, application programming interface (API) specifications, application configurations, deployment scripts, and variables.
[0019] A pipeline artifact is a by-product produced during the software development process. It may consist of project source code, dependencies, binaries or resources. Examples of pipeline artifacts include, but are not limited to, the capabilities, processes, and components available in the pipeline at that time.
[0020] Referring to FIG. 1, a computer system (100) having tools for supporting automated pipeline selection and optimization is provided. As shown, a server (110) is provided in communication with a number of computing devices (180), (182), (184), (186), (188), and (190) over a network connection (105). The server (110) is configured with a processor (112) in communication with a memory (116) over a bus (114). The server (110) is shown having an artificial intelligence (AI) platform (150) for supporting selection of pipeline artifacts based on application artifacts. More specifically, the AI platform (150) is configured with one or more tools for capturing and learning dependencies between application artifacts and pipeline artifacts, and for selectively optimizing the pipeline based on the learned dependencies. The computing devices (180), (182), (184), (186), (188), and (190) communicate with each other and with other devices or components via one or more wired and / or wireless communications links, each of which may include one or more wires, routers, switches, transmitters, receivers, etc. Other embodiments of the server (110) may be used with components, systems, subsystems, and / or devices other than those illustrated herein.
[0021] An artificial intelligence (AI) platform (150) is depicted herein as configured to receive input (102) from various sources. For example, the AI platform (150) may receive input via a network (105) and utilize a knowledge base (170), also referred to herein as a corpus or data source, to apply automated selection and encoding of one or more pipeline artifacts to a pipeline, also known as a CI / CD pipeline. As shown, the knowledge base (170) is comprised of one or more libraries, depicted herein as a first library (162), a second library (164), and a third library (166). The number of libraries depicted herein is for illustrative purposes, and in one embodiment, the knowledge base (170) may have fewer libraries. For example, in one embodiment, the knowledge base (170) may have one library or data structure for a dependency graph, a model, and corresponding feedback.
[0022] The first library (162) is a graph A (162 A ),graph B (162 B ), and graphs N (162 N ) are shown to have a number of dependency graphs, referred to herein as graphs, including the dependency graphs (162 A ), (162 B ), and (162 N) are for illustrative purposes and should not be considered limiting. Each dependency graph shows the relationships between parts of an application, and more specifically, the graph articulates how each part of an application change may affect one or more other parts of that application or related applications. The dependency graph(s) show code dependencies, configuration dependencies, API dependencies, deployment dependencies, etc., between application artifacts. For example, in one embodiment, the graph can articulate how a change detected to one application may require API testing of another application.
[0023] The second library (164) is the model A (164 A ), Model B (164 B ), Model N (164 N ) is shown to have multiple ML models, including the ML model (164 A ), (164 B ), and (164 N The number of ML models (164) is for illustrative purposes and should not be considered limiting. Each ML model captures relationships between the dependency graph and pipeline artifacts stored in the first library (162). The ML manager (154) evaluates the ML models (164) to identify the impact of the captured relationships. A ), (164 B ), and (164 N ) association rule learning is employed or applied.
[0024] A third library (166) is shown having feedback associated with the ML models. In the example shown here, each model has a corresponding feedback (166 A ), (166 B ), and (166 N ) Feedback (166 A )~(166 N) are shown and explained in detail below.
[0025] Association rule learning is a rule-based ML method for discovering relationships between variables in a database. Given a set of transactions, association rule learning predicts the occurrence of an item based on the occurrence of other items in the transactions. Association rule learning is intended to identify strong rules to be discovered in the database using some measure of the discovered relationships between variables, for example, the frequency of occurrence in the data. One example of association rule learning is the deductive algorithm. Deductive algorithms use a breadth-first search strategy that sweeps through the program code and a candidate generation function that exploits the downward closure of the support. This algorithm is an exemplary association rule learning algorithm that can be utilized by the ML model(s) shown and described below. In this specification, (164 A ), (164 B ), and (164 N The ML model(s), denoted as ML_Model_Impact_Map, efficiently creates a mapping between application artifacts and pipeline artifact(s). More specifically, the ML model(s) employ algorithms to analyze the impact between collected application artifacts and pipeline artifact(s) reflected in a dependency graph. The mapping provides dependency information on how detected changes in one or more application artifacts will impact the microservices in the pipeline. In one embodiment, the ML manager (154) analyzes the impact of the detected changes and uses the analysis to predict the outcome of application changes in the pipeline. The analysis is described below in FIG. 3. In one embodiment, the mapping is referred to as an impact mapping that captures how one or more selected application artifacts impact one or more pipeline artifacts to efficiently propose or implement changes to the pipeline.
[0026] The AI platform (150) is shown herein to have several tools that support optimizing a pipeline based on detecting or identifying changes to one or more application artifacts, learning or learned dependencies of any impacts of the change(s) to the pipeline artifacts, selectively optimizing pipeline artifacts in the pipeline based on the impacts, and running the selectively optimized pipeline. The tools of the AI platform (150) are shown herein as a graph manager (152), a machine learning (ML) manager (154), a monitor (156), and a feedback manager (158). The tools (152), (154), (156), and (158) are operatively coupled to each other, directly or indirectly, to provide the functionality described below.
[0027] The graph manager (152) generates a dependency graph that is shown in the first library (162). Details of the collection of application and pipeline artifacts and dependency graph generation are shown and described in FIG. 3. The exemplary graph (162) shown herein is A ), (162 B ), and (162 N The graph manager (152) generates one or more dependency graphs that are based on two or more collected application artifacts and represent one or more relationships between the collected application artifacts. Accordingly, the graph manager (152) generates one or more dependency graphs that represent the identified relationships that exist within the collected application artifacts.
[0028] It should be appreciated that application changes may result in or may cause redeployment of one or more microservices in the corresponding application. As shown, a monitor (156) is operatively coupled to both the graph manager (152) and the ML manager (154). The monitor (156) functions to detect changes in the application that are reflected in the corresponding application artifacts. The monitor (156) monitors for any changes in the application artifacts that would cause redeployment of one or more microservices. In response to detecting such changes, the monitor (156) interacts with the ML manager (154) to retrieve corresponding ML models (164) from a second library (164). A )~(164 N ), where the identified ML model corresponds to the detected change. The ML manager (154) utilizes the identified ML model to identify the impact of the detected change on one or more pipeline artifacts and selectively optimize the pipeline based on the impact. As each ML model captures the relationships between the dependency graph and the pipeline artifacts, the ML manager (154) identifies an appropriate model from the knowledge base (170), which uses an algorithm that, in one embodiment, is an association rule learning algorithm. Additionally, the ML manager (154) compiles the ML models (164) ... A )~(164 N) into the pipeline. In one embodiment, the encoding is automated to transparently create an optimized pipeline in response to the identified impacts. In one embodiment, the optimization of the pipeline is selective by limiting the encoding to only one or more pipeline artifacts of the pipeline that are affected or otherwise affected by the change(s) in the application artifact. The selective encoding of the pipeline supports active learning methods that further teach the ML model to recognize repeatable patterns in future processes. In one embodiment, the introduction of active learning methods improves efficiency with respect to pipeline deployment by preventing unaffected pipeline artifacts from being unnecessarily evaluated. The processing unit (112) executes the optimized pipeline with the pipeline or pipeline artifacts selectively encoded based on the identified impacts to improve pipeline efficiency and deployment time. Thus, AI and ML are employed to automatically and selectively optimize the pipeline and pipeline artifacts.
[0029] As further shown, a feedback manager (158) is operatively coupled to the monitor (156). The feedback manager (158) receives feedback (166), as referred to herein as a feedback manager, stored in a knowledge base (170). A )~(166 N ), which is stored in a knowledge base (170) and collects feedback corresponding to the detected effects. In one embodiment, the collected feedback (166), collectively referred to herein as feedback, is A )~(166 N) or feedback data is user-generated feedback. As shown and described in FIG. 4, the collected feedback may include real-time data that originates from subject matter experts (SMEs) and corresponds to the deployment or redeployment of affected pipeline artifacts. As shown herein, the feedback (166 A )~(166 N ) is the ML model(s) (164 A )~(164 N ) is operatively coupled to the ML model (164 A ) is feedback (166 A ) and the ML model (164 B ) is feedback (166 B ) and the ML model (164 N ) is feedback (166 N In one embodiment, one or more ML models (164 A )~(164 N It should be understood that the feedback manager (158) may not have to have feedback operatively coupled to it. A )~(164 N ) to further train or be trained. Thus, a feedback manager (158) utilizes the feedback associated with the corresponding ML model to further train the ML model for application in future detection of application artifact changes.
[0030] In some exemplary embodiments, the server 110 may be an IBM Watson® system available from International Business Machines, Inc., Armunk, NY, augmented by features of the exemplary embodiments described below. Graph Manager (152), ML Manager (154), Monitor (156), and Feedback Manager (158), hereafter collectively referred to as AI tools, are shown embodied or integrated within the AI platform (150) of the server (110). In one embodiment, the AI tools may be implemented in another computing system (e.g., 190) connected to the server (110) by a network (105). Wherever embodied, the AI tools function to support the automatic encoding of processing rules into the pipeline based on changes identified by learning dependencies.
[0031] The types of information processing systems that may utilize the AI platform (150) range from small handheld devices, such as handheld computers / cell phones (180), to large mainframe systems, such as mainframe computers (182). Examples of handheld computers (180) include personal digital assistants (PDAs), personal entertainment devices, such as MP4 players, portable televisions, and compact disc players. Other examples of information processing systems include pen or tablet computers (184), laptop or notebook computers (186), personal computer systems (188), and servers (190). As shown, the various information processing systems may be networked together using a computer network (105). Types of computer networks that can be used to interconnect various information handling systems include local area networks (LANs), wireless local area networks (WLANs), the Internet, the Public Switched Telephone Network (PSTN), other wireless networks, and any other network topology that can be used to interconnect information handling systems. Many information handling systems include a non-volatile data store, such as a hard drive, or non-volatile memory, or both. Some information handling systems include a non-volatile data store (190 A ) and a separate non-volatile data store (e.g., server (190)) that utilizes the non-volatile data store (182 A A mainframe computer (182) utilizing a non-volatile data store (182) may be used. A ) may be components external to the various information processing systems or internal to one of the information processing systems.
[0032] The information processing system used to support the AI platform (150) can take many forms, some of which are shown in FIG. 1. For example, the information processing system can take the form of a desktop, server, mobile, laptop, notebook, or other form factor computer or data processing system. Additionally, the information processing system can take other form factors, such as personal digital assistants (PDAs), gaming devices, ATM machines, mobile phones, communication devices, or other devices that include a processor and memory. Additionally, the information processing system can embody a north bridge / south bridge controller architecture, although it should be recognized that other architectures can be used.
[0033] An application program interface (API) is understood in the art as intermediate software between two or more applications. With respect to the AI platform (150) shown and described in FIG. 1, one or more APIs may be utilized to support one or more tools (152), (154), (156), and (158) and their associated functions. Referring to FIG. 2, a block diagram (200) is provided illustrating the tools (152), (154), (156), and (158) and their associated APIs. As shown, multiple tools are embedded within the artificial intelligence platform (205), including a graph manager (252) associated with API0 (212), an ML manager (254) associated with API1 (222), a feedback manager (256) associated with API2 (232), and a monitor (258) associated with API3 (242). Each API can be implemented in one or more languages and interface specifications.
[0034] As shown, API0 (212) is configured to support and enable the functionality indicated by the Graph Manager (252): API0 (212) provides functional support for generating one or more dependency graphs based on two or more collected application artifacts, API1 (222) provides functional support for capturing relationships between the dependency graphs and one or more collected pipeline artifacts, identifying impacts of detected application changes on one or more pipeline artifacts in the form of mappings, and encoding the mapped changes corresponding to the identified impacts into the pipeline, API2 (232) provides functional support for monitoring the application for changes, and API3 (242) provides functional support for utilizing user feedback in an active learning method to further train the ML model for future processes. As shown, each of the APIs 212, 222, 232, and 242 is operatively coupled to an API orchestrator 260, otherwise known as an integration layer, which is understood in the art to act as an abstraction layer for transparently connecting the separate APIs. In one embodiment, the functionality of the separate APIs may be combined or combined. As such, the arrangement of APIs shown herein should not be considered limiting. Thus, the functionality of the tools, as shown herein, may be embodied or supported by their respective APIs.
[0035] Referring to FIG. 3, a flow chart (300) is provided illustrating a process of performing an impact analysis to determine relationships between application artifacts and pipeline artifacts. As shown, two or more application artifacts are identified (302). Examples of application artifacts include, but are not limited to, application source code, test code, application programming interface (API) specifications, application configurations, deployment scripts, and variables. Following the identification of the application artifacts, one or more pipeline artifact(s) are identified (304). Examples of pipeline artifacts include, but are not limited to, capabilities, processes, and components. A dependency graph is generated (306) for the identified application artifacts. Each dependency graph shows the relationships between parts of the application, and more specifically, the graph clearly depicts how each part of an application change may impact one or more other parts of the application or related applications. For example, a change detected to one of the applications may trigger one or more API tests of one or more related applications. Thus, the dependency graph captures the interdependencies of application artifacts.
[0036] Following the creation of the dependency graph, an impact analysis is performed between the application artifact and one or more pipeline artifact(s) reflected in the dependency graph (308). In one embodiment, an association rule learning algorithm is utilized to perform the impact analysis. It is understood in the art that an application artifact may interact with one or more pipeline artifacts. The interaction may be a direct or indirect relationship between the application artifact and the pipeline artifact. The impact analysis captures both direct and indirect relationships. More specifically, the impact analysis indicates both direct and indirect relationships of the corresponding application artifacts, if any, with various pipeline artifact(s). For example, the mapping may provide dependency information on how a detected change to an application will impact a microservice associated with the pipeline artifact. The impact analysis indicating the relationship between the application artifact and the pipeline artifact is used to train one or more associated ML model(s) (310). Thus, the steps illustrated herein depict collecting application artifacts and collecting pipeline artifacts, creating a dependency graph that captures direct and indirect relationships of the application artifacts, and performing impact analysis to determine relationships between the application artifacts and the pipeline artifacts to train one or more ML model(s).
[0037] As shown and described in FIG. 1, the ML manager (154) manages one or more ML model(s) (164) to support and enable optimization of the pipeline. A )~(164 N), which functions to support and enable the optimization of pipelines. Referring to FIG. 4, a flow chart (400) illustrating a process for automatic and selective optimization of a pipeline is provided. As shown, the pipeline is monitored (402) to identify any changes to one or more application artifacts that will impact or cause redeployment of a microservice. A determination is made (404) utilizing one or more ML model(s) created in FIG. 3 to evaluate whether the identified change is relevant to one or more pipeline artifact(s). A negative response to the determination indicates that none of the pipeline artifact(s) are impacted by the identified change and the pipeline is maintained in its current configuration (406), and then returns to step (402) for continued monitoring. A positive response to the determination in step (404) indicates that the identified change impacts one or more pipeline artifacts and rules are created for the impacted pipeline artifacts (408). More specifically, in step (408), a rule is created that is directed to invoke a subroutine of each affected pipeline artifact. For example, the rule(s) may direct a deployment or redeployment of a particular affected pipeline artifact. In addition to creating one or more rules as shown herein, corresponding relationship information from a positive determination made in step (404) is shown stored (410) in a knowledge base as shown herein. In one embodiment, updating the rule in step (408) and storing the relationship information in step (410) may occur in parallel. Thus, a change that would cause a redeployment of one or more microservices is identified, and an evaluation is performed to determine whether the identified change impacts one or more pipeline artifact(s).
[0038] Following step (408), feedback is collected based on the created rules and applied to the corresponding ML model(s) (412). In one embodiment, the feedback collected in step (412) originates from subject matter experts (SMEs). Similarly, in one embodiment, the collected feedback may include real-time data corresponding to the deployment or redeployment of the affected pipeline artifacts. In one embodiment, the application of the feedback in step (412) is optional. The created rules are encoded into the pipeline (414). The identified relationships between the application artifacts and the pipeline artifacts associated with the created rules, also referred to as dependency patterns, are captured (416) to identify repeatable patterns in future processes, and the captured dependency patterns are stored in a knowledge base as feedback to the ML model (418). In one embodiment, data processing can be used to identify software elements related to the pipeline artifacts. One example of an identified pattern is the deployment or redeployment of one or more microservices in response to monitored or detected changes in the application artifacts. Any user feedback (420) collected and incorporated into the encoded rules is also shown stored in the knowledge base (418). Following capture of dependency patterns in step (416), the pipeline with encoded rules is deployed (422). Deployment of the pipeline with encoded rules creates an optimized pipeline in which only pipeline artifacts affected by the changes identified in step (402) have been encoded into the pipeline for redeployment. The optimized pipeline creates an efficient process for redeployment of the pipeline while maintaining a level of reliability. Following step (422), the process returns to step (402) for ongoing monitoring.The rules created are then encoded into the pipeline, creating an optimized pipeline for deployment.
[0039] Aspects of automatically and selectively optimizing pipeline artifacts based on changes identified by learning dependencies are shown and described by the tools and APIs shown in Figures 1 and 2, and the processes shown in Figures 3 and 4, respectively. Aspects of the functional tools (152), (154), (156), and (158) and their associated functionalities may be embodied in a computer system / server at a single location, or in one embodiment, may be located in a cloud-based system sharing computing resources. Referring to Figure 5, a block diagram (500) is provided illustrating an example of a computer system / server (502), hereafter referred to as a host (502), in communication with a cloud-based support system to implement the processes described above with respect to Figures 3 and 4. The host (502) may be operated using many other general purpose or special purpose computing system environments or configurations. Examples of any or all of the known computing systems, environments, or configurations that may be suitable for use by the host (502) may include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputer systems, mainframe computer systems, and file systems that include any of the above systems, devices, and equivalents thereof (e.g., distributed storage environments and distributed cloud computing environments).
[0040] The host (502) may be described in the general context of computer system executable instructions, such as program modules, executed by a computer system. Generally, program modules that perform particular tasks or implement particular abstract data types may include routines, programs, objects, components, logic, data structures, etc. The host (502) may be implemented in a distributed cloud computing environment (510) where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.
[0041] As shown in FIG. 5, the host (502) is shown in the form of a general-purpose computing device. Components of the host (502) may include, but are not limited to, a processor or processing unit (504), such as a hardware processor, a system memory (506), and a bus (508) that couples various system components, including the system memory (506), to the processing unit (504). The bus (508) may represent any one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example and not limitation, such architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. The host (502) typically includes a variety of computer system readable media, including both volatile and non-volatile media, and both removable and removable media.
[0042] The memory (506) may include computer system readable media in the form of volatile memory, such as random access memory (RAM) (530) or cache memory (532), or both. By way of example only, a storage system (534) may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown, and commonly referred to as a "hard drive"). Although not shown, a magnetic disk drive may be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive may be provided for reading from and writing to a removable, non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical medium. In such an example, each may be connected to the bus (508) by one or more data media interfaces.
[0043] By way of example and not limitation, within the memory (506) may be stored a program / utility (540) having a set (at least one) of program modules (542), as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or any combination thereof, may include an implementation of a networking environment. The program modules (542) generally perform the functions and / or methods of the embodiments to integrate and automatically select DevOps artifacts based on changes identified by learning dependencies. For example, the program modules (542) may include tools (152), (154), (156), and (158) as shown in FIG. 1 .
[0044] The host (502) may also communicate with one or more external devices (514), such as a keyboard, a pointing device, a display (524), one or more devices that allow a user to interact with the host (502), or any device that allows the host (502) to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), or combinations thereof. Such communication may occur through an input / output (I / O) interface (522). Furthermore, the host (502) may also communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or combinations thereof, through a network adapter (520). As shown, the network adapter (520) communicates with other components of the host (502) through a bus (508). In one embodiment, multiple nodes of a distributed file system (not shown) communicate with a host (502) through an I / O interface (522) or through a network adapter (520). Although not shown, it is understood that other hardware and / or software components may be used in conjunction with the host (502). Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.
[0045] In this document, the terms "computer program medium," "computer usable medium," and "computer readable medium" are used to generally refer to media such as main memory, such as RAM (530), cache (532) and storage system (534), as well as removable storage drives and hard disks installed in hard disk drives.
[0046] Computer programs (also called computer control logic) are stored in the main memory (506). Such computer programs, when executed, enable the computer system to perform the features of the present embodiments discussed herein. In particular, the computer programs, when executed, enable the processing unit (504) to perform the features of the computer system. Thus, such computer programs represent the controller of the computer system.
[0047] In one embodiment, the host (502) is a node of a cloud computing environment. As is known in the art, cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with the service provider. The cloud model can include at least five characteristics, at least three service models, and at least four deployment models. Examples of such characteristics are:
[0048] On-Demand Self-Service: Cloud consumers can automatically and unilaterally provision computing capacity, such as server time and network storage, as they need it, without the need for human interaction with the provider of the service.
[0049] Pervasive Network Access: Functionality is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin- or thick-client platforms (e.g., cell phones, laptops, and PDAs).
[0050] Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated on demand. Consumers are location independent in that they generally have no control or knowledge of the exact location of the resources provided to them, although they may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center).
[0051] Rapid Elasticity: Capabilities can be rapidly provisioned and rapidly scaled out, and rapidly released and rapidly scaled in, rapidly and elastically, sometimes automatically. To the consumer, the capabilities available for provisioning often appear unlimited, and can be purchased in any quantity at any time.
[0052] Metered Services: Cloud systems automatically control and optimize resource usage by using metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services being utilized.
[0053] The service model is as follows:
[0054] Software as a Service (SaaS): The functionality provided to the consumer is the use of the provider's applications running on a cloud infrastructure. These applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or individual application capabilities, with the expected exception of limited user-specific application configuration settings.
[0055] Platform as a Service (PaaS): The functionality offered to the consumer is the deployment onto a cloud infrastructure of applications the consumer creates or acquires, written using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does control the deployed applications and, in some cases, the configuration of the environment that hosts the applications.
[0056] Infrastructure as a Service (IaaS): The functionality provided to a consumer is to provision processing, storage, network, and other basic computing resources onto which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and possibly limited control over the selection of network components (e.g., host firewalls).
[0057] The deployment model is as follows:
[0058] Private Cloud: Cloud infrastructure is operated exclusively for an organization. It can be managed by that organization or a third party and can exist on-premise or off-premise.
[0059] Community Cloud: Cloud infrastructure is shared by several organizations to support a specific community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by those organizations or by a third party and can reside on-premise or off-premise.
[0060] Public Cloud: The cloud infrastructure is available to the general public or large industry groups and is owned by organizations that sell cloud services.
[0061] Hybrid Cloud: A cloud infrastructure is a blend of two or more clouds (private, community, or public) that remain unique entities but are tied together by standardized or proprietary technologies (e.g., cloud bursting for load balancing between clouds) that enable data and application portability.
[0062] Cloud computing environments are service-oriented and focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.
[0063] Referring now to FIG. 6, an exemplary cloud computing environment (600) is shown. As shown, the cloud computing environment (600) includes a cloud computing environment (650) having one or more cloud nodes (610) with which local computing devices used by cloud consumers can communicate. Examples of these local computing devices include, but are not limited to, a personal digital assistant (PDA) or mobile phone (654A), a desktop computer (654B), a laptop computer (654C), or an automobile computer system (654N), or combinations thereof. Individual nodes within the nodes (610) can further communicate with each other. The nodes (610) can be physically or virtually grouped in one or more networks, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud, or combinations thereof, as described above. This allows the cloud computing environment (600) to provide Infrastructure as a Service, Platform as a Service, or Software as a Service, or a combination thereof, without the need for the cloud consumer to maintain resources on a local computing device. It will be understood that the types of computing devices (654A-N) shown in Figure 6 are intended to be merely exemplary, and that the cloud computing environment (650) is capable of communicating (e.g., using a web browser) with any type of computerized device over any type of network or network addressable connection or both.
[0064] Referring now to Figure 7, there is shown a set of functional abstraction layers (700) provided by the cloud computing network of Figure 6. It should be understood in advance that the components, layers and functions shown in Figure 7 are intended to be merely illustrative, and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided: a hardware and software layer (710), a virtualization layer (720), a management layer (730), and a workload layer (740).
[0065] The hardware and software layer (710) includes hardware components and software components. Examples of hardware components include mainframes, such as IBM® zSeries® systems, servers based on RISC (Reduced Instruction Set Computer) architecture, such as IBM pSeries® systems, IBM xSeries® systems, IBM BladeCenter® systems, storage devices, and networks and networking components. Examples of software components include network application server software, such as IBM WebSphere® application server software, and database software, such as IBM DB2® database software. (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide.)
[0066] The virtualization layer (720) provides an abstraction layer that can provide the following examples of virtual entities: virtual servers, virtual storage, virtual networks including virtual private networks, virtual applications and operating systems, and virtual clients.
[0067] In one example, the management layer (730) can provide the following functions: resource provisioning, metering and pricing, user portal, service level management, and SLA planning and fulfillment. Resource provisioning provides dynamic procurement of computing and other resources utilized to perform tasks within the cloud computing environment. Metering and pricing provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources can include application software licenses. Security provides identity verification for cloud consumers and tasks, and protection for data and other resources. User portal provides access to the cloud computing environment for consumers and system administrators. Service level management provides allocation and management of cloud computing resources such that requested service tiers are met. Service Layer Agreement (SLA) planning and fulfillment provides pre-allocation and procurement of cloud computing resources for predicted future needs according to SLAs.
[0068] The Workloads tier (740) provides examples of functionality that can utilize a cloud computing environment. Examples of workloads and functionality that can be provided from this tier include, but are not limited to, mapping and navigation, software development and lifecycle management, virtual classroom instructional delivery, data analytics processing, transaction processing, and automatic and selective pipeline optimization.
[0069] Although specific embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that changes and modifications may be made based on the teachings herein without departing from the embodiments and their broader aspects. Accordingly, the appended claims are intended to encompass within their scope all such changes and modifications as are within the true spirit and scope of the embodiments. It is to be further understood that the embodiments are defined solely by the appended claims. Where a particular number of introduced claim elements is intended, such intent will be expressly recited in the claims, and in the absence of such recitation, it will be understood by those skilled in the art that no such limitation exists. In a limited example, and as an aid to understanding, the following appended claims include the use of the introductory phrases "at least one" and "one or more" to introduce claim elements. However, the use of such phrases should not be construed to mean that the introduction of a claim element by the indefinite article "a" or "an" limits any particular claim including such introduced claim element to embodiments containing only one of such elements, even if it includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an." The same applies to the use of definite articles in the claims.
[0070] The present invention may include a system, method, or computer program product, or combination thereof. Additionally, selected aspects of the present embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (such as firmware, resident software, microcode, etc.), or an embodiment including software or hardware aspects or both, all of which may be referred to generally herein as a "circuit," "module," or "system." Additionally, aspects of the present embodiments may take the form of a computer program product embodied in a computer-readable storage medium or media having computer-readable program instructions for causing a processor to execute aspects of the present embodiments. The disclosed system, method, or computer program product, or combination thereof, so embodied, is operable to improve the functionality and operation of an AI platform to automatically and selectively optimize DevOps artifacts based on changes identified by learning dependencies.
[0071] A computer readable storage medium may be any tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes the following: portable computer diskettes, hard disks, dynamic or static random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), next-generation storage devices, portable compact disk read-only memory (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the above. As used herein, a computer-readable storage medium is not to be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses through a fiber optic cable), or an electrical signal sent through a wire.
[0072] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in a computer readable storage medium in the respective computing / processing device.
[0073] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object oriented programming languages such as Smalltalk, C++, and traditional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the last scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer readable program instructions to individualize the electronic circuitry by utilizing state information of the computer readable program instructions to perform aspects of the invention.
[0074] Aspects of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.
[0075] These computer readable program instructions may be provided to a computer processor or other programmable data processing apparatus to produce a machine such that the instructions, executed by the computer processor or other programmable data processing apparatus, produce means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer program instructions may also be stored in a computer readable medium capable of instructing a computer, programmable data processing apparatus or other device, or combination thereof, to function in a particular manner such that the computer readable storage medium having instructions stored thereon comprises an article of manufacture including instructions that perform aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.
[0076] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to generate a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus, or other device, perform the functions / operations specified in one or more blocks of the flowcharts or block diagrams, or both.
[0077] The flowcharts and block diagrams in the figures illustrate the architecture, functionality and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment or part of instructions, including one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the figures. For example, depending on the functionality involved, two blocks shown in succession may actually be executed simultaneously, substantially simultaneously, in a partially or fully overlapping manner in time, or the blocks may even be executed in reverse order in some cases. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.
[0078] Although specific embodiments have been described herein for illustrative purposes, various modifications may be made without departing from the spirit and scope of the embodiments. In particular, the knowledge base may be localized, remote, or distributed across multiple systems. Accordingly, the scope of protection of the embodiments is limited only by the following claims and their equivalents.
Claims
1. a processing unit operatively coupled to the memory; an artificial intelligence (AI) platform operatively coupled to the processing unit and configured to support a continuous integration and continuous deployment (CI / CD) pipeline for software development and operations (DevOps); Equipped with The one or more tools include: a graph manager that generates one or more dependency graphs based on two or more application artifacts; a machine learning (ML) manager operatively coupled to the graph manager, the ML manager utilizing an ML model to capture relationships between the one or more dependency graphs and one or more pipeline artifacts; a monitor for detecting changes to one or more of the application artifacts; Including, In response to the detected change, the ML manager: utilizing the captured relationships to identify an impact of the detected changes on the one or more pipeline artifacts; selectively optimizing the pipeline in response to the identified impacts, the optimizing including automatically encoding mapped changes into the pipeline that correspond to the identified impacts; Do the following: the processing unit executes the optimized pipeline. Computer system.
2. 2. The computer system of claim 1, wherein the identifying an impact of the detected change on the one or more pipeline artifacts further comprises the ML manager creating a measure of the impact and using the measure to predict a result of the change to the application artifact in the pipeline.
3. 10. The computer system of claim 1, further comprising a feedback manager that captures any dependency patterns in the optimized pipeline.
4. 4. The computer system of claim 3, further comprising the feedback manager employing an active learning method to capture real-time data and real-time data feedback and to utilize the captured data to train the ML model.
5. 2. The computer system of claim 1, wherein the application artifacts include source code, test code, application programming interface (API) specifications, configurations, deployment scripts, or combinations thereof.
6. 2. The computer system of claim 1, wherein the captured relationships between the one or more dependency graphs and the one or more pipeline artifacts represent impact dependencies between the application artifacts and the one or more pipeline artifacts.
7. 2. The computer system of claim 1, wherein each of the one or more pipeline artifacts has a corresponding dependency graph.
8. generating one or more dependency graphs based on two or more application artifacts; utilizing a machine learning (ML) model to capture relationships between the one or more dependency graphs and one or more pipeline artifacts; Detecting changes to one or more of the application artifacts; In response to the detected changes, the ML model: utilizing the captured relationships to identify an impact of the detected changes on the one or more pipeline artifacts; selectively optimizing a continuous integration and continuous deployment (CI / CD) pipeline in response to the identified impacts, the optimizing including automatically encoding mapped changes into the pipeline that correspond to the identified impacts; Do the following: executing the optimized pipeline; The method includes:
9. 10. The method of claim 8, wherein identifying an impact of the detected change on the one or more pipeline artifacts further comprises the ML model creating a measure of the impact and using the measure to predict an outcome of the application artifact change in the pipeline.
10. 10. The method of claim 8, further comprising: capturing any dependency patterns of the optimized pipeline; and employing an active learning method to capture real-time data and real-time data feedback and utilize the captured data to train the ML model.
11. 10. The method of claim 8, wherein the application artifacts include source code, test code, application programming interface (API) specifications, configurations, deployment scripts, or combinations thereof.
12. 10. The method of claim 8, wherein the captured relationships between the one or more dependency graphs and the one or more pipeline artifacts represent impact dependencies between the application artifacts and the one or more pipeline artifacts.
13. The method of claim 8, wherein each of the one or more pipeline artifacts has a corresponding dependency graph.
14. A computer program comprising computer readable program instructions causing a computer to carry out a method according to any one of claims 8 to 13.
15. A computer-readable storage medium storing the computer program of claim 14.
Citation Information
Patent Citations
Congnitive development of devops pipeline
US20180364985A1
Cognitive Hierarchical Content Distribution
US20200125636A1
