Dynamic automation of pipelined job selection

Abstract

An artificial intelligence (AI) platform to support continuous integration and deployment (CI / CD) pipelines for software development and operations (DevOps). One or more dependency graphs are generated based on application artifacts. Machine learning (ML) models are utilized to capture relationships between components in the dependency graph(s) and one or more pipeline artifacts. In response to changes to the application artifacts, the captured relationships are utilized to identify impacts of the detected changes to the pipeline artifacts. The CI / CD pipeline is selectively optimized and executed based on the identified impacts to improve efficiency and deployment time of the pipeline.

Description

Background Technology

[0001] This embodiment relates to a continuous integration / continuous delivery (CI / CD) pipeline for software development and operations (DevOps). More specifically, the embodiments relate to automatically and selectively optimizing CI / CD pipeline artifacts based on identified changes by learning the dependencies between application components and CI / CD pipeline components.

[0002] Machine learning (ML) is a subset of artificial intelligence (AI) that uses algorithms to learn from data and create predictions based on that data. AI refers to the intelligence of machines when they can make decisions based on information, maximizing the chances of success in a given subject. More specifically, AI can learn from datasets to solve problems and provide relevant recommendations. Cognitive computing is a hybrid of computer science and cognitive science. Cognitive computing utilizes self-learning algorithms that use data minimization, visual recognition, and natural language processing to solve problems and optimize human processes.

[0003] At the heart of AI and related reasoning lies the concept of similarity. Understanding natural language and objects requires reasoning from a relational perspective, which can be challenging. Structures, including static and dynamic structures, define definite outputs or actions for a given, definite input. More specifically, the determined outputs or actions are based on expressions or inherent relationships within the structure. This arrangement may be satisfactory for the chosen environment and conditions. However, it should be understood that dynamic structures are inherently susceptible to change, and the outputs or actions may therefore be susceptible to change, especially in pipeline environments used to deliver software. Existing solutions for efficiently identifying objects and processing changes in content and structure are extremely difficult at the time level. Summary of the Invention

[0004] Examples include systems, computer program products, and methods for automatically and selectively optimizing CI / CD pipelines by learning the dependencies between application workpieces and pipeline workpieces.

[0005] In one aspect, a computer system has a processing unit operatively coupled to memory, and an artificial intelligence (AI) platform operatively coupled to the processing unit. The AI ​​platform supports a continuous integration and deployment (CI / CD) pipeline for software development and operations (DevOps). The AI ​​platform includes tools in the form of a graph manager, a machine learning (ML) manager, and a monitor. The graph manager is used to generate one or more dependency graphs based on two or more application artifacts. The ML manager, operatively coupled to the graph manager, utilizes ML models to capture the relationships between the dependency graph(s) and one or more pipeline artifacts. The monitor, operatively coupled to the ML manager as shown here, is used to detect changes to the application artifact(s), and in response to the detected changes, the ML manager uses the captured relationships to identify the impact of the detected changes on the pipeline artifact(s), and selectively optimizes the pipeline in response to the identified impact. Optimization involves automatically encoding the changes in the mapping corresponding to the identified impacts into the pipeline. The processing unit executes the optimized pipeline.

[0006] On the other hand, a computer program product supporting continuous integration and continuous deployment (CI / CD) pipeline software development and operations (DevOps) is provided. This computer program product includes a computer-readable storage medium containing program code. Processor-executable program code is provided to generate one or more dependency graphs based on two or more application artifacts. The program code utilizes a machine learning (ML) model to capture the relationships between the dependency graph(s) and one or more pipeline artifacts. The program code detects changes to the application artifact(s), and in response to the detected changes, it uses the captured relationships to identify the impact of the detected changes on the pipeline artifact(s), and selectively optimizes the pipeline in response to the identified impact. Optimization includes automatically encoding changes to the mappings corresponding to the identified impacts into the pipeline. The optimized pipeline is then executed.

[0007] In another aspect, a method is provided to support continuous integration and deployment (CI / CD) pipelines for software development and operations (DevOps). This method includes generating one or more dependency graphs based on two or more application artifacts. A machine learning (ML) model is used to capture the relationships between the dependency graphs and one or more pipeline artifacts. The pipeline is selectively optimized in response to identified effects, including automatically encoding mapping changes corresponding to the identified effects into the pipeline and executing the optimized CI / CD pipeline.

[0008] These and other features and advantages will become apparent from the following detailed description of the presently preferred embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0009] The accompanying drawings, which are referenced herein, form part of the specification. The features shown in the drawings are merely illustrative of some embodiments, and not of all embodiments, unless explicitly indicated otherwise.

[0010] Figure 1 A schematic diagram is depicted of a computer system that supports and enables automatic selection and optimization of the CI / CD pipeline.

[0011] Figure 2 Depicting as shown Figure 1 The diagram shows the AI ​​platform tools and their associated application programming interfaces (APIs).

[0012] Figure 3 A flowchart illustrating the process of performing an impact analysis to determine the relationship between the applied workpiece and the production line workpiece is described.

[0013] Figure 4 A flowchart illustrating the process of automatic and selective optimization of the CI / CD pipeline is presented.

[0014] Figure 5 A block diagram illustrating an example of a cloud-based supporting computer system / server is provided to achieve the above-mentioned... Figure 1-4 The system and process described.

[0015] Figure 6 A block diagram illustrating a cloud computing environment is described.

[0016] Figure 7 A block diagram is described, illustrating a set of functional abstraction model layers provided by a cloud computing environment. Detailed Implementation

[0017] It is readily understood that, as generally described and illustrated in the accompanying drawings, the components of this embodiment can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of embodiments of the apparatus, system, method, and computer program product of this embodiment, as presented in the drawings, is not intended to limit the scope of the claimed embodiments, but is merely representative of selected embodiments.

[0018] Throughout this specification, references to "selective embodiment," "one embodiment," or "embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases "selective embodiment," "in one embodiment," or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment.

[0019] The illustrated embodiments will be better understood by referring to the accompanying drawings, in which the same components are always designated by the same reference numerals. The following description is intended to be illustrative only and shows only some selected embodiments of devices, systems, and processes consistent with the embodiments claimed herein.

[0020] DevOps (a combination of "development" and "operations") is a software development approach that emphasizes communication, collaboration, integration, automation, and collaborative metrics among software developers and other information technology (IT) professionals. DevOps acknowledges the interdependencies between software development, quality assurance, and IT operations and aims to help organizations rapidly produce software products and services while improving reliability and security through faster development and deployment cycles. Continuous Integration (CI) is the software development process where developers integrate their code to identify integration issues. More specifically, CI reduces the amount of code development effort, resulting in higher-quality software and predictable delivery schedules. Therefore, continuous integration is a DevOps practice where developed software code is integrated with existing software code. Continuous Delivery (CD) automates the delivery of applications to chosen infrastructure environments. It ensures that code changes are automatically pushed to different environments such as development, testing, and production. In DevOps, CI serves as a prerequisite for the test deployment and release phases of CD.

[0021] A Continuous Integration / Continuous Delivery (CI / CD) pipeline, hereinafter referred to as a pipeline, is a set of automated processes used as part of or integrated into DevOps. A pipeline consists of several stages. In one embodiment, these stages may include build, unit testing, deployment development, integration testing, compliance checks, and deployment of the product. Each stage includes one or more actions or options. For example, the testing stage may employ a simple tester, an advanced tester, and / or a compliance scanner. In one embodiment, one or more stages of the pipeline may only require selecting fewer actions than all available actions to avoid redundancy and inefficiency. Pipelines automate the build, testing, and deployment stages.

[0022] Microservices, or microservice architecture, refers to a computing environment where applications are built as a set of modular components or services based on functional definitions. Each component or service corresponds to a functional definition and runs in its own process, communicating through lightweight mechanisms. In microservice architecture, data is stored outside of services, and therefore services are stateless. Services or components are often referred to as "atomic services." Each atomic service is a lightweight component used to independently execute a modular service. For example, an atomic service might receive and combine keywords, process algorithms, or make decisions based on the results of algorithmic processing. Each module supports a specific task and uses defined interfaces such as Application Programming Interfaces (APIs) to communicate with other services. Microservice architecture supports and enables scalability in hybrid networks.

[0023] The number of microservices used in pipelines is growing exponentially because microservices offer more benefits when partitioned in a cloud-native environment. Microservices offer several advantages in development and deployment, but they also come with their own challenges. As the number of applications grows, managing microservice repositories and pipelines becomes more difficult. Pipelines are used to automate the deployment of microservices. As some companies run hundreds or even thousands of microservices to run their services, pipelines become robust over time and continue to grow larger. Whenever there are any changes in code, configuration, variables, network ports, etc., microservices need to be redeployed. Redeployment follows the same process to serve microservices, and this process becomes more complex and slow due to the sheer number of processes involved. As shown and described in this article, an optimized solution is provided to support and enable pipelines to efficiently accelerate the redeployment process while maintaining a certain level of reliability.

[0024] Application artifacts are referred to in the art as tangible byproducts generated during software development. Artifacts serve as resources for sharing files between stages in a pipeline or between different pipelines. Examples of application artifacts include, but are not limited to, application source code, test code, application programming interface (API) specifications, application configurations, deployment scripts, and variables.

[0025] Pipeline artifacts are byproducts generated during the software development process. They can consist of project source code, dependencies, and binaries or resources. Examples of pipeline artifacts include, but are not limited to, capabilities, processes, and components currently available in the pipeline.

[0026] refer to Figure 1A schematic diagram of the computer system (100) is provided, which includes tools to support the automatic selection and optimization of the pipeline. As shown, a server (110) is provided that communicates with multiple computing devices (180), (182), (184), (186), (188), and (190) via a network connection (105). The server (110) is configured with a processor (112) that communicates with a memory (116) via a bus (114). The server (110) is shown to have an artificial intelligence (AI) platform (150) to support the selection of pipeline artifacts based on application artifacts. More specifically, the AI ​​platform (150) is configured to have one or more tools for capturing and learning dependencies between application artifacts and pipeline artifacts, and for selectively optimizing the pipeline based on the learned dependencies. The computing devices (180), (182), (184), (186), (188), and (190) communicate with each other and with other devices or components via one or more wired and / or wireless data communication links, wherein each communication link may include one or more of the following: wires, routers, switches, transmitters, receivers, etc. Other embodiments of the server (110) may be used with components, systems, subsystems, and / or devices other than those described herein.

[0027] This document illustrates an artificial intelligence (AI) platform (150) configured to receive input (102) from various sources. For example, the AI ​​platform (150) can receive input via a network (105) and utilize a knowledge base (170) (also referred to herein as a corpus or data source) to apply the automatic selection and encoding of one or more pipeline artifacts to a pipeline (also referred to herein as a CI / CD pipeline). As shown, the knowledge base (170) is configured with one or more libraries, shown herein as a first library (162), a second library (164), and a third library (166). The number of libraries shown herein is for illustrative purposes, and in one embodiment, the knowledge base (170) may have a smaller number of libraries. For example, in one embodiment, the knowledge base (170) may have a single library or data structure for dependency graphs, models, and corresponding feedback.

[0028] The first library (162) is shown as having multiple dependency graphs, referred to in this paper as graphs, including graphs A (162) A ),picture B (162) B ) and diagram N (162) N Dependency graph (162) A ), (162) B ) and (162 NThe number of dependencies is for illustrative purposes and should not be considered restrictive. Each dependency graph depicts the relationships between parts of the application, and more specifically, clarifies how changes to each part of the application can affect the application or one or more other parts of related applications. Dependency graphs (multiple) indicate code dependencies, configuration dependencies, API dependencies, deployment dependencies, etc., between application artifacts. For example, in one embodiment, the graph may clearly illustrate how changes detected in one application can require API testing in another application.

[0029] The second library (164) is shown to have multiple ML models, including models A (164) A ),Model B (164) B ) and model N (164) N ). ML model (164) A (164) B ) and (164 N The number of ( ) is for illustrative purposes and should not be considered restrictive. Each ML model captures the relationship between the dependency graph stored in the first library (162) and the pipeline artifacts. The ML manager (154) takes or applies the dependency graph to the ML model (164). A (164) B ) and (164 N Association rules are learned to identify the impact of the captured relationships.

[0030] The third library (166) is shown to have feedback associated with the ML model. In the example shown in this paper, each model has a corresponding feedback (166). A ), (166) B ) and (166 N Feedback (166) A )-(166) N The details are shown and described below.

[0031] Association rule learning is a rule-based machine learning method used to discover relationships between variables in a database. Given a set of transactions, association rule learning predicts the occurrence of an item based on the occurrence of other items in the transaction. Association rule learning aims to identify strong rules found in a database using some measure of the relationships discovered between variables, such as the frequency of occurrence in the data. An example of association rule learning is the Apriori algorithm. The Apriori algorithm uses a breadth-first search strategy to scan program code and uses candidate generating functions to leverage supporting downward closure properties. This algorithm is an example association rule learning algorithm that can be utilized by the ML model shown and described below. The ML model (shown herein as (164) A), (164) B ) and (164 N This effectively creates a mapping between application artifacts and pipeline artifacts(s). More specifically, the ML model employs algorithms to analyze the impact between the collected application artifacts and pipeline artifacts reflected in the dependency graph. This mapping provides information on how detected changes to one or more application artifacts will affect dependencies in the pipeline of microservices. In one embodiment, the ML manager (154) analyzes the impact of the detected changes and uses this analysis to predict the consequences of application changes in the pipeline. This analysis is described in the following... Figure 3 As described, in one embodiment, the mapping is referred to as an influence mapping, used to capture how changes to one or more selected application artifacts will affect one or more pipeline artifacts, and to effectively suggest or implement changes to the pipeline.

[0032] The AI ​​platform (150) is shown herein as having several tools to support pipeline optimization based on the detection or identification of changes to one or more application artifacts, dependence on any effect of learned or acquired changes(s) on the pipeline artifacts, selective optimization of the pipeline artifacts in the pipeline based on the effects, and execution of the selectively optimized pipeline. The AI ​​platform (150) tools are shown herein as a graph manager (152), a machine learning (ML) manager (154), a monitor (156), and a feedback manager (158). Tools (152), (154), (156), and (158) are directly or indirectly operationally connected and provide the functionality described below.

[0033] The graph manager (152) generates the dependency graph shown in the first library (162). Figure 3 The application and pipeline workpiece collection and dependency graph generation details are shown and described here, with example graph (162) shown. A ), (162) B ) and (162 N Based on two or more collected application artifacts, and expressing one or more relationships between the collected application artifacts. Therefore, the graph manager (152) generates one or more dependency graphs to express the relationships presented and identified in the collected application artifacts.

[0034] It should be understood that application changes can lead to or cause the redeployment of one or more microservices within the corresponding application. As shown in the figure, the monitor (156) is operationally coupled to both the graph manager (152) and the ML manager (154). The monitor (156) is used to detect changes in the application graph as reflected in the corresponding application artifacts. The monitor (156) monitors for any changes in the application artifacts that trigger the redeployment of one or more microservices. In response to detecting such a change, the monitor (156) interfaces with the ML manager (154) to identify the corresponding ML model (164) from the second library (164). A )-(164 N The identified ML model corresponds to the detected change. The ML manager (154) uses the identified ML model to identify the impact of the detected change on one or more pipeline artifacts and selectively optimizes the pipeline based on this impact. Since each ML model captures the relationship between the dependency graph and the pipeline artifacts, the ML manager (154) identifies the appropriate model from the knowledge base (170), and the model employs the algorithm, in one embodiment, an association rule learning algorithm. Furthermore, the ML manager (154) will interact with the ML model (164) A )-(164 N Any changes in the mapping corresponding to the impact of the identified effects are encoded into the pipeline. In one embodiment, the encoding is automatic to transparently create an optimized pipeline in response to the identified effects. In one embodiment, pipeline optimization is selective by limiting the encoding to only one or more pipeline artifacts that are impacted or otherwise affected by application artifact changes. Selective encoding of the pipeline supports active learning methods to further teach the ML model to identify repeatable patterns in future processes. In one embodiment, the active learning method improves the efficiency of pipeline deployment because unaffected pipeline artifacts are not unnecessarily evaluated. The processing unit (112) performs an optimized pipeline based on the identified effects using selectively encoded pipelines or pipeline artifacts to improve pipeline efficiency and deployment time. Thus, AI and ML are used to automatically and selectively optimize pipelines and pipeline artifacts.

[0035] As further illustrated, the feedback manager (158) is operatively coupled to the monitor (156). The feedback manager (158) collects feedback, shown here as feedback (166) stored in the knowledge base (170). A )-(166) N ), corresponding to the detected impact. In one embodiment, the collected feedback (166 A )-(166)N This refers to user-generated feedback, or feedback data. Figure 4 As shown and described, the collected feedback originates from subject matter experts (SMEs) and may include real-time data corresponding to the deployment or redeployment of affected production line workpieces. As illustrated in this paper, the feedback (166...) A )-(166 N ) Operationally coupled to the ML model (164 A )-(164 N More specifically, ML models (164) A ) was shown to have feedback (166) A ), ML model (164) B ) was shown to have feedback (166) B ), and ML model (164 N ) was shown to have feedback (166) N It should be understood that, in one embodiment, the ML model (164) A )-(164 N One or more ML models in the model may not have operationally coupled feedback. The feedback manager (158) utilizes feedback to further train or enable the corresponding ML model (164). A )-(164 N The feedback manager (158) is trained. Therefore, the feedback manager (158) uses the feedback associated with the corresponding ML model to further train the ML model for future application when changes to the application artifact are detected.

[0036] In some illustrative embodiments, the server (110) may be IBM Watson, available from International Business Machines Corporation in Armonk, New York. ® The system is enhanced with the illustrative embodiments described below. A graph manager (152), an ML manager (154), a monitor (156), and a feedback manager (158) (collectively referred to below as AI tools) are shown as being contained within or integrated into an AI platform (150) on a server (110). In one embodiment, the AI ​​tools may be implemented in a separate computing system (e.g., 190) connected to the server (110) via a network (105). Wherever embodied, the AI ​​tools are used to support the automatic encoding of processing rules into a pipeline based on changes identified by learning dependencies.

[0037] The range of information processing systems that can utilize the AI ​​platform (150) ranges from small handheld devices such as handheld computers / mobile phones (180) to large systems such as mainframe computers (182). Examples of handheld computers (180) include personal digital assistants (PDAs), personal entertainment devices such as MP4 players, portable televisions, and CD players. Other examples of information processing systems include pen or tablet computers (184), laptop or notebook computers (186), personal computer systems (188), and servers (190). As shown, various information processing systems can be networked together using computer networks (105). Types of computer networks (105) that can be used to interconnect various information processing systems include local area networks (LANs), wireless local area networks (WLANs), the Internet, the public switched telephone network (PSTN), other wireless networks, and any other network topologies that can be used to interconnect information processing systems. Many information processing systems include non-volatile data storage, such as hard disk drives and / or non-volatile memory. Some information processing systems may use separate non-volatile data repositories (e.g., servers (190) use non-volatile data repositories (190... A ), and mainframe computers (182) use non-volatile data storage (182a). Non-volatile data storage (182) A It can be a component outside of various information processing systems, or it can be a component inside one of the information processing systems.

[0038] Information processing systems used to support AI platforms (150) can take many forms, some of which are... Figure 1 As shown, for example, an information processing system can take the form of a desktop computer, server, portable computer, laptop computer, notebook computer, or other form factor computer or data processing system. Furthermore, an information processing system can take other form factors, such as a personal digital assistant (PDA), gaming device, ATM machine, portable telephone device, communication device, or other device including a processor and memory. Additionally, an information processing system can be embodied in a Northbridge / Southbridge controller architecture, although it will be understood that other architectures may also be used.

[0039] Application Programming Interface (API) is understood in this field as a software intermediary between two or more applications. About Figure 1 The AI ​​platform (150) shown and described may have one or more APIs that can be utilized to support one or more of the tools (152), (154), (156), and (158) and their associated functionality. References Figure 2A block diagram (200) is provided showing tools (152), (154), (156), and (158) and their associated APIs. As shown, multiple tools are embedded within the AI ​​platform (205), including a graph manager (252) associated with API0 (212), an ML manager (254) associated with API1 (222), a feedback manager (256) associated with API2 (232), and a monitor (258) associated with API3 (242). Each of these APIs can be implemented using one or more languages ​​and interface specifications.

[0040] As shown in the figure, API0 (212) is configured to support and enable the functionality represented by the graph manager (252). API0 (212) provides functional support for generating one or more dependency graphs based on two or more collected application artifacts; API1 (222) provides functional support for capturing the relationships between dependency graphs and one or more collected pipeline artifacts, identifying the impact of detected application changes on one or more pipeline artifacts in the form of a mapping, and encoding the changes in the mapping corresponding to the identified impacts into the pipeline; API2 (232) provides functional support for monitoring changes in the application; and API3 (242) provides functional support for leveraging user feedback in an active learning approach to further train the ML model for use in future processes. As shown in the figure, each of APIs (212), (222), (232), and (242) is operationally coupled to the API coordinator (260), or coordination layer, which is understood in the art as serving as an abstraction layer that transparently strings the individual APIs together. In one embodiment, the functionality of the individual APIs may be combined or combined. Thus, the configuration of the APIs shown herein should not be considered limiting. Therefore, as this article shows, the functionality of a tool can be concretized or supported by its corresponding API.

[0041] refer to Figure 3A flowchart (300) is provided illustrating the process of performing impact analysis to determine the relationships between application artifacts and pipeline artifacts. As shown, two or more application artifacts are identified (302). Examples of application artifacts include, but are not limited to, application source code, test code, application programming interface (API) specifications, application configuration, deployment scripts, and variables. Following the identification of application artifacts, one or more pipeline artifacts are identified (304). Examples of pipeline artifacts include, but are not limited to, capabilities, processes, and components. A dependency graph (306) is generated for the identified application artifacts. Each dependency graph depicts the relationships between parts of the application, and more specifically, the graph clarifies how each part of the application changes can affect the application or one or more other parts of related applications. For example, a detected change to one of the applications can trigger one or more API tests of one or more related applications. Thus, the dependency graph captures the interdependencies of application artifacts.

[0042] After the dependency graph is created, an impact analysis of the application artifacts reflected in the dependency graph with one or more pipeline artifacts is performed (308). In one embodiment, an association rule learning algorithm is used to perform the impact analysis. It will be understood in the art that an application artifact may interface with one or more pipeline artifacts. This interface may be a direct or indirect relationship between the application artifact and the pipeline artifact. The impact analysis captures both direct and indirect relationships. More specifically, the impact analysis represents the direct and indirect relationships with the application artifacts corresponding to (if any) different pipeline artifacts. For example, a mapping can provide dependency information on how detected changes to the application will affect the microservices associated with the pipeline artifacts. One or more associated ML models are trained using the impact analysis that indicates the relationship between the application artifacts and the pipeline artifacts (310). Thus, the steps shown here represent the collection of application artifacts and pipeline artifacts, the creation of a dependency graph to capture the direct and indirect relationships between application artifacts, and the performance of an impact analysis to determine the relationship between the application and the pipeline artifacts to train one or more ML models.

[0043] like Figure 1 As shown and described, the ML manager (154) is used to support and enable one or more ML models (164) A )-(164 N This enables and supports pipeline optimization. (See reference) Figure 4 A flowchart (400) illustrating the process for automated and selective pipeline optimization is provided. As shown, the pipeline is monitored to identify any changes (402) to one or more application artifacts that will affect or cause a redeployment of the microservice. Utilizing... Figure 3One or more ML models created in step (404) determine whether the identified change is relevant to one or more pipeline artifacts. A negative response to the determination indicates that no pipeline artifact is affected by the identified change, and the pipeline is maintained in its current configuration (406), then returns to step (402) to continue monitoring. A positive response to the determination at step (404) indicates that the identified change affects one or more pipeline artifacts, and rules are created for the affected pipeline artifacts (408). More specifically, at step (408), rules are created for calling subroutines of each affected pipeline artifact. For example, the rules(s) may guide the deployment or redeployment of a particular affected pipeline artifact. As shown herein, in addition to creating one or more rules, this document also shows storing correspondence information from the positive determination made at step (404) in a knowledge base (410). In one embodiment, rule updates at step (408) and relation information storage at step (410) can occur in parallel. Therefore, changes that trigger the redeployment of one or more microservices are identified and evaluated to determine whether the identified changes affect one or more pipeline artifacts.

[0044] Following step (408), feedback based on the created rules is collected and applied to the corresponding ML model (412). In one embodiment, the feedback collected in step (412) comes from a subject matter expert (SME). Similarly, in one embodiment, the collected feedback may include real-time data corresponding to the deployment or redeployment of an affected pipeline artifact. In one embodiment, the application of the feedback at step (412) is optional. The created rules are encoded into the pipeline (414). Identified relationships (also known as dependency patterns) between application artifacts and pipeline artifacts associated with the created rules are captured to identify repeatable patterns in future processes (416), and the captured dependency patterns are stored in a knowledge base (418) as feedback to the ML model. In one embodiment, data processing may be used to identify software elements associated with pipeline artifacts. An example of an identified pattern is the deployment or redeployment of one or more microservices corresponding to a monitored or detected change in an application artifact. Any user feedback (420) that has been collected and incorporated into the encoded rules is also shown here as being stored in the knowledge base (418). After capturing the dependency pattern in step (416), the pipeline is deployed using coded rules (422). Pipeline deployment using coded rules creates an optimized pipeline where only pipeline artifacts affected by the changes identified in step (402) are coded into the pipeline for redeployment. The optimized pipeline creates an efficient process for redeploying the pipeline while maintaining a certain level of reliability. After step (422), the process returns to step (402) for continued monitoring. Thus, the created rules are coded into the pipeline that creates the optimized pipeline for deployment.

[0045] Regarding respectively Figure 1 and Figure 2 The tools and APIs shown are as follows Figure 3 and Figure 4 The process illustrated demonstrates and describes aspects of automatically and selectively optimizing pipeline workpieces based on learned dependencies identified in the process. Functional tools (152), (154), (156), and (158), and their functionality, can be embodied in a single-location computer system / server, or, in one embodiment, can be configured in a cloud-based system sharing computing resources. References Figure 5 A block diagram (500) is provided illustrating an example of a computer system / server (502), which is hereafter referred to as communicating with a cloud-based support system to implement the above references. Figure 3 and Figure 4The host (502) described in the process. The host (502) may operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments and / or configurations suitable for use with the host (502) include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and file systems (e.g., distributed storage environments and distributed cloud computing environments) that include any of the aforementioned systems, devices and their equivalents.

[0046] The host (502) can be described in the general context of executable instructions in a computer system, such as program modules executed by the computer system. Typically, program modules can include routines, programs, objects, components, logic, data structures, etc., that perform a specific task or implement a specific abstract data type. The host (502) can be implemented in a distributed cloud computing environment (510), where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0047] like Figure 5 As shown, the host (502) is illustrated as a general-purpose computing device. Components of the host (502) may include, but are not limited to, one or more processors or processing units (504), such as a hardware processor, system memory (506), and a bus (508) that couples various system components, including the system memory (506), to the processing unit (504). The bus (508) represents one or more of several types of bus architectures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of various bus architectures. By way of example and not limitation, such architectures include the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus. The host (502) typically includes various computer system readable media. Such media can be any available media accessible to the host (502), and it includes volatile and non-volatile media, removable and non-removable media.

[0048] The memory (506) may include computer system readable media in the form of volatile memory, such as random access memory (RAM) (530) and / or cache memory (532). By way of example only, the storage system (534) may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown, and generally referred to as a "hard disk drive"). Although not shown, a disk drive may be provided for reading from and writing to a removable, non-volatile disk (e.g., a "floppy disk"), and an optical disk drive may be provided for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical media. In this case, each may be connected to a bus (508) via one or more data media interfaces.

[0049] A program / utility (540) having a set (at least one) of program modules (542), along with an operating system, one or more applications, other program modules, and program data, may be stored in memory (506), as an example and not a limitation. Each of the operating system, one or more applications, other program modules, and program data, or some combination thereof, may include an implementation of a networked environment. The program modules (542) typically perform the functions and / or methods of the embodiment to integrate automated selection of DevOps pipeline artifacts based on changes identified by learning dependencies. For example, the set of program modules (542) may include, for example, Figure 1 The tools described in (152), (154), (156) and (158).

[0050] The host (502) can also communicate with one or more external devices (514), such as a keyboard, pointing device, etc.; a display (524); one or more devices that enable a user to interact with the host (502); and / or any device that enables the host (502) to communicate with one or more other computing devices (e.g., a network card, modem, etc.). Such communication can occur via input / output (I / O) interfaces (522). Furthermore, the host (502) can communicate with one or more networks via a network adapter (520), such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet). As described, the network adapter (520) communicates with other components of the host (502) via a bus (508). In one embodiment, multiple nodes of a distributed file system (not shown) communicate with the host (502) via I / O interfaces (522) or via the network adapter (520). It should be understood that, although not shown, other hardware and / or software components can be used in conjunction with the host (502). Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0051] In this document, the terms “computer program medium,” “computer-usable medium,” and “computer-readable medium” are used to generally refer to media such as main memory (506), including RAM (530), cache (532), and storage systems (534), such as removable storage drives and hard disks installed in hard disk drives.

[0052] A computer program (also referred to as computer control logic) is stored in memory (506). The computer program may also be received via a communication interface (such as a network adapter (520)). When run, such a computer program enables the computer system to perform the features of this embodiment as discussed herein. In particular, when run, the computer program enables the processing unit (504) to perform the features of the computer system. Thus, such a computer program represents the controller of the computer system.

[0053] In one embodiment, the host (502) is a node in a cloud computing environment. As is well known in the art, cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model may include at least five features, at least three service models, and at least four deployment models. Examples of such features are as follows:

[0054] On-demand self-service: Cloud consumers can automatically and unilaterally supply computing power, such as server time and network storage, on demand, without requiring human interaction with the service provider.

[0055] Extensive network access: Capabilities are available through the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0056] Resource pooling: The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. Location independence is significant because consumers typically do not have control or knowledge of the exact location of the provided resources, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0057] Rapid flexibility: Capacity can be supplied quickly and flexibly, and in some cases automatically, to rapidly shrink and rapidly expand. For the consumer, the available supply capacity often appears unlimited and can be purchased at any time and in any quantity.

[0058] Measuring services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at an abstraction layer appropriate to service types (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the providers and consumers of the services being utilized.

[0059] The service model is as follows:

[0060] Software as a Service (SaaS): The capability provided to the consumer is the use of the provider's applications running on cloud infrastructure. These applications are accessible from various client devices through thin client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage devices, or even the individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0061] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created or acquired by them onto cloud infrastructure. These applications are created using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage devices, but they have control over the deployed applications and the configuration of any application hosting environment.

[0062] Infrastructure as a Service (IaaS): The capability provided to consumers is the provision of processing, storage, networking, and other basic computing resources that enable consumers to deploy and run any software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage devices, deployed applications, and possibly limited control over the selection of networking components (e.g., host firewalls).

[0063] The deployment model is as follows:

[0064] Private cloud: Cloud infrastructure operated solely by an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0065] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It may be managed by an organization or a third party and may reside on-site or off-site.

[0066] Public cloud: Cloud infrastructure that is made available to the public or large groups of industries and is owned by an organization that sells cloud services.

[0067] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that maintain distinct entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursts for load balancing between clouds).

[0068] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. The core of cloud computing is its infrastructure, which includes a network of interconnected nodes.

[0069] Now for reference Figure 6 An illustrative cloud computing network (600) is shown. As illustrated, the cloud computing network (600) includes a cloud computing environment (650) with one or more cloud computing nodes (610), whose local computing devices used by the cloud consumer can communicate with said cloud computing nodes. Examples of such local computing devices include, but are not limited to, personal digital assistants (PDAs) or cellular phones (654A), desktop computers (654B), laptop computers (654C), and / or automotive computer systems (654N). Individual nodes within the nodes (610) can also communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment (600) to provide infrastructure, platform, and / or software as a service, without requiring the cloud consumer to maintain resources on their local computing devices. It should be understood that... Figure 6 The types of computing devices (654A-N) shown are for illustrative purposes only, and the cloud computing environment (650) can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0070] Now for reference Figure 7 This shows the result of Figure 6 The cloud computing network provides a set of functional abstraction layers (700). It should be understood beforehand that... Figure 7 The components, layers, and functions shown are for illustrative purposes only, and the embodiments are not limited thereto. As depicted, the following layers and corresponding functions are provided: hardware and software layer (710), virtualization layer (720), management layer (730), and workload layer (740).

[0071] The hardware and software layer (710) includes hardware and software components. Examples of hardware components include mainframes, which in one example are... System; a server based on a RISC (Reduced Instruction Set Computer) architecture, in one example being... system; System; IBM Systems; storage devices; network and networking components. Examples of software components include network application server software, in one example being... Application server software; and database software, in one instance, for Database software. (IBM, zSeries, pSeries, xSeries, BladeCerter, WebSphere, and DB2 are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide.)

[0072] The virtualization layer (72) provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers; virtual storage devices; virtual networks, including virtual private networks; virtual applications and operating systems; and virtual clients.

[0073] In one example, the management layer (730) can provide the following functions: resource provisioning, metering and pricing, a user portal, service layer management, and SLA planning and enforcement. Resource provisioning provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing provides cost tracking when utilizing resources in the cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, and protection for data and other resources. The user portal provides access to the cloud computing environment for consumers and system administrators. Service layer management provides the allocation and management of cloud computing resources to meet the required service layer requirements. Service layer agreement (SLA) planning and enforcement provides the pre-scheduling and procurement of cloud computing resources, where future demand is anticipated based on the SLA.

[0074] The workload layer (740) provides examples of functionalities that can be leveraged in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include, but are not limited to: mapping and navigation; software development and lifecycle management; virtual classroom education delivery; data analytics and processing; transaction processing; and automated and selective optimization of pipelines.

[0075] Although specific embodiments of this embodiment have been shown and described, it will be apparent to those skilled in the art that changes and modifications can be made based on the teachings herein without departing from the present embodiments and their broader aspects. Therefore, the appended claims are intended to cover within their scope all such changes and modifications that fall within the true spirit and scope of the embodiments. Furthermore, it should be understood that the embodiments are defined solely by the appended claims. Those skilled in the art will understand that if a specific number of claim elements is intentional, such intention will be explicitly stated in the claims, and without such statement, there is no such limitation. For non-limiting examples, to aid understanding, the appended claims include the use of the introductory phrases “at least one” and “one or more” to introduce claim elements. However, the use of such phrases should not be construed as implying that introducing a claim element by the indefinite article “a” or “an” limits any particular claim containing such an introduced claim element to an embodiment containing only one such element, even when the same claim includes the introductory phrase “one or more” or “at least one” and the indefinite article such as “a” or “an”; the same applies to the use of definite articles in the claims.

[0076] This embodiment may be a system, method, and / or computer program product. Furthermore, selected aspects of this embodiment may take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and / or hardware aspects, all of which may be collectively referred to herein as a "circuit," "module," or "system." Additionally, aspects of this embodiment may take the form of a computer program product embodied in a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to execute aspects of this embodiment. The disclosed systems, methods, and / or computer program products thus implemented are operable to improve the functionality and operation of an AI platform to automatically and selectively optimize DevOps pipeline artifacts based on learned dependencies identified as changes.

[0077] A computer-readable storage medium can be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, dynamic or static random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), magnetic storage devices, portable compact disc read-only memory (CD-ROM), digital universal disc (DVD), memory sticks, floppy disks, mechanically encoded devices (such as punched cards or raised structures in grooves with instructions recorded thereon), and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0078] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device.

[0079] Computer-readable program instructions used to perform the operations of this embodiment may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer, server, or server cluster. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet provided by an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry devices, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry devices for performing aspects of this embodiment.

[0080] This document describes aspects of the embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to the present embodiments. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0081] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices in a particular manner, such that the computer-readable storage medium having the instructions stored therein includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0082] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to produce a computer-implemented process on the computer, other programmable apparatus, or other device, such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0083] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this invention. Each block in a flowchart or block diagram may identify a portion of a module, segment, or instruction, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functionality involved, two consecutively shown blocks may actually execute substantially simultaneously, or these blocks may sometimes execute in reverse order. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or performs a combination of dedicated hardware and computer instructions.

[0084] It should be understood that although specific embodiments have been described herein for illustrative purposes, various modifications may be made without departing from the spirit and scope of the embodiments. In particular, the knowledge base may be localized, remote, or distributed across multiple systems. Therefore, the scope of protection of the embodiments is defined only by the appended claims and their equivalents.

Claims

1. A computer system, comprising: The processing unit is operatively coupled to the memory; An artificial intelligence (AI) platform is operatively coupled to the processing unit. The AI ​​platform is configured with one or more tools to support continuous integration and continuous deployment (CI / CD) pipelines for software development and DevOps operations. The one or more tools include: A graph manager is used to generate one or more dependency graphs based on two or more application artifacts; A machine learning ML manager, operationally coupled to the graph manager, which utilizes ML models to capture relationships between one or more dependency graphs and one or more pipeline artifacts; A monitor for detecting changes to one or more application artifacts in the application artifacts, and in response to the detected changes, the ML manager: The captured relationships are used to identify the impact of the detected changes on the one or more production line workpieces; and The pipeline is selectively optimized in response to identified effects, the optimization including automatically encoding changes in the mapping corresponding to the identified effects into the pipeline; and The processing unit is used to execute the optimized pipeline.

2. The computer system according to claim 1, wherein, The identification of the impact of the detected changes on one or more production line workpieces also includes: The ML manager creates a metric for the impact and uses the metric to predict the consequences of changes to the applied artifacts in the pipeline.

3. The computer system of claim 1 further includes a feedback manager for capturing any dependency patterns in the optimized pipeline.

4. The computer system of claim 3 further includes the feedback manager employing an active learning method to capture real-time data and real-time data feedback, and using the captured data to train the ML model.

5. The computer system of claim 1, wherein the application artifacts include source code, test code, application programming interface (API) specifications, configuration, deployment scripts, or combinations thereof.

6. The computer system of claim 1, wherein the captured relationship between the one or more dependency graphs and the one or more pipeline workpieces represents the influence dependency between the application workpiece and the one or more pipeline workpieces.

7. The computer system of claim 1, wherein each of the one or more assembly line workpieces has a corresponding dependency graph.

8. A computer program product supporting continuous integration and continuous deployment (CI / CD) pipelines for software development and DevOps operations, the computer program product comprising a computer-readable storage medium having program code embodied therein, the program code being processor-executable to: Generate one or more dependency graphs based on two or more application artifacts; A machine learning (ML) model is used to capture the relationship between the one or more dependency graphs and one or more pipeline artifacts. Detect changes to the one or more application artifacts, and in response to the detected changes, the ML model: The captured relationships are used to identify the impact of the detected changes on the one or more production line workpieces; as well as The pipeline is selectively optimized in response to the identified effects, the optimization including automatically encoding changes in the mapping corresponding to the identified effects into the pipeline; as well as Execute the optimized pipeline.

9. The computer program product of claim 8, wherein identifying the effect of the detected change on the one or more production line workpieces further comprises program code to: Create a metric for the impact, and use the metric to predict the consequences of changes to the applied workpiece in the pipeline.

10. The computer program product of claim 8, further comprising program code for capturing any dependency patterns in the optimized pipeline.

11. The computer program product of claim 10, further comprising program code for employing an active learning method to capture real-time data and real-time data feedback, and for using the captured data to train the ML model.

12. The computer program product of claim 8, wherein the application artifact includes source code, test code, application programming interface (API) specifications, configuration, deployment scripts, or a combination thereof.

13. The computer program product of claim 8, wherein the application artifact includes source code, test code, application programming interface (API) specifications, configuration, deployment scripts, or combinations thereof.

14. The computer program product of claim 8, wherein the captured relationship between the one or more dependency graphs and the one or more pipeline workpieces represents the influence dependency between the application workpiece and the one or more pipeline workpieces.

15. The computer program product of claim 8, wherein each of the one or more production line products has a corresponding dependency graph.

16. A method for software development and operation, comprising: Generate one or more dependency graphs based on two or more application artifacts; A machine learning (ML) model is used to capture the relationship between the dependency graph and one or more pipeline artifacts; The ML model detects changes to one or more of the application artifacts and, in response to the detected changes, performs the following: The captured relationships are used to identify the impact of the detected changes on the one or more production line workpieces; as well as In response to the identified impacts, the continuous integration and continuous deployment CI / CD pipeline is selectively optimized, the optimization including automatically encoding changes in the mappings corresponding to the identified impacts into the pipeline; as well as Execute the optimized pipeline.

17. The method of claim 16, wherein identifying the effect of the detected change on the one or more production line workpieces further comprises: The ML model creates a metric for the impact and uses the metric to predict the consequences of changes to the applied workpiece in the pipeline.

18. The method of claim 16, further comprising: Capture any dependency patterns in the optimized pipeline, employ an active learning approach to capture real-time data and real-time data feedback, and use the captured data to train the ML model.

19. The method of claim 16, wherein the application artifact includes source code, test code, application programming interface (API) specifications, configuration, deployment scripts, or a combination thereof.

20. The method of claim 16, wherein the captured relationship between the one or more dependency graphs and the one or more production line workpieces represents the influence dependency between the application workpiece and the one or more production line workpieces.

Images