Left-shift topology construction and information enhancement using machine learning

Abstract

A method, computer system and computer program product for left-shift topology construction are provided. The method may include: collecting data sets; extracting topological entities; correlating multiple data from topological entities; mapping topological entities; marking entry points for multiple subgraphs of topological entities; and constructing topological graphs.

Description

Background Art

[0001] The present invention relates generally to the field of computing, and more particularly to topology construction and machine learning. Microservices architecture can be implemented in a distributed manner using a variety of different topologies. Some include using centralized messaging topologies, topologies based on representational state transfer (REST), and topologies based on representational state transfer application interfaces (REST-APIs). Summary of the invention

[0002] Embodiments of the present invention disclose a method, computer system, and computer program product for left-shift topology construction. Embodiments of the present invention may include collecting a data set. Embodiments of the present invention may include extracting a topological entity from the data set. Embodiments of the present invention may include correlating multiple data from the topological entity. Embodiments of the present invention may include mapping the topological entity. Embodiments of the present invention may include marking entry points for multiple subgraphs of the topological entity. Embodiments of the present invention may include building a topological graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof read in conjunction with the accompanying drawings. The various features of the drawings are not to scale, as for the sake of clarity, the illustrations are intended to aid those skilled in the art in understanding the present invention in conjunction with the detailed description. In the drawings:

[0004] Figure 1 illustrates a networked computer environment according to at least one embodiment;

[0005] Figure 2 is a block diagram example of a microservices topology constructed according to at least one embodiment;

[0006] Figure 3 is an operational flow diagram illustrating a process of topology construction and information enhancement using machine learning in accordance with at least one embodiment;

[0007] Figure 4 According to at least one embodiment Figure 1 a block diagram of the internal and external components of the computers and servers depicted in;

[0008] Figure 5 According to an embodiment of the present disclosure, Figure 1 A block diagram of an illustrative cloud computing environment for a computer system as depicted in FIG. 1 ; and

[0009] Figure 6 According to the embodiment of the present disclosure Figure 5 A block diagram of the functional layers of an illustrative cloud computing environment. DETAILED DESCRIPTION

[0010] Detailed embodiments of the claimed structures and methods are disclosed herein, however, it is understood that the disclosed embodiments are merely illustrative of the claimed structures and methods that may be embodied in various forms. However, the present invention may be implemented in a variety of different forms and should not be construed as being limited to the exemplary embodiments set forth herein. On the contrary, these exemplary embodiments are provided to make the present disclosure thorough and complete and to fully convey the scope of the present invention to those skilled in the art. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0011] As mentioned earlier, microservices architecture can be implemented in a distributed manner using a variety of different topologies. Some include using a centralized messaging topology, a Representational State Transfer (REST)-based topology, and a Representational State Transfer Application Interface (REST-API)-based topology. An application programming interface (API) using a Representational State Transfer (REST) ​​architecture can use Hypertext Transfer Protocol (HTTP) requests to POST, DELETE, GET, or PUT data. An application programming interface (API) allows applications or software programs to communicate.

[0012] A microservices framework can be constructed using smaller independent architectural components that operate at high speeds. Microservices allow applications to be more easily managed and allow applications to be scalable. Independent components in a microservices framework can perform centralized operations, and other independent components in the same microservices framework can operate using different technologies. The independent scalability of microservices allows services to use larger amounts of resources to provide multiple cloud-based products.

[0013] Multiple microservice frameworks operate using container-based infrastructure. Containers can include isolated infrastructure that uses minimal resources, shares a hosted operating system, and is easy to integrate. Containers also allow for minimal software configuration because containers can use the same software code, for example, in a delivery pipeline. Different containers can operate within the same microservice framework and operate alternative software codes with minimal configuration. An application program interface (API) can include a set of functions or communication protocols that allow, for example, a container to access data on a virtual machine, operating system, service, or application.

[0014] The topology information or topology data obtained from the microservices is necessary for the microservice management stack and analysis, however, in the microservice or container framework, there may be no common components for building the topology without deploying the entire microservice. For example, building a topology of 500 or more microservices is a major task during runtime. Microservices are deployed together with distributed tracing tools and service mesh tools to build the topology, and additional information such as latency, application program interface (API) connections, and total resource consumption can be added. Distributed tracing tools may include, for example, Jaeger (open source software that performs end-to-end distributed tracing). Service mesh tools may include, for example, Istio (open source software that helps create a load balancing network of deployed services).

[0015] As microservice systems report data, topology information is built incrementally. Therefore, management tools such as distributed tracing and service mesh tools use the incrementally built topology information when reporting data, however, the management tools cannot take advantage of the properties until the entire topology is constructed. In addition, analytical services in a microservice framework often require topology information before deploying the entire service, especially when training offline. Therefore, it may be advantageous to create a system, method, and program product to continuously monitor information changes to a microservice framework, update the topology map, and predict dependencies between microservices.

[0016] The exemplary embodiments described below provide a system, method, and program product for topology construction using machine learning. Thus, embodiments of the present invention have the ability to improve the technical field of microservice topology construction by using machine learning to learn the operational characteristics of microservices in a microservice framework using static and dynamic information. More specifically, machine learning is used to learn operational characteristics from data such as logs, communication transmissions, documents, and subject matter expert (SME) input and feedback to predict topology graphs that can be used by analysis services and users.

[0017] According to an embodiment, left-shift analysis and machine learning can be used to build microservice topologies and to validate topology information. Left-shift analysis can be obtained from source code and can include, for example, deployment configurations, deployment specifications, or environment variables to add more in-depth data to the construction of the topology. An aspect of machine learning can be used to build the topology of microservices by analyzing deployment configurations or specifications without runtime information or by offline analysis. Offline analysis can provide information that can be used to extract performance estimates to build the topology.

[0018] According to an embodiment, various types of machine learning (ML) models can be constructed and used to create prediction results for building topologies for services and microservices. Data or information from different fields providing services and microservices can be obtained to build topologies. Fields can include, for example, retail, entertainment, social media, business, technology, medical, academic, government, industry, food chain, law or automobiles. Machine learning (ML) models can also include deep learning models and artificial intelligence (AI). Training and updating machine learning (ML) models can include supervised, unsupervised and semi-supervised machine learning (ML) processes. Supervised learning can use labeled data sets or labeled training sets to construct, train and update models. Unsupervised learning can use all unlabeled data to train models. Semi-supervised learning can use labeled data sets and unlabeled data sets to train models.

[0019] Supervised learning and semi-supervised learning can incorporate feedback or ground truth by individuals checking the accuracy of the data, data labels, and data classifications. Individuals are typically subject matter experts (SMEs) with extensive knowledge of a specific domain of a data set. Subject matter expert (SME) input can represent the ground truth of a machine learning (ML) model, and the ground truth provided can improve the accuracy and predictive power of the model. Subject matter experts (SMEs) can correct, modify, update, or remove classifications of data or data labels by manually updating the labeled data set. Using subject matter experts (SMEs) to provide feedback to the model can improve the accuracy of the model when updating or correcting the data set.

[0020] According to an embodiment, supervised or semi-supervised machine learning (ML) can be used to allow individuals (e.g., users, experts, or administrators) to have some improved effects on model training by having the ability to verify, change, update, or modify training sets. While users may not provide input to the data, experts can provide input or feedback to the model by changing the training set rather than an unsupervised machine learning (ML) environment. The training data set may include parameters of a classifier or label for learning purposes, and the supervised or semi-supervised machine learning (ML) environment may allow the user to update the label or classifier of the training set.

[0021] Cognitive analysis such as natural language processing (NLP) and semantic analysis can be used during the construction and training of the model. Cognitive analysis can analyze structured data and unstructured data to be incorporated into the machine learning (ML) process. Natural language processing (NLP) can be used to analyze the quality of data or receive feedback for training and model building. Structured data may include highly organized data, such as spreadsheets, relational databases, or data stored in fixed fields. Unstructured data may include data that is not organized and has a non-traditional internal structure, such as portable document formats (PDF), images, reports, web pages, video content, audio content, emails, word processing documents, or multimedia content.

[0022] The received data can be processed using natural language processing (NLP) to extract meaningful information to train and build models. Semantic analysis can be used to infer the complexity, meaning, and intent of the received information or data to identify the dependencies of services in the microservice network. Semantic analysis can also consider the syntactic structure of information at different levels to infer the meaning of words, phrases, sentences, and paragraphs of the received, collected, or extracted data.

[0023] According to an embodiment, portions of a framework for a left-shift topology program may include collecting data, extracting entities, retrieving meta information, correlating information, and building topologies. The collected data or information may be used to analyze and train models to improve microservices and microservice management. Data may be added to, stored in, and obtained from one or more databases, repositories, or corpora. For example, accessible data repositories may include databases that maintain configuration data, test instance data, open source data, DevOps pipeline logs, and other static and dynamic information associated with microservices. Feedback provided to a model being trained or fully trained (such as subject matter expert (SME) feedback) may also be stored on one or more databases.

[0024] Some examples of the collected data may include test documents, automation data, deployment scripts, configuration parameters, data from Swagger, Travis logs, or Jenkins logs. Data can be obtained from various sources. Other examples of the collected data may be data received based on microservice deployment, such as how often a user utilizes a microservice or how long a user utilizes a microservice. Some data may be received from devices such as Internet of Things (IoT) devices, global positioning systems (GPS), sensors, smart watches, smart phones, smart tablets, personal computers, or automotive devices. The collected data or information may be used to create a data set for model training and model use. Some of the collected data may be considered as static data, dynamic data, or real-time data, and the information may be widely used in microservice technologies and frameworks.

[0025] A microservice deployment, microservice, or container may have an average lifespan that decreases over time. Many microservices, such as social media microservices, have an average lifespan (i.e., the amount of time a user accesses the service) of less than 5 minutes, and therefore, topology information may be abstracted at high-level information to ensure continuous updating of the topology. Abstract information may be used to build and update connectivity or dependencies between microservices, and analysis may be provided without actual deployment of the microservices. The semantic context of services and dependencies may be identified using abstract information, and may be used to enhance and update the constructed topology graph.

[0026] Use case examples can include microservice management systems and analytics components. For example, topology information can be used to train analytical models for alert grouping or localization. Topology information can also be used for validation and configuration information and to develop deployment plans. Deployment specifications such as SIG applications or kube configurations can be generated. Fine-grained topology information (data) of upstream services such as application program interfaces (APIs) is necessary for microservice operations. For example, a microservice used by a consumer depends on or depends on an upstream service.

[0027] refer to Figure 1 , depicts an exemplary networked computer environment 100 according to one embodiment. The networked computer environment 100 may include a computer 102 having a processor 104 and a data storage device 106, which is capable of running a software program 108 and a left-shifted topology program 110a. The networked computer environment 100 may also include a server 112, which is capable of running a left-shifted topology program 110b that may interact with a database 114 and a communication network 116. The computer 102 may also be referred to as a client computer, and the server 112 may also be referred to as a server computer. The networked computer environment 100 may include multiple computers 102 and servers 112, only one of which is shown. The communication network 116 may include various types of communication networks, such as a wide area network (WAN), a local area network (LAN), a telecommunications network, a wireless network, a public switched network, and / or a satellite network. It should be understood that Figure 1 This merely provides an illustration of one implementation and does not imply any limitation on the environments in which different embodiments may be implemented. Many modifications to the described environment may be made based on design and implementation requirements.

[0028] The computer 102 can communicate with the server 112 via a communication network 116. The communication network 116 may include connections such as wired, wireless communication links, or fiber optic cables. Figure 4As discussed, the server 112 may include an internal component 902a and an external component 904a, respectively, and the computer 102 may include an internal component 902b and an external component 904b, respectively. The server 112 may also operate in a cloud computing service model, such as software as a service (SaaS), analysis as a service (AaaS), blockchain as a service (BaaS), platform as a service (PaaS), or infrastructure as a service (IaaS). The server 112 may also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud. The computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of running a program, accessing a network, and accessing a database 114. According to various implementations of the present embodiment, the left shift topology program 110a, 110b may interact with a database 114 that may be embedded in various storage devices (such as, but not limited to, a computer / mobile device 102, a server 112, or a cloud storage service).

[0029] According to the present embodiment, a user operating the computer 102 or the server 112 can use the left-shift topology program 110a, 110b (respectively) to construct a topology using left-shift analysis and machine learning. Figure 2 and Figure 3 Explain the left-shift topology method in more detail.

[0030] Reference now Figure 2 , depicts a block diagram example of a constructed microservice topology 200 used by the left-shift topology program 110a, 110b according to at least one embodiment. Topology entities are shown as being associated with application program interface (API) routes. Microservice topology 200 shows an example result of mapping topology entities to source code. Microservice topology 200 can represent a call graph that identifies the relationships of services, application program interface (API) connections, and resource connections.

[0031] The service API 202 provides an interface that allows services to interact, send data, or receive data. Services 1 to 5 (204, 206, 208, 210, and 212, respectively) may represent services at application program interface (API) endpoints. The application program interface (API) endpoints of each microservice may include / auth, / login, / home, / search, / recommend, or / member. For example, service 1 204 is / auth, service 2 206 is / get_info, service 3 208 is / recommend, service 4 210 is / graph, and service 5 212 is / nodes. Entity extraction and application program interface (API) endpoints may form a call control graph or call graph. A call graph may include a graph of a flow representing a call relationship between computer programs.

[0032] Edge AB is a bidirectional connection from A to B, which refers to a message or transmission sent from A to B. Blocking or non-blocking can refer to input-output (IO) operations during the process of performing an operation in a microservice framework. For example, a blocking operation takes a longer time period to process because the operation waits for a response to return before moving to the next operation. Therefore, blocking operations may have a large impact on performance, such as latency. Non-blocking operations can be an effective way to operate on the network and on storage to avoid the additional waiting time required in blocking operations. For example, while a blocking operation is waiting for a response to continue the operation, other processes can operate and execute until the response returns to the blocking operation.

[0033] Databases 1-2 (214 and 216, respectively) may include a database as a special network storage that blocks one or more processes until a result or response is returned. File system 218 is an additional blocking device or system that holds one or more processes until a result or response is returned from the file system. Databases 1-2 and file system 218 may be examples of blocking systems used by left-shift topology programs 110a, 110b.

[0034] Reference now Figure 3 , illustrates an operational flow diagram of an exemplary topology construction and information enhancement process 300 used by the left-shift topology programs 110a, 110b, according to at least one embodiment.

[0035] At 302, a data set is collected. Static data and real-time data can be collected from one or more repositories (e.g., databases 1 to 2) and stored thereon. The collected data can be, for example, static or dynamic information associated with a microservice, such as configuration data, test instance data, open source data, or DevOps pipeline logs. Other examples of the collected data can include test documents, automation data, deployment scripts, configuration parameters, data from Swagger, Travis logs, or Jenkins logs, and the data is obtained from various source databases. Additional data can be collected based on microservice deployment information. The collected data can be used as a data set for multiple purposes, such as extracting data, constructing or training models, verifying, constructing topology graphs, and using basic facts to refine the constructed model to update the data set.

[0036] At 304, topological entities are extracted from the data set. Entity extraction may include extracting topological entities from the data set, such as authorization entities, login entities, frontend entities, elastic search entities, graph database entities, recommendation entities, or member entities related to the microservice network. The related topological entities may be accessed to extract authorization information, login information, frontend information, search information, recommendation information, or member information. For example, the entity extraction process uses keyword search, language modeling, or simple field extraction to obtain endpoints for each related topological entity microservice. Reference Figure 2 An example of an endpoint for .

[0037] For each application, service, or microservice, a top list of representative keywords is identified. The top list of representative keywords is based on an analysis of word distribution in the collected data for each topic. For example, for each microservice, topological keywords are identified from a data set of collected documents and source code configurations. Based on the identified keywords and microservices, an unsupervised model can be constructed and used to identify important or relevant topics or names for the microservices. For example, topic modeling, clustering, or a priori models can be used to identify the topics of the services.

[0038] For example, the following representation can be used to extract keywords and identify related topics.

[0039] P(word=Auth,topic=1|θ d , β1)

[0040] =P(word=Auth|topic=1,β1)P(topic=1|θ d )

[0041] where P(word=Auth|topic=1,β1) represents each topic with a distribution over global words across all documents or collected data, and P(topic=1|θd ) represents each topic with a distribution over local words across all documents or collected data. The probabilities can be iterated using an iterative method such as estimate maximization until convergence is achieved and important topics are identified.

[0042] At 306, data from the extracted topological entities are correlated. Data can be correlated to match the same topological entity from different sources. Relevance of the topological entity to the application program interface (API) routing can be identified. Data can be correlated for each microservice application by retrieving data for each application or from each application program interface (API) or both.

[0043] At 308, topological entities are mapped. Interrelated data is mapped as a graph. For example, topological entities can be mapped to source code, and relationships can be identified from the mapped topological entities, such as caller-callee and application program interface (API) routing. Topological entities that are not called anywhere can be excluded. As an example, an application program interface (API) call can have multiple service calls in sequence or in parallel, and can be identified as blocking or non-blocking. Figure 2 An example of correlation of mapped topological entities, service calls, and application program interface (API) routes is shown.

[0044] At 310, central processing unit (CPU) cycles of the subtree are marked for each entry point. Meta information may be marked for each application program interface (API) entry point. Meta information or metadata may be retrieved from a call control graph and a call control graph program flow. Meta information may be retrieved for each application program interface (API). Meta information retrieved from each application program interface (API) may include, for example, central processing unit (CPU) cycles, processing time, estimated maximum calls per second, blocking code for network or storage access, threading or parallelism. Central processing unit (CPU) cycles may be marked for each entry point. Entry points may be determined by each application program interface (API). Processing time may be calculated for each subtree. The subtree may include traces for each application program interface (API) and portions of the entire program.

[0045] Learning models can be used to obtain meta information from subtrees or subgraphs. Subgraphs for nodes can have data represented as labels, such as central processing unit (CPU) cycles, memory, network read / write, or disk read / write. Graph convolution operations can be used to obtain the average of the features of a node and the features of the node's neighbors. Unlike image or text data, node neighbors can be unordered and of variable size. For example, subgraph nodes A and node X are processed by one or more graph convolutional networks (GCNs) before being processed by a multilayer perceptron (MLP) network, followed by a pooling function. The output can provide a softmax layer representing meta information of one or more node features.

[0046] At 312, a topology graph is constructed. The entire topology graph can be constructed using the correlation steps, mapping steps, and learning models. Additional information such as performance metadata can be marked for the nodes of the entire topology graph, such as estimated latency, endpoint routing, resource consumption, or estimated processing time for each link (i.e., edge). The represented nodes can be service endpoints, and the edges can be dependencies between endpoints. Additional information can be marked on the edges or links, and can include information such as latency, application program interface (API) connections, or round-trip time (RTT). The entire topology graph can be constructed by connecting sub-graphs. Additional information can be marked on subtrees or application program interfaces (APIs), such as estimated latency for each edge in the program. Figure 2 A reference diagram that is an example of an entire topology graph of nodes, edges, routes, and endpoints.

[0047] At 314, continuous topology information is verified. The entire constructed topology can be stored, and the information can be continuously monitored when new code is submitted or when configuration changes occur. The constructed topology can be verified using additional information, information changes, and information updates. Updates or changes to the information can be reflected in the topology graph. For example, the entire topology service graph can be stored and maintained in a database, and the topology graph can be changed or modified over time. The stored topology graph can be changed or enhanced with information using, for example, update data from a static log, configuration data, source code, or pipeline data.

[0048] At 316, additional or alternative embodiments allow for the insertion of active learning feedback models or components. The continuous validation of the constructed topological graph using updated static data, real-time information, subject matter expert (SME) feedback, or updated temporal topological data can be used as an active learning component. The active learning component can utilize supervised and semi-supervised models to inject real-time information or expert feedback to be added as basic facts. Active learning can be used as basic facts by obtaining, for example, subject matter expert (SME) or software developer feedback during model training, testing, and verification phases. User knowledge or expert knowledge can be captured as real-time data.

[0049] Understandably, Figure 2 and 3 Only a description of one embodiment is provided without implying any limitations on how different embodiments may be implemented. Various modifications may be made to the described embodiment(s) based on design and implementation requirements.

[0050] Figure 4 According to an illustrative embodiment of the present invention Figure 1 900 is a block diagram of the internal and external components of a computer shown in FIG. Figure 4 This merely provides an illustration of one implementation and does not imply any limitation on the environments in which different embodiments may be implemented. Many modifications to the described environment may be made based on design and implementation requirements.

[0051] Data processing systems 902, 904 represent any electronic device capable of executing machine-readable program instructions. Data processing systems 902, 904 may represent a smart phone, a computer system, a PDA, or other electronic device. Examples of computing systems, environments, and / or configurations that may be represented by data processing systems 902, 904 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network PCs, small computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0052] Computer 102 and server 112 may include Figure 4, and the corresponding sets of internal components 902a, b and external components 904a, b shown in . Each of the sets of internal components 902a, b includes one or more processors 906, one or more computer-readable RAMs 908 and one or more computer-readable ROMs 910 on one or more buses 912, and one or more operating systems 914 and one or more computer-readable tangible storage devices 916. The one or more operating systems 914, software programs 108, and left-shifted topology programs 110a in the client computer 102 and the left-shifted topology programs 110b in the network server 112 can be stored on one or more computer-readable tangible storage devices 916 for execution by the one or more processors 906 via one or more RAMs 908 (which typically include cache memories). Figure 4 In the embodiment shown in , each of the computer-readable tangible storage devices 916 is a disk storage device of an internal hard drive. Alternatively, each computer-readable tangible storage device 916 is a semiconductor memory device such as ROM 910, EPROM, flash memory or any other computer-readable tangible storage device capable of storing computer programs and digital information.

[0053] Each set of internal components 902a, b also includes a R / W drive or interface 918 to read from and write to one or more portable computer-readable tangible storage devices 920, such as CD-ROMs, DVDs, memory sticks, tapes, disks, optical disks, or semiconductor storage devices. Software programs, such as the software program 108 and the left-shift topology program 110a, 110b, can be stored on one or more corresponding portable computer-readable tangible storage devices 920, read via the corresponding R / W drive or interface 918, and loaded into the corresponding hard disk drive 916.

[0054] Each set of internal components 902a, b may also include a network adapter (or switch port card) or interface 922, such as a TCP / IP adapter card, a wireless wi-fi interface card, or a 3G / 4G / 5G wireless interface card or other wired or wireless communication link. The software program 108 and the left shift topology program 110a in the computer 102 and the left shift topology program 110b in the network server 112 may be downloaded from an external computer (e.g., a server) via a network (e.g., the Internet, a local area network, or other wide area network) and a corresponding network adapter or interface 922. From the network adapter (or switch port adapter) or interface 922, the software program 108 and the left shift topology program 110a in the computer 102 and the left shift topology program 110b in the network server computer 112 are loaded into the corresponding hard disk drive 916. The network may include copper wires, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers.

[0055] Each of the set of external components 904a, b may include a computer display monitor 924, a keyboard 926, and a computer mouse 928. External components 904a, b may also include touch screens, virtual keyboards, touch pads, pointing devices, and other human-computer interface devices. Each of the set of internal components 902a, b also includes a device driver 930 to interface to the computer display monitor 924, keyboard 926, and computer mouse 928. The device driver 930, the R / W driver or interface 918, and the network adapter or interface 922 include hardware and software (stored in the storage device 916 and / or ROM 910).

[0056] It is understood in advance that although the present disclosure includes detailed descriptions about cloud computing, the implementation of the teachings recorded herein is not limited to a cloud computing environment. Instead, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0057] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the provider of the service. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0058] Features are as follows:

[0059] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capabilities, such as server time and network storage, as needed without requiring manual interaction with the service provider.

[0060] Wide Area Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0061] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is location independence in the sense that consumers typically do not control or know the exact location of the resources provided, but are able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0062] Rapid elasticity: In some cases, the ability to scale out quickly and in quickly can be provisioned quickly and elastically. To the consumer, the capacity available for provisioning often appears to be unlimited and can be purchased in any quantity at any time.

[0063] Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0064] The service model is as follows:

[0065] Software as a Service (SaaS): The capability provided to consumers is to use the provider's applications running on a cloud infrastructure or hybrid cloud infrastructure. The applications are accessible from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0066] Platform as a Service (PaaS): The capability provided to consumers is to deploy consumer-created or acquired applications onto cloud infrastructure, where the applications are created using programming languages ​​and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possible configuration of the application hosting environment.

[0067] Analytics as a Service (AaaS): The capability provided to consumers is to access an analytics platform using a web-based or cloud-based network (i.e., infrastructure). The analytics platform may include access to analytics software resources, or may include access to related databases, corpora, servers, operating systems, or storage. The consumer does not manage or control the underlying web-based or cloud-based infrastructure including databases, corpora, servers, operating systems, or storage, but has control over the deployed applications and possibly the configuration of the application hosting environment.

[0068] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0069] The deployment model is as follows:

[0070] Private Cloud: The cloud infrastructure is operated only for the organization. It can be managed by the organization or a third party and can exist inside or outside the building.

[0071] Community cloud: The cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0072] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by an organization that sells cloud services.

[0073] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community or public) that remain a unique entity but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0074] The cloud computing environment is service-oriented, with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure consisting of a network of interconnected nodes.

[0075] Reference now Figure 5 , depicts an illustrative cloud computing environment 1000. As shown, the cloud computing environment 1000 includes one or more cloud computing nodes 1050 with which a local computing device used by a cloud consumer can communicate, such as a personal digital assistant (PDA) or cellular phone 1000A, a desktop computer 1000B, a laptop computer 1000C, and / or an automobile computer system 1000N. The cloud computing nodes 1050 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment 1000 to provide infrastructure, platform, and / or software as a service for which the cloud consumer does not need to maintain resources on a local computing device. It should be understood that Figure 5 The types of computing devices 1000A-N shown in are intended to be illustrative only, and computing node 1050 and cloud computing environment 1000 may communicate with any type of computing device over any type of network and / or network addressable connection (eg, using a web browser).

[0076] Reference now Figure 6 , shows a set of functional abstraction layers 1100 provided by the cloud computing environment 1000. It should be understood in advance that Figure 6 The components, layers, and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0077] The hardware and software layer 1102 includes hardware and software components. Examples of hardware components include: host 1104; server 1106 based on RISC (Reduced Instruction Set Computer) architecture; server 1108; blade server 1110; storage device 1112; and network and networking components 1114. In some embodiments, software components include network application server software 1116 and database software 1118.

[0078] Virtualization layer 1120 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 1122 ; virtual storage 1124 ; virtual networks 1126 , including virtual private networks; virtual applications and operating systems 1128 ; and virtual clients 1130 .

[0079] In one example, the management layer 1132 may provide the functionality described below. Resource provisioning 1134 provides dynamic procurement of computing resources and other resources for performing tasks within a cloud computing environment. Metering and pricing 1136 provides cost tracking when resources are utilized in a cloud computing environment, as well as accounting or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 1138 provides access to the cloud computing environment for consumers and system administrators. Service level management 1140 provides cloud computing resource allocation and management so that the required service levels are met. Service level agreement (SLA) planning and fulfillment 1142 provides pre-scheduling and procurement of cloud computing resources, where future demand is anticipated based on the SLA.

[0080] The workload layer 1144 provides examples of functionality that can take advantage of a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include: mapping and navigation 1146; software development and lifecycle management 1148; virtual classroom education delivery 1150; data analytics processing 1152; transaction processing 1154; and shift-left topology construction 1156. The shift-left topology program 110A, 110b provides a way to use machine learning to learn the operational characteristics of microservices to build a topological representation of services and service characteristics.

[0081] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, and the computer-readable program instructions are used to cause a processor to perform various aspects of the present invention.

[0082] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0083] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.

[0084] The computer-readable program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages, such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as "C" programming language, Python programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider via the Internet). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute a computer-readable program instruction by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0085] Aspects of the present invention are described herein with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to embodiments of the present invention. It will be understood that each frame of the flow chart and / or block diagram and the combination of frames in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0086] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0087] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0088] Flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each frame in the flow chart or block diagram can represent the module, segment or part of instruction, which includes one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure.For example, two frames shown continuously can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the function involved.It will also be noted that the combination of the frames in each frame of the block diagram and / or flow chart illustration and the block diagram and / or flow chart illustration can be realized by a dedicated hardware-based system that performs a specified function or action or performs a combination of special hardware and computer instructions.

[0089] The description of various embodiments of the present invention has been given for the purpose of illustration, but it is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements existing in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for left-shift topology construction, the method comprising: Determining operational characteristics associated with different microservices in a microservice framework based on the collected data sets to create prediction results for building topologies for the different microservices, wherein the determining further includes training and using a machine learning model to retrieve the collected data sets and learn the operational characteristics; Extracting topological entities from the collected data set using the machine learning model, wherein extracting the topological entities comprises: identifying keywords and topics distributed among the collected data set, and determining related topics among the identified topics associated with the different microservices; Correlating a plurality of data from the topological entities with each other, wherein the correlating further comprises: matching the same topological entity from a plurality of different sources associated with the collected data sets, and identifying the correlation of the topological entity with an application program interface (API) route between the different microservices; Mapping the topological entity; marking entry points for a plurality of subgraphs of the topological entity; and Based on the associations for the topological entities, the mappings, and the markings of the entry points, the machine learning model is used to construct an entire topological graph, wherein nodes represent service endpoints and edges represent dependencies between the service endpoints.

2. The method according to claim 1, further comprising: updating the data set using static data; as well as The updated data set is continuously validated.

3. The method according to claim 1, further comprising: Using real-time data from subject matter experts (SMEs) as an active learning feedback model to update the dataset; as well as The updated data set is continuously validated.

4. The method of claim 1, wherein extracting the topological entity comprises extracting information from an authorization entity, a login entity, a front-end entity, an elastic search entity, and a graph database entity.

5. The method of claim 1, wherein marking the entry points for the plurality of sub-graphs comprises marking a central processing unit (CPU) cycle of a sub-graph for each of the entry points.

6. The method according to any one of claims 1 to 5, wherein a learning model is used to obtain meta-information from the plurality of sub-graphs.

7. The method according to any one of claims 2 to 6, wherein the updated data set is continuously verified when configuration changes occur.

8. A computer system for left-shift topology construction, the computer system comprising: One or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored on at least one of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing a method comprising: Determining operational characteristics associated with different microservices in a microservice framework based on the collected data sets to create prediction results for building topologies for the different microservices, wherein the determining further includes training and using a machine learning model to retrieve the collected data sets and learn the operational characteristics; Extracting topological entities from the collected data set using the machine learning model, wherein extracting the topological entities comprises: identifying keywords and topics distributed among the collected data set, and determining related topics among the identified topics associated with the different microservices; Correlating a plurality of data from the topological entities with each other, wherein the correlating further comprises: matching the same topological entity from a plurality of different sources associated with the collected data sets, and identifying the correlation of the topological entity with an application program interface (API) route between the different microservices; Mapping the topological entity; marking entry points for a plurality of subgraphs of the topological entity; and Based on the associations for the topological entities, the mappings, and the markings of the entry points, the machine learning model is used to construct an entire topological graph, wherein nodes represent service endpoints and edges represent dependencies between the service endpoints.

9. The computer system of claim 8, further comprising: updating the data set using static data; as well as The updated data set is continuously validated.

10. The computer system according to claim 8, further comprising: Using real-time data from subject matter experts (SMEs) as an active learning feedback model to update the dataset; as well as The updated data set is continuously validated.

11. The computer system of claim 8, wherein extracting the topological entity comprises extracting information from an authorization entity, a login entity, a front-end entity, an elastic search entity, and a graph database entity.

12. The computer system of claim 8, wherein marking the entry points for the plurality of sub-graphs comprises marking a central processing unit (CPU) cycle of a sub-graph for each of the entry points.

13. The computer system of claim 8, wherein a learning model is used to obtain meta-information from the plurality of sub-graphs.

14. The computer system of claim 9, wherein the updated data set is continuously validated as configuration changes occur.

15. A computer program product for left-shift topology construction, the computer program product comprising: One or more computer-readable tangible storage media and program instructions stored on at least one of the one or more computer-readable tangible storage media, the program instructions being executable by a processor to cause the processor to perform a method comprising: Determining operational characteristics associated with different microservices in a microservice framework based on the collected data sets to create prediction results for building topologies for the different microservices, wherein the determining further includes training and using a machine learning model to retrieve the collected data sets and learn the operational characteristics; Extracting topological entities from the collected data set using the machine learning model, wherein extracting the topological entities comprises: identifying keywords and topics distributed among the collected data set, and determining related topics among the identified topics associated with the different microservices; Correlating a plurality of data from the topological entities with each other, wherein the correlating further comprises: matching the same topological entity from a plurality of different sources associated with the collected data sets, and identifying the correlation of the topological entity with an application program interface (API) route between the different microservices; Mapping the topological entity; marking entry points for a plurality of subgraphs of the topological entity; and Based on the associations for the topological entities, the mappings, and the markings of the entry points, the machine learning model is used to construct an entire topological graph, wherein nodes represent service endpoints and edges represent dependencies between the service endpoints.

16. The computer program product of claim 15, further comprising: updating the data set using static data; as well as The updated data set is continuously validated.

17. The computer program product of claim 15, further comprising: Using real-time data from subject matter experts (SMEs) as an active learning feedback model to update the dataset; as well as The updated data set is continuously validated.

18. The computer program product of claim 15, wherein extracting the topological entity comprises extracting information from an authorization entity, a login entity, a frontend entity, an elastic search entity, and a graph database entity.

19. The computer program product of claim 15, wherein a learning model is used to obtain meta-information from the plurality of sub-graphs.

Images