Provisioning using computer vision

Abstract

To provide a system and method for provisioning computing resources using computer vision.SOLUTION: A method comprises: receiving an image comprising a plurality of visual elements; and recognizing the visual elements within the image as objects that represent network elements. The recognized objects have network element types such as a database, a load balancer, a sub-network, and a virtual machine instance and comprise one or more of these types of network elements. The method also comprises: recognizing an arrangement of the recognized objects that represents a network architecture; and provisioning computing resources corresponding to the objects by provisioning network elements that comprise one or more of the network element types. The provisioning generates a network architecture for the network elements based on the recognized arrangement.SELECTED DRAWING: Figure 17

Description

Technical Field

[0001] Field Embodiments of the present disclosure generally relate to using computer vision to provision computing resources.

Background Art

[0002] Background Computing services implemented by modern service providers are often required to be robust, flexible, and highly available. For example, modern demands for computing often arise from the opportunity costs associated with inefficient or unavailable computing resources. Some settings enable on-demand provisioning of computing resources, such as those provided by cloud-based service providers, but this provisioning often involves multi-step processes, manual labor, and other inconveniences. Considering the stringent demands of modern computing, a tool that can efficiently and quickly provide available computing resources would benefit the users of the tool.

Summary of the Invention

Means for Solving the Problems

[0003] Summary Embodiments of this disclosure are generally directed to systems and methods for provisioning computing resources using computer vision. An image comprising multiple visual elements can be received. The visual elements in the image can be recognized as objects representing network elements, the recognized objects representing network elements of a certain network element type, the network element type comprising one or more of a database, load balancer, subnetwork, or virtual machine instance, and the arrangement of the recognized objects representing a network architecture is recognized. Computing resources can be provisioned in correspondence with the recognized objects by provisioning network elements comprising one or more of the network element types, and the provisioning comprises the step of generating a network architecture for the provisioned network elements based on the recognized arrangement.

[0004] Features and advantages of the embodiments are described below, or are evident from the description, or may be recognized by implementing the disclosure.

[0005] Further embodiments, details, advantages, and variations will become apparent from the following detailed description of preferred embodiments, in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0006] [Figure 1] This figure shows a system for provisioning computing resources using computer vision, according to an exemplary embodiment. [Figure 2] This is a block diagram of computing devices operationally coupled to a system according to an exemplary embodiment. [Figure 3] This figure shows a diagram of a computing architecture according to an exemplary embodiment. [Figure 4] This figure shows another diagram of a computing architecture according to an exemplary embodiment. [Figure 5] This figure shows an image of a computing architecture according to an exemplary embodiment. [Figure 6] This figure shows the inputs for a computing architecture according to an exemplary embodiment. [Figure 7] This figure shows a processed image of a computing architecture according to an exemplary embodiment. [Figure 8] This figure shows objects recognized in an image of a computing architecture according to an exemplary embodiment. [Figure 9A] This figure shows a configuration file for provisioning a computing architecture according to an exemplary embodiment. [Figure 9B] This figure shows a configuration file for provisioning a computing architecture according to an exemplary embodiment. [Figure 10] This figure shows a user interface related to provisioning a computing architecture according to an exemplary embodiment. [Figure 11] This figure shows a user interface related to provisioning a computing architecture according to an exemplary embodiment. [Figure 12] This figure shows a user interface related to provisioning a computing architecture according to an exemplary embodiment. [Figure 13] This figure shows a user interface related to provisioning a computing architecture according to an exemplary embodiment. [Figure 14] This figure shows a user interface related to provisioning a computing architecture according to an exemplary embodiment. [Figure 15]This figure shows a user interface related to provisioning a computing architecture according to an exemplary embodiment. [Figure 16] This figure shows a user interface related to provisioning a computing architecture according to an exemplary embodiment. [Figure 17] This is a flowchart for provisioning computing resources using computer vision, according to an exemplary embodiment. [Modes for carrying out the invention]

[0007] Detailed explanation The embodiment provisions computing resources using computer vision. For example, an image containing a diagram of a computer architecture may be received. This diagram may include visual elements representing network computing elements such as virtual machines, load balancers, databases, subnets, and other suitable computing elements. For example, some contours can be detected on the visual elements in the received image, and objects can be recognized based on the detected contours.

[0008] In some embodiments, the arrangement of visual elements can also be detected, and this arrangement represents an architecture for computing resources. Based on the detected objects and their arrangement, computing resources can be provisioned. For example, one or more virtual machines, load balancers, databases, subnets, and other suitable computing elements can be provisioned.

[0009] In some embodiments, computing resources can be provisioned using one or more generated configuration files. For example, a configuration file These can be generated based on the detected objects and their placement. In some embodiments, the provisioning tool may be configured to provision computing resources according to one or more generated configuration files. For example, configuration files can be generated using the detected placements so that the network architecture based on the detected placements is defined by the configuration files. Based on these definitions, the computing resources to be provisioned later can be organized according to the network architecture.

[0010] Hereinafter, embodiments of the present disclosure are given in detail. Examples of embodiments are shown in the accompanying drawings. The following detailed description includes numerous specific details to allow for a full understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be carried out without these specific details. In other examples, well-known methods, procedures, components and circuits are not described in detail so as not to unnecessarily obscure aspects of the embodiments. Wherever possible, similar reference numerals are used for similar elements.

[0011] Figure 1 shows a system for provisioning computing resources using computer vision according to an exemplary embodiment. The system 100 includes a computer vision tool 102, an image acquisition device 104, and a diagram 106. In some embodiments, the diagram 106 may represent a network architecture. For example, a predetermined correspondence between visual elements and network computing elements can be defined, and the diagram 106 may include an arrangement of these visual elements. In some embodiments, the arrangement of these visual elements may represent a network architecture for the network elements represented in the diagram 106.

[0012] In some embodiments, the image capture device 104 can be used to capture an image of the diagram 106. For example, the diagram 106 can be drawn on a surface such as a whiteboard, physical paper, blackboard, display, or other suitable surface. In some embodiments, the diagram 106 can be generated as part of a brainstorming session, meeting, or other discussion (e.g., a prior discussion or planning meeting). For example, team members can discuss computing requirements for a project, service, product, or other implementation, and based on that discussion, the team can generate (e.g., draw by hand) the diagram 106. Then, the image capture device 104 can be used to capture an image of the diagram 106. Other embodiments can include other techniques for generating the diagram 106 and / or capturing an image of the diagram 106.

[0013] In some embodiments, the captured image can be received by a computer vision tool 102. For example, the image acquisition device 104 can provide the image to the computer vision tool 102 (for example, by sending the image to a computing device that implements the computer vision tool 102). Embodiments include other preferred techniques for receiving images of a computer network architecture. The computer vision tool 102 can process the image to recognize objects within the image. For example, the image can be transformed using one or more filters or transformations, and objects can be recognized by the computer vision tool 102 based on this processed image. In some embodiments, the recognized objects may be visual elements in diagram 106, such as visual elements that include a predetermined correspondence with network computing elements. The arrangement of objects can also be recognized, and this arrangement represents a network architecture for network computing elements. Embodiments perform object detection and / or object recognition functions and algorithms, but these terms are not used throughout this disclosure. They are used with the same meaning.

[0014] In some embodiments, network computing elements corresponding to the detected / recognized objects can be provisioned. For example, the computer vision tool 102 can cause the provisioning of these network computing elements. In some embodiments, one or more configuration files can be generated by the computer vision tool 102, and these configuration files are used to provision network computing elements. For example, the provisioning tool can be configured to provision computing resources according to the generated configuration files. In some embodiments, the provisioned computing resources may be part of a cloud computing environment, and the provisioning tool can provision network computing elements within the cloud. For example, the network computing elements can include one or more of virtual machines, sub-networks, load balancers, databases, etc.

[0015] In some embodiments, the generated configuration files can be based on the detected arrangement of the detected objects in the processed image. For example, the detected arrangement of the detected objects can represent a load balancer that balances the load between two sub-networks. In some embodiments, the generated configuration files can be configured such that the provisioned computing elements include the relationships represented by the detected arrangement (for example, the load balancer provisioned according to the configuration file is configured to balance the load between two sub-networks).

[0016] The embodiment offers several advantages compared to conventional computing resource provisioning technologies. Conventional computing resource provisioning tools often require cumbersome requirements definition, particularly the clear definition of the relationships between computing resources, virtual machine or database instances, and network elements such as load balancers. For example, users often have to navigate numerous user interface pages to enter all the necessary information. At other times, users have to generate data files containing this information.

[0017] Embodiments of this disclosure provide improved speed and efficiency for availability by reducing user effort and the complexity of requirements definition. For example, a predetermined correspondence between visual and network elements allows for the definition of computing resource requirements using simple diagrams. Often, the creation of simple diagrams is part of a separate flow from provisioning, such as a business flow (or other technical flow) for determining how to realize a service or product. Embodiments allow for the reuse of these simple diagrams for provisioning, thus reducing the excessive effort associated with prior art for defining resource requirements.

[0018] Furthermore, the detected placement of these visual elements, as performed by some embodiments, further reduces complexity and improves the speed of availability. For example, embodiments enable the provisioning of advanced networks of computing resources, such as networks with several subnetworks, connections, and heterogeneous elements. Embodiments of provisioning computer vision tools provide a robust solution that can improve the entire provisioning process, even in complex computing resource realizations.

[0019] The cumbersome nature of traditional provisioning can be particularly cumbersome with cloud resources. For example, modern cloud service providers are required to provide efficient, fast, and flexible technologies for realizing computing resources. This is often the result of the opportunity cost of time-consuming, inefficient, or cumbersome technologies, which would be very expensive for organizations (e.g., requiring time, money, and brand reputation). Therefore, the embodiments used to provision cloud resources would be particularly useful given the demanding requirements placed on modern cloud service providers.

[0020] Figure 2 is a block diagram of a computer server / system 200 according to an embodiment. All or part of the system 200 may be used to realize any of the elements shown in Figure 1. As shown in Figure 2, the system 200 may include a bus device 212 and / or other communication mechanisms configured to exchange information between various components of the system 200, such as the processor 222 and memory 214. A communication device 220 may also enable connectivity between the processor 222 and other devices by encoding data sent from the processor 222 to another device over a network (not shown) and decoding data received from another system over the network for the processor 222.

[0021] For example, the communication device 220 may include a network interface card configured to provide wireless network communication. Various wireless communication technologies may be used, including infrared communication, wireless communication, Bluetooth® communication, Wi-Fi communication, and / or cellular communication. Alternatively, the communication device 220 may be configured to provide wired network connectivity, such as Ethernet® connectivity.

[0022] The processor 222 may include one or more general-purpose or specific-purpose processors to perform calculations and control the functions of the system 200. The processor 222 may include a single integrated circuit, such as a microprocessing device, or it may include multiple integrated circuit devices and / or circuit boards that work together to perform the functions of the processor 222. The processor 222 may also run computer programs, such as an operating system 215, a computer vision tool 216, and other applications 218, which are stored in memory 214.

[0023] System 200 may include memory 214 for storing information and instructions for execution by processor 222. Memory 214 may include various components for retrieving, displaying, modifying, and storing data. For example, memory 214 may store software modules that, when executed by processor 222, provide functionality. These modules may include an operating system 215 that provides the operating system functionality of system 200. These modules may include the operating system 215, a computer vision tool 216 that implements the computer vision and provisioning functions disclosed herein, and other application modules 218. The operating system 215 provides the operating system functionality of system 200. In some examples, the computer vision tool 216 may be implemented as an in-memory configuration. In some implementations, when system 200 performs the functions of the computer vision tool 216, it implements an unconventional specialized computer system that performs the functions disclosed herein.

[0024] Non-temporary memory 214 may include various computer-readable media that can be accessed by the processor 222. For example, memory 214 may include any combination of random access memory ("RAM"), dynamic RAM ("DRAM"), static RAM ("SRAM"), read-only memory ("ROM"), flash memory, cache memory, and / or other types of non-temporary computer-readable media. The processor 222 is further coupled to a display 224, such as a liquid crystal display ("LCD"), via a bus 212. A keyboard 226 and a cursor control device 228, such as a computer mouse, are further coupled to a communication device 212, enabling a user to connect with the system 200.

[0025] In some embodiments, system 200 may be part of a larger system. Therefore, system 200 may include one or more additional functionality modules 218 to include additional functionality. Other application modules 218 may include, for example, various modules of Oracle® Cloud Infrastructure, Oracle® Cloud Platform, or Oracle® Cloud Applications. Computer vision tools 216, other application modules 218, and other suitable components of system 200 may include various modules of Terraform, OpenCV, other suitable open-source tools, or other suitable Oracle® products or services from HasiCorp.

[0026] Database 217 is coupled to bus 212 to provide centralized storage for modules 216 and 218, storing data received by, for example, computer vision tool 216 or other data sources. Database 217 can store data in a unified collection of logically related records or files. Database 217 may be an operational database, analytical database, data warehouse, distributed database, end-user database, external database, navigational database, in-memory database, document-oriented database, real-time database, relational database, object-oriented database, non-relational database, NoSQL database, HADUP® Distributed File System ("HFDS"), or any other database known in the art.

[0027] Although the functionality of System 200 is presented as a single system, it may be implemented as a distributed system. For example, the memory 214 and processor 222 may be distributed across multiple different computers that collectively represent System 200. In one embodiment, System 200 may be part of a device (e.g., a smartphone, tablet, or computer). In embodiments, System 200 may be separate from the device and may remotely provide the disclosed functionality to the device. Furthermore, one or more components of System 200 may not be included. For example, for functionality as a user or consumer device, System 200 may be a smartphone or other wireless device that includes a processor, memory, and a display, and does not include one or more of the other components shown in Figure 2, and includes additional components not shown in Figure 2 (e.g., an antenna, transceiver, or other suitable wireless device components). Furthermore, when System 200 is implemented to perform the functionality disclosed herein, it is a special-purpose computer specifically adapted to provide demand forecasting.

[0028] The embodiment uses computer vision to provision computing resources. Returning to Figure 1, the computer vision tool 102 is network computer An image having a diagram representing the computing elements can be received. Figure 3 shows a diagram of a computing architecture according to an exemplary embodiment. For example, diagram 302 may include visual elements such as shapes and connections representing computing network elements and computing architecture. For example, diagram 302 may be drawn on a surface such as a whiteboard, physical paper, blackboard, display or other suitable surface.

[0029] In some embodiments, a predetermined correspondence between visual elements and network computing elements can be defined. For example, the visual elements represented in diagram 302 may include shapes, and various shapes may correspond to various network computing elements. In some embodiments, the visual elements and their arrangement may represent a network architecture.

[0030] Figure 4 shows a computing architecture according to an exemplary embodiment. For example, diagram 302 may represent a network architecture 400, which may include a load balancer 402, a network 404, subnetworks 406 and 408, virtual machines 410 and 412, connections 414 and 416, and a database 418. In some embodiments, these network elements are represented by shapes drawn in diagram 302. For example, the load balancer 402 may be represented by a triangle, the network 404 by a rectangle, subnetworks 406 and 408 by rectangles contained within another drawn network element (e.g., network 404), virtual machine 410 by a rectangle contained within another drawn network element (e.g., subnetwork 406), virtual machine 412 by a pentagon contained within another drawn network element (e.g., subnetwork 408), and database 418 by a circle. In some embodiments, these representations are realized based on a predetermined correspondence between a shape (and / or the orientation of a shape, such as being contained within another shape) and the specific network elements they represent.

[0031] In some embodiments, shapes can represent specific network computing entities, such as database or virtual machine types. For example, a circle representing database 418 may correspond to an autonomous database. Other shapes may represent other types of databases. Also, a rectangle contained within another visual element may represent virtual machine 410 or a first type of virtual machine, and a pentagon contained within another visual element may represent virtual machine 412 or a second type of virtual machine. In some embodiments, the differences between virtual machines range from the number of CPU cores provisioned to the different operating systems running on the virtual machines, depending on how the system is configured. For example, virtual machine 410 may be VM Standard 2.4, and virtual machine 412 may be VM Standard 2.8. In some embodiments, the virtual machine options are determined and / or provided by the cloud vendor. Figures 3 and 4 illustrate specific correspondences between shapes and network elements, but other preferred definitions / representations may be realized.

[0032] In some embodiments, diagram 302 includes the arrangement of visual elements. For example, the arrangement of visual elements may represent architecture 400. The visual elements may show that load balancer 402 connects to subnetworks 406 and 408 using connections 414 and 416. This representation may represent that load balancer 402 balances the load between subnetworks 406 and 408. [Demonstrated Implementation] In this embodiment, subnetwork 406 includes virtual machine 410, and subnetwork 408 includes virtual machine 412. Therefore, in the shown embodiment, the load balancer 402 effectively balances the load between virtual machine 410 and virtual machine 412.

[0033] Returning to Figure 1, an image of the diagram 302 can be captured using the image acquisition device 104. In some embodiments, the captured image can be provided / transmitted to the computer vision tool 102. Other preferred implementations for receiving the diagram image in the computer vision tool 102 can also be used.

[0034] Figure 5 shows an image of a computing architecture according to an exemplary embodiment. For example, image 502 may be an image of diagram 302 from Figure 3, which may further represent architecture 400 from Figure 4. In other words, image 502 may include a diagram that includes visual elements representing network computing elements and the arrangement of visual elements representing the network architecture. In some embodiments, image 502 is received by / transmitted to the computer vision tool 102 of Figure 1.

[0035] In some embodiments, the computer vision tool 102 may be software that receives an image, recognizes objects in the image, and provisions computing resources. For example, the computer vision tool may be compiled code (e.g., executable) or script code that receives the image 502 of Figure 5 as input. In some embodiments, additional information may be received along with the image 502. Figure 6 shows inputs for a computing architecture according to an exemplary embodiment.

[0036] In some embodiments, the input parameter 602 may include the number of network elements represented in image 502. For example, the input parameter 502 can be broken down into network element types (e.g., networks, subnets, virtual machine instances, load balancers, connections, databases, etc.). In the embodiments shown, the input parameter 602 for image 502 is one network, two subnets, two virtual machine instances, one load balancer, two connections, and one database. Figure 6 also shows the computer vision tool 102 as a script (e.g., a Python script) that accepts the input parameter 602. The computer vision tool 102 may be other forms of software, hardware, or any combination thereof.

[0037] In some embodiments, once the image 502 and input parameters 602 are received by the computer vision tool 102, the image 502 is processed. For example, one or more filters or masks can be applied to the image 502 to transform it. In some embodiments, convolution or cross-correlation functions can be used to adjust the pixel values ​​of the image 502 to produce a processed image. In some implementations, multiple processed images can be produced (for example, based on multiple filters, masks, convolution, cross-correlation, and other image processing techniques). In some embodiments, image processing is implemented using one or more open-source tools (e.g., OpenCV) or other image processing tools.

[0038] Figure 7 shows a processed image of a computing architecture according to an exemplary embodiment. For example, one or more processing techniques can be performed to generate a processed image 702 from image 502. In the embodiment shown, the color of image 502 The scheme has been modified (for example, black and white in the image have been inverted), and other image quality adjustments have been performed to arrive at the processed image 702.

[0039] In some embodiments, the image processing algorithm can convert an initial image to grayscale and then cycle through multiple image processing parameters, such as filters, masks, and transformations described herein, or any preferred techniques, to generate a processed image. For example, an object recognition algorithm can be used to attempt to detect a desired architecture provided by the user (e.g., defined by user input) for one or more of the processed images. If the object recognition algorithm detects the desired architecture, the associated processed image can be considered to have been correctly processed by the current set of image processing parameters. If the object recognition algorithm does not detect the desired architecture, the processing parameters can be repeatedly modified, the image can be reprocessed with the new configuration, and the object recognition algorithm can be run on the newly processed image. In some embodiments, this iterative technique can be repeated until the desired architecture is detected by the object recognition algorithm.

[0040] In some embodiments, object recognition is performed on one or more processed images. For example, a computer vision tool 102 can detect objects in an image by running an object recognition algorithm or function on a processed image 702. In some embodiments, object recognition is performed by running one or more open-source tools (e.g., OpenCV). For example, a shape object recognition algorithm or function (e.g., OpenCV's findContours function) or other suitable recognition protocol can be used.

[0041] Figure 8 shows objects recognized in an image of a computing architecture according to an exemplary embodiment. For example, Figure 8 includes a triangle 802, a rectangle 804, squares 806 and 808, a square 810, a pentagon 812, lines 814 and 816, and a circle 818. In some embodiments, a shape object recognition algorithm or function can be executed, which is followed by an edge detection algorithm or function for detecting the edges of an object, and then an object recognition algorithm or function for recognizing the detected object. For example, the detected object shown in Figure 8 is based on the visual elements in diagram 302 of Figure 3, because the processed image is based on the captured image of the diagram. In the embodiments shown, the visual elements in diagram 302 include shapes, so an object recognition algorithm can be executed that readily identifies and distinguishes shapes. For example, the shapes in diagram 302 and the objects shown in Figure 8 can be distinguished based on the number of sides they have (for example, zero for a circle, one for a line, three for a triangle, four for squares and rectangles, and five for a pentagon). Furthermore, by comparing the side lengths of the detected objects, it is possible to distinguish between squares and rectangles.

[0042] In some embodiments, an object recognition algorithm can detect whether an object is contained within another object by comparing its relative position to another object. For example, it can determine that square 806 is inside rectangle 804 and square 810 is inside square 806. Similar determinations can be made for square 808 and pentagon 812. Thus, an object recognition algorithm can use the number of sides of an object and whether it is contained within one or more other objects when recognizing / detecting it.

[0043] In some embodiments, the relative position of an object is determined based on the object's position identifier. The position can be determined. For example, an image can be thought of as a two-dimensional grid, where a first position identifier value can define a position in the first dimension of the grid (e.g., horizontal), and a second position identifier can define a position in the second dimension of the grid (e.g., vertical). Thus, a position in an image can be defined by a coordinate pair (e.g., (x,y)). The position of a given object can be defined using several different rules, such as the object's center point (e.g., single-point definition), the object's vertices (e.g., multi-point definition), or other preferred definitions. Embodiments of object recognition algorithms can also use, for example, one of these rules to detect the position of a detected object. In some embodiments, when the object's position is defined using vertices (or when some other multi-point definition indicating a region is used), the region within the object can be easily determined. Considering multiple detected objects and their positions, the object recognition algorithm can determine whether one object overlaps another or contains another object.

[0044] In some embodiments, an object recognition algorithm or function can detect / recognize objects using object definitions. For example, the number of sides may be used in the definition, such as a triangular object being defined as a load balancer object and a circular object being defined as a database object. In some embodiments, the object definition may also include whether or not the object is contained within another object. For example, a rectangular object that is not contained within another object may be defined as a network. Also, a square object contained within a rectangular object (or an object recognized as a network) may be defined as a subnetwork. A square or pentagonal object contained within a square object (or an object recognized as a subnetwork) may be defined as a virtual machine instance. In another example, a square or pentagonal object contained within two objects (e.g., a network and a subnetwork) may be defined as a virtual machine instance. These are merely illustrative object definitions, and other suitable definitions may be implemented.

[0045] In some embodiments, similar algorithms can be used to detect the arrangement of detected objects. For example, an object with one edge (e.g., a line) can be defined as a connection between objects. In some embodiments, once objects are recognized and the position of each object is known, these positions can be analyzed to determine the relative positions of the objects and whether any of the objects are contained within another object (or within multiple objects).

[0046] In some embodiments, a hierarchy indicating the arrangement of recognized objects can be determined. For example, consider two squares, square A and square B (square B is inside square A). In square A, it can be determined that it has no parent because it is the outermost shape. In this case, square A may be given an index of 0 because it is the outermost shape, and the index of its parent may be -1. Then, it can be determined that square B has a parent (e.g., square A - index 0), and its parent has no parent (e.g., the index of the parent's parent is -1). Using this, it can be determined that square B is a child of square A and is nested one level deeper. Thus, a hierarchy indicating the nesting levels of objects and their parents can be generated.

[0047] In some embodiments, the network architecture elements described with reference to Figure 6 are used. An object recognition algorithm can be constructed using the input received from the user. For example, the system may be notified that it should find a compute instance, and the system may be further configured to recognize that such a compute instance is located within a subnet, and that this subnet is located within a network. Thus, it is understood that objects nested at level 2 (within the network object and within the subnet object) should be recognized. In some embodiments, the discovered placement of an object may be based on the discovered hierarchy of the object.

[0048] The embodiments also include the detection / recognition of connection objects such as lines. In some embodiments, a line may be expected to have two endpoints (for example, not intersecting with other lines). Lines, such as lines representing load balancer connections, may be used to connect various other objects. In some embodiments, lines can be tied to network elements based on their proximity to their endpoints (for example, a load balancer can be connected to a virtual machine / subnet that receives traffic from the load balancer).

[0049] Similar to object hierarchy recognition, line recognition can be configured using user-received input that defines network architecture elements. For example, if the input defines that there should be two load balancer connections (e.g., the load balancer directs traffic to two different VMs), object detection / recognition could be configured to look for a processed image containing two lines whose endpoints are 1) near (e.g., very close to) a load balancer object (e.g., an outer layer triangle) and 2) near (e.g., very close to) two different compute instance objects (e.g., two different nested squares). As with other object detection / recognition functions such as hierarchy determination, the relative positions of objects can be used to detect which objects are close to the line endpoints. In some embodiments, one or more of these functions can be performed using libraries from open-source tools (e.g., OpenCV). For example, findCountours() from OpenCV can be used to return a hierarchy matrix based on the detected shapes.

[0050] In some embodiments, based on these object definitions (for example, defined as the number of sides, and optionally, whether an object is contained within another object), each of the objects recognized in Figure 8 can be associated with a network element. For example, triangle 802 can be associated with a load balancer, rectangle 804 can be associated with a network, and circle 818 can be associated with a database. Also, squares 806 and 808 can be associated with subnetworks (for example, located within rectangle 804 associated with a network). Squares 810 and pentagon 812 can be associated with virtual machine instances (for example, squares 806 and 808 associated with subnetworks, as well as located within rectangle 804 associated with a network). Also, lines 814 and 816 can be recognized as connections between the load balancer represented by triangle 802 and the subnetworks represented by squares 806 and 808.

[0051] In some embodiments, object recognition is performed based on input parameters received along with the image. For example, object recognition can be validated against input parameters described with reference to Figure 6. In some embodiments, the number of recognized objects associated with each network element type is validated against the number of each network element type defined in the input. For example, input parameter 602 in Figure 6 includes the number of network elements represented in image 502 in Figure 5. In this configuration, the input parameters 602 consist of one network, two subnetworks, two virtual machine instances, one load balancer, two connections, and one database. By verifying the recognized objects and their associated network element types against these numbers in Figure 8, we can confirm that the computer vision tool 102 in Figure 1 accurately determined the network architecture.

[0052] In some embodiments, other object definitions, recognized objects, and placements can be performed similarly. Thus, based on image processing, object definition, and object recognition capabilities, the network elements and network architecture represented in diagram 302 of Figure 3 and image 502 of Figure 5 can be detected by the computer vision tool 102 of Figure 1.

[0053] In some embodiments, recognized objects and recognized placements can be used to generate one or more configuration files for provisioning actual network elements. For example, one or more data files can be generated for each network element associated with each recognized object. Figures 9A and 9B show configuration files for provisioning a computing architecture according to an exemplary embodiment. In some embodiments, data 902 and 904 may be part of a data file (for example, a JavaScript Object Notation ("JSON") file) that holds configuration information about the network elements being processed and the relationships between them.

[0054] For example, data file 902 includes a database input "adbs" denoted as adb0 and a load balancer input "lbs" denoted as lb0, where the load balancer input "lbs" indicates connections to instance0 / subnet0 and instance1 / subnet1. Data file 902 also shows a virtual network input "vcns", which is further detailed by data file 904. For example, the virtual network defined in data file 904 has two subnets denoted as subnet1 and subnet0, each of which has virtual machine instances denoted as instance0 and instance1. Data file 904 also specifies that the shapes of instance0 and instance1 are rectangles and pentagons, respectively. Data file 904 indicates that the virtual network is denoted as vnc0.

[0055] The embodiment provides this configuration data file to a provisioning tool that provisions the network architecture defined in the file. For example, computing resources can be provisioned in cloud infrastructure (e.g., Oracle Cloud Infrastructure) using Terraform by HasiCorp. Terraform (sometimes referred to as “Infrastructure as Code”) can read a properly formatted file (e.g., formatted according to JSON or HasiCorp’s proprietary language, i.e., HasiCorp Configuration Language (“HCL”)) and provision the infrastructure based on the definitions in the file. For example, it can generate an execution plan that describes which functions will be performed to provision the computing resources defined in the configuration file, and then execute this plan to build the infrastructure described. In some embodiments, a graph of computing resources can be built, and the provisioning of non-dependent resources can be performed in parallel. The embodiment may also include a Terraform provisioner that can provision the infrastructure by executing a set of scripts and / or commands. Thus, Using a configuration data file, you can launch a set of processes, scripts, and / or commands (for example, using Terraform) to provision the computing resources defined in the configuration file. Other suitable provisioning products, services, or functions can be used in a similar manner.

[0056] Figures 10 to 16 show user interfaces related to provisioning a computing architecture according to an exemplary embodiment. For example, Figure 10 shows user interface 1002, which states that virtual machine instances 1004 and 1006 are being provisioned. Figure 11 further shows information about virtual machine instance 1006, including shape 1204, virtual cloud network 1206, and private IP address 1208.

[0057] Figure 12 further shows information about virtual machine instance 1004, including shape 1204, virtual cloud network 1206, and private IP address 1208. Figure 12 also shows the public IP address 1210 and subnet 1212 for virtual machine instance 1004, and the user interface 1102 may include similar information about virtual machine instance 1006.

[0058] The user interface 1302 in Figure 13 shows the provisioned and available virtual network 1304. The user interface 1402 in Figure 14 shows the provisioned and available subnets within the virtual network 1304, such as subnets 1404 and 1406. Figure 14 also shows the classless inter-domain routing blocks CIDR 1408 and 1410 for the subnets. Certain default settings can also be used to provision computing resources. For example, the default settings can be used to provision route tables 1412 and 1416 and security lists 1414 and 1418 for the subnets. In some embodiments, an internet gateway can also be created by default, which can provide connectivity to the internet for the provisioned computing resources. For example, the route tables (e.g., route tables 1412 and 1416) can be automatically configured to route traffic (e.g., outbound traffic) through that internet gateway.

[0059] Figure 15 shows a user interface 1502 describing a provisioned active load balancer 1504 and a public IP address 1506 for the load balancer. Figure 16 shows a user interface 1602 describing a provisioning database 1604. In some embodiments, network elements may take different amounts of time to provision, and once the network elements are provisioned, the network architecture may become fully functional and available.

[0060] Figure 17 is a flowchart for provisioning computing resources using computer vision according to an exemplary embodiment. In some embodiments, the functions of Figure 17 are performed by software stored in memory or other computer-readable or tangible media and executable by a processor. In other embodiments, each function may be performed by hardware (for example, by using application-specific integrated circuits ("ASICs"), programmable gate arrays ("PGAs"), field-programmable gate arrays ("FPGAs"), etc.), or by any combination of hardware and software. In embodiments, the functions of Figure 17 are executable by one or more elements of the system 200 of Figure 2.

[0061] In 1702, an image containing multiple visual elements can be received. For example, an image containing a diagram of a network architecture can be received. This diagram may include the arrangement of visual elements representing network elements and visual elements representing the network architecture.

[0062] In 1704, inputs associated with an image containing several network elements of a network element type can be received. For example, input parameters corresponding to visual / network elements represented in a diagram can be received. In some embodiments, the network element type may include one or more of databases, load balancers, networks, subnetworks, and / or virtual machine instances. The input parameters may include the number of each network element type present in the diagram (e.g., one load balancer, two databases, two virtual machines, etc.).

[0063] In 1706, an image can be processed using one or more transformations. For example, one or more image processing techniques can be performed on an image to perform one or more transformations. In some embodiments, the image processing technique can adjust the pixel values ​​of the image. The image processing technique may include masks, filters, convolutions, cross-correlations and other suitable image processing techniques.

[0064] In 1708, based on the above processing, visual elements within an image can be recognized as objects representing network elements, and the recognized objects represent network elements of a network element type. For example, an object can be recognized based on multiple object definitions, and each object definition can be associated with at least one of the network element types.

[0065] In some embodiments, the object definition includes at least the number of sides detected. The object definition may also include shapes such as one or more of triangles, rectangles, squares, circles, or pentagons.

[0066] In some embodiments, object recognition is performed based on the input. For example, object recognition can be validated against the input so that the number of recognized objects associated with each network element type is validated against the number of each network element type defined in the input.

[0067] In some embodiments, the arrangement of recognized objects representing a network architecture is recognized. For example, a first portion of an image may include at least one first visual element which includes at least two second visual elements, and a network architecture for provisioned network elements based on the first portion of the image may include a network which includes two subnetworks. In this example, a first recognized object corresponding to a first visual element may be associated with a network, a second recognized object corresponding to a second visual element may be associated with a subnetwork, and the recognized arrangement between the first and second objects may be associated with a network which includes a subnetwork.

[0068] In another example, the second part of the image may include at least a third visual element visually connected to the second visual element, and the network architecture for the provisioned network elements based on the second part of the image may include a load balancer that balances the load between two subnetworks. In this example, the third visual element corresponds to the third perception The object may be associated with a load balancer, and the recognized arrangement between the third object and the second object may be associated with the connection between the load balancer and the two subnetworks.

[0069] In 1710, multiple network element configuration files can be generated based on recognized objects, and these configuration files can be used to provision network elements associated with the recognized objects. For example, the configuration files may be data files generated according to a predetermined protocol. In some embodiments, the provisioning tool may be configured to receive the data files and perform the provisioning of network elements.

[0070] In some embodiments, one or more of the network element configuration files are defined based on the recognized placement of recognized objects. For example, the relationships between network elements corresponding to recognized objects can be represented by the placement of the recognized objects, and these relationships can be defined in the generated data files.

[0071] In 1712, computing resources can be provisioned in response to recognized objects by provisioning network elements that include one or more network element types. For example, provisioned network elements may correspond to recognized objects such that the number of each provisioned network element type corresponds to the number of recognized objects associated with each network element type.

[0072] In some embodiments, provisioning includes generating a network architecture for provisioned network elements based on known placements. For example, known placements of known objects can represent subnets within a network, virtual machines within a network or subnet, connections between load balancers and networks, subnets, virtual machines, or other load balancers. Provisioning may include provisioning network elements according to their relationships with one another (e.g., represented by known placements). In some embodiments, provisioning computing resources includes provisioning network elements according to a network architecture within a cloud infrastructure.

[0073] The features, structures, or characteristics described throughout this Specification may be combined in any preferred manner in one or more embodiments. For example, the use of “one embodiment,” “several embodiments,” “a particular embodiment,” “a particular set of embodiments,” or other similar expressions throughout this Specification refers to the fact that a particular feature, structure, or characteristic described in relation to such embodiment may be included in at least one embodiment of this Disclosure. Thus, the appearance of phrases such as “one embodiment,” “several embodiments,” “a particular embodiment,” “a particular set of embodiments,” or other similar expressions throughout this Specification does not necessarily refer to the same set of embodiments, and the described features, structures, or characteristics may be combined in any preferred manner in one or more embodiments.

[0074] Those skilled in the art will readily understand that the embodiments described above may be carried out in a different order of steps and / or with elements of a configuration different from that disclosed. Therefore, while this disclosure considers the embodiments outlined, certain modifications, alterations and alternative structures may become apparent while remaining within the spirit and scope of this disclosure. It will be obvious to those skilled in the art that this is the case. Therefore, to determine the boundaries of this disclosure, one should refer to the appended claims.

Claims

1. A method for provisioning computing resources using computer vision, The steps include receiving an image that has multiple visual elements, The process includes the step of recognizing the visual elements in the image as objects representing network elements, The recognized object represents a network element of a certain network element type, the network element type comprising one or more of a database, load balancer, subnetwork, or virtual machine instance. The arrangement of the recognized objects representing the network architecture is recognized, The above method further, The provisioning step includes provisioning computing resources in response to the recognized object by provisioning a network element having one or more of the aforementioned network element types, the provisioning step further includes generating a network architecture for the provisioned network element based on the recognized placement. The above method further, A method comprising the step of receiving an input associated with an image having several network elements of each network element type, wherein object recognition is performed based on the input.

2. The method according to claim 1, wherein the object is recognized based on a plurality of object definitions, and each object definition is associated with at least one of the network element types.

3. The method according to claim 2, wherein the object definition comprises at least the number of detected edges.

4. The method according to claim 3, wherein the object definition comprises one or more of a triangle, rectangle, square, circle, or pentagon.

5. The method according to claim 2, wherein the provisioned network elements correspond to the recognized objects such that the number of each provisioned network element type corresponds to the number of recognized objects associated with each network element type.

6. The method according to claim 5, wherein the first portion of the image comprises at least one first visual element comprising at least two second visual elements, and the network architecture for the provisioned network element based on the first portion of the image comprises a network comprising two subnetworks.

7. The method according to claim 6, wherein a first recognized object corresponding to the first visual element is associated with a network, a second recognized object corresponding to the second visual element is associated with a subnetwork, and a recognized arrangement between the first and second objects is associated with a network comprising a subnetwork.

8. The method according to claim 7, wherein the second portion of the image comprises at least a third visual element visually connected to the second visual element, and the network architecture for the provisioned network element based on the second portion of the image comprises a load balancer for balancing the load between the two subnetworks.

9. The method according to claim 8, wherein the third recognized object corresponding to the third visual element is associated with a load balancer, and the recognized arrangement between the third object and the second object is associated with the connection between the load balancer and the two subnetworks.

10. The method according to claim 2, wherein the step of provisioning the computing resources comprises the step of provisioning the network elements in accordance with the network architecture within the cloud infrastructure.

11. The method according to claim 2, further comprising the step of generating a plurality of network element configuration files based on the recognized object, wherein the configuration files are used to provision the network elements associated with the recognized object, and one or more of the network element configuration files are defined based on the recognized placement of the recognized object.

12. The method according to claim 1, wherein the object recognition is validated against the input so that the number of recognized objects associated with each network element type is validated against the number of each network element type defined in the input.

13. The method according to claim 2, further comprising the step of processing the image using one or more transformations, wherein the object recognition is performed based on the processed image.

14. A program for causing a computer to perform the method described in any one of claims 1 to 13.

15. A memory storing the program described in claim 14, A system comprising a processor for executing the aforementioned program.

Images