Provisioning using computer vision

Abstract

To provide a system and method for provisioning computing resources using computer vision.SOLUTION: A method comprises: receiving an image comprising a plurality of visual elements; and recognizing the visual elements within the image as objects that represent network elements. The recognized objects have network element types such as a database, a load balancer, a sub-network, and a virtual machine instance and comprise one or more of these types of network elements. The method also comprises: recognizing an arrangement of the recognized objects that represents a network architecture; and provisioning computing resources corresponding to the objects by provisioning network elements that comprise one or more of the network element types. The provisioning generates a network architecture for the network elements based on the recognized arrangement.SELECTED DRAWING: Figure 17

Description

Technical Field

[0001] Field Embodiments of the present disclosure generally relate to provisioning computing resources using computer vision.

Background Art

[0002] Background Computing services implemented by modern service providers are often required to be robust, flexible, and highly available. For example, modern demands for computing often arise from the opportunity costs associated with inefficient or unavailable computing resources. Some settings enable on-demand provisioning of computing resources, such as those of cloud-based service providers, but this provisioning often involves multi-step processes, manual labor, and other inconveniences. Given the stringent demands of modern computing, tools that can efficiently and quickly provide available computing resources would be beneficial to tool users.

Summary of the Invention

Means for Solving the Problems

[0003] Summary Embodiments of the present disclosure are generally directed to systems and methods for provisioning computing resources using computer vision. An image comprising a plurality of visual elements can be received. The visual elements in the image can be recognized as objects representing network elements, and the recognized objects represent network elements of a certain network element type, where the network element type comprises one or more of a database, a load balancer, a subnetwork, or a virtual machine instance, and an arrangement of the recognized objects representing a network architecture is recognized. By provisioning network elements comprising one or more of the network element types, computing resources can be provisioned corresponding to the recognized objects, and the provisioning comprises generating a network architecture for the provisioned network elements based on the recognized arrangement.

[0004] The features and advantages of the embodiments are described in, or will be apparent from, the following description and can be recognized by practicing the present disclosure.

[0005] Further embodiments, details, advantages, and modifications will become apparent from the following detailed description of the preferred embodiments when read in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9A

Figure 9B

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

[0007] Detailed Description Embodiments provision computing resources using computer vision. For example, an image including a diagram of a computer architecture can be received. This diagram can include visual elements representing network computing elements such as virtual machines, load balancers, databases, sub-networks, and other suitable computing elements. For example, some contours can be detected for the visual elements in the received image, and objects can be recognized based on the detected contours.

[0008] In some embodiments, the arrangement of the visual elements can also be detected, and this arrangement represents an architecture for computing resources. Based on the detected objects and their arrangements, computing resources can be provisioned. For example, one or more virtual machines, load balancers, databases, sub-networks, and other suitable computing elements can be provisioned.

[0009] In some embodiments, one or more generated configuration files can be used to provision computing resources. For example, the configuration file , can be generated based on the detected objects and their arrangements. In some embodiments, the provisioning tool can be configured to provision computing resources according to one or more generated configuration files. For example, a configuration file can be generated using the detected arrangement such that the network architecture based on the detected arrangement is defined by the configuration file. Based on these definitions, the computing resources to be provisioned later can be orchestrated according to the network architecture.

[0010] Here, embodiments of the present disclosure are referred to in detail. Examples of the embodiments are shown in the accompanying drawings. The following detailed description sets forth numerous specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits are not described in detail so as not to unnecessarily obscure aspects of the embodiments. As far as possible, like reference numerals are used for like elements.

[0011] FIG. 1 is a diagram showing a system for provisioning computing resources using computer vision according to an exemplary embodiment. System 100 includes a computer vision tool 102, an image capture device 104, and a diagram 106. In some embodiments, diagram 106 can represent a network architecture. For example, a predefined correspondence between visual elements and network computing elements can be defined, and diagram 106 can include the arrangement of these visual elements. In some embodiments, the arrangement of these visual elements can represent a network architecture for the network elements represented in diagram 106.

[0012] In some embodiments, the image capture device 104 can be used to capture an image of the diagram 106. For example, the diagram 106 can be drawn on a surface such as a whiteboard, physical paper, blackboard, display, or other suitable surface. In some embodiments, the diagram 106 can be generated as part of a brainstorming session, meeting, or other discussion (e.g., a pre - discussion or planning meeting). For example, team members can discuss computing requirements for a project, service, product, or other implementation, and based on that discussion, the team can generate (e.g., draw by hand) the diagram 106. Then, the image capture device 104 can be used to capture an image of the diagram 106. Other embodiments can include other techniques for generating the diagram 106 and / or capturing an image of the diagram 106.

[0013] In some embodiments, the captured image can be received by the computer vision tool 102. For example, the image capture device 104 can provide the image to the computer vision tool 102 (e.g., transmit the image to the computing device that implements the computer vision tool 102). Embodiments include other suitable techniques for receiving images of computer network architectures. The computer vision tool 102 can process the image to recognize objects within the image. For example, the image can be transformed using one or more filters or transforms, and the objects can be recognized by the computer vision tool 102 based on this processed image. In some embodiments, the recognized objects can be visual elements within the diagram 106, such as visual elements that include a predefined correspondence with network computing elements. The placement of the objects can also be recognized, and this placement represents the network architecture for the network computing elements. Embodiments perform object detection and / or object recognition functions and algorithms, although these terms are used throughout this disclosure in the same sense.

[0014] In some embodiments, network computing elements corresponding to the detected / recognized objects can be provisioned. For example, computer vision tool 102 can cause the provisioning of these network computing elements. In some embodiments, one or more configuration files can be generated by computer vision tool 102 and used to provision network computing elements. For example, the provisioning tool can be configured to provision computing resources according to the generated configuration files. In some embodiments, the provisioned computing resources can be part of a cloud computing environment, and the provisioning tool can provision network computing elements within the cloud. For example, the network computing elements can include one or more of virtual machines, sub-networks, load balancers, databases, etc.

[0015] In some embodiments, the generated configuration files can be based on the detected placement of the detected objects in the processed image. For example, the placement of the detected objects can represent a load balancer that balances the load between two sub-networks. In some embodiments, the generated configuration files can be configured such that the provisioned computing elements include the relationships represented by the detected placement (e.g., a load balancer provisioned according to the configuration file is configured to balance the load between two sub-networks).

[0016] Embodiments provide several advantages compared to conventional computing resource provisioning techniques. Conventional computing resource provisioning tools often require cumbersome requirement definitions, such as clearly defining the relationship between the shape of computing resources, instances of virtual machines or databases, and network elements such as load balancers, among numerous requirements. For example, in many cases, users have to navigate through multiple user interface pages to enter all the required information. In some cases, users also have to generate data files containing this information.

[0017] Embodiments of the present disclosure provide an improvement in speed and efficiency for usability by reducing the user effort and complexity of requirement definition. For example, a predefined correspondence between visual elements and network elements enables defining computing resource requirements using simple diagrams. In many cases, creating a simple diagram is part of a flow separate from provisioning, such as a business flow (or other technical flow) for determining how to implement a service or product. Embodiments can reuse these simple diagrams for provisioning, thus reducing the excessive effort associated with prior art for defining resource requirements.

[0018] Also, the detected placement of these visual elements, as performed by some embodiments, further reduces complexity and further improves speed for usability. For example, embodiments enable provisioning advanced networks of computing resources, such as networks having several sub-networks, connections, and heterogeneous elements. Embodiments of provisioning computer vision tools provide a robust solution that can improve the entire provisioning process even for complex computer resource realizations.

[0019] The troublesome nature of traditional provisioning can be particularly cumbersome with cloud resources. For example, modern cloud service providers are required to provide efficient, fast, and flexible technologies for realizing computing resources. This is often the result of the opportunity cost of technologies that are very expensive for organizations (e.g., especially in terms of time, money, and brand reputation), time-consuming, inefficient, or cumbersome. Therefore, the embodiments used to provision cloud resources would be particularly useful considering the stringent requirements for modern cloud service providers.

[0020] FIG. 2 is a block diagram of a computer server / system 200 according to an embodiment. All or part of system 200 may be used to implement any of the elements shown in FIG. 1. As shown in FIG. 2, system 200 may include a bus device 212 and / or other communication mechanisms configured to exchange information among various components of system 200, such as a processor 222 and a memory 214. Also, a communication device 220 may enable a connection between processor 222 and other devices by encoding data sent from processor 222 to another device via a network (not shown) and decoding data received from another system via the network for processor 222.

[0021] For example, communication device 220 may include a network interface card configured to provide wireless network communication. Various wireless communication technologies may be used, including infrared communication, wireless communication, Bluetooth® communication, Wi-Fi communication, and / or cellular communication. Alternatively, communication device 220 may be configured to provide a wired network connection, such as an Ethernet® connection.

[0022] Processor 222 may include one or more general-purpose or special-purpose processors to execute calculations and control the functions of system 200. Processor 222 may include a single integrated circuit such as a microprocessing device, or may include multiple integrated circuit devices and / or circuit boards that cooperate to implement the functions of processor 222. Also, processor 222 may execute computer programs such as operating system 215, computer vision tool 216, and other applications 218 stored in memory 214.

[0023] System 200 may include a memory 214 for storing information and instructions for execution by processor 222. Memory 214 may include various components for retrieving, displaying, modifying, and storing data. For example, memory 214 may store software modules that provide functions when executed by processor 222. These modules may include an operating system 215 that provides the operating system functions of system 200. These modules may include an operating system 215, a computer vision tool 216 that implements the computer vision and provisioning functions disclosed herein, and other application modules 218. The operating system 215 provides the operating system functions of system 200. In some examples, the computer vision tool 216 may be implemented as an in-memory configuration. In some implementations, when system 200 executes the functions of computer vision tool 216, it implements a non-conventional specialized computer system that executes the functions disclosed herein.

[0024] The non-transitory memory 214 may include various computer-readable media that can be accessed by the processor 222. For example, the memory 214 may include any combination of random access memory (“RAM”), dynamic RAM (“DRAM”), static RAM (“SRAM”), read-only memory (“ROM”), flash memory, cache memory, and / or other types of non-transitory computer-readable media. The processor 222 is further coupled to a display 224, such as a liquid crystal display (“LCD”), via a bus 212. A keyboard 226 and a cursor control device 228, such as a computer mouse, are further coupled to the communication device 212 to enable a user to connect to the system 200.

[0025] In some embodiments, the system 200 may be part of a larger system. Thus, the system 200 may include one or more additional functional modules 218 to include additional functionality. Other application modules 218 may include, for example, various modules of Oracle® Cloud Infrastructure, Oracle® Cloud Platform, Oracle® Cloud Applications. The computer vision tool 216, other application modules 218, and other suitable components of the system 200 may include various modules of Terraform by HashiCorp, OpenCV, other suitable open source tools, or other suitable Oracle® products or services.

[0026] The database 217 is coupled to the bus 212 to provide centralized storage for the modules 216 and 218 and store data received, for example, by the computer vision tool 216 or other data sources. The database 217 can store data within an integrated collection of logically related records or files. The database 217 may be an operational database, an analytical database, a data warehouse, a distributed database, an end-user database, an external database, a navigational database, an in-memory database, a document-oriented database, a real-time database, a relational database, an object-oriented database, a non-relational database, a NoSQL database, a Hadoop (registered trademark) distributed file system ("HFDS"), or any other database known in the art.

[0027] Although the functionality of system 200 is shown as a single system, it may be implemented as a distributed system. For example, memory 214 and processor 222 may be distributed across multiple different computers that together represent system 200. In one embodiment, system 200 may be part of a device (e.g., a smartphone, tablet, computer, etc.). In an embodiment, system 200 may be separate from the device and may provide the disclosed functionality remotely to the device. Further, one or more components of system 200 may be omitted. For example, for functionality as a user or consumer device, system 200 may be a smartphone or other wireless device that includes a processor, memory, and display and does not include one or more of the other components shown in FIG. 2 and includes additional components (such as an antenna, transceiver, or other suitable wireless device components) not shown in FIG. 2. Further, when system 200 is implemented to perform the functionality disclosed herein, it is a special-purpose computer specially adapted to provide demand forecasting.

[0028] Embodiments use computer vision to provision computing resources. Returning to FIG. 1, computer vision tool 102 can receive an image having a diagram representing computing elements of a network. FIG. 3 is a diagram showing a diagram of a computing architecture according to an exemplary embodiment. For example, diagram 302 may include visual elements such as shapes and connections representing computing network elements and a computing architecture. For example, diagram 302 may be drawn on a surface such as a whiteboard, physical paper, blackboard, display, or other suitable surface.

[0029] ​In some embodiments, a predefined correspondence between visual elements and network computing elements can be defined. For example, the visual elements represented in diagram 302 may include shapes, and the various shapes may correspond to various network computing elements. In some embodiments, the visual elements and their arrangements may represent a network architecture.

[0030] FIG. 4 is a diagram showing an exemplary computing architecture according to an embodiment. For example, diagram 302 may represent a network architecture 400, and the network architecture 400 may include a load balancer 402, a network 404, sub-networks 406 and 408, virtual machines 410 and 412, connections 414 and 416, and a database 418. In some embodiments, these network elements are represented by the shapes drawn in diagram 302. For example, the load balancer 402 can be represented by a triangle, the network 404 can be represented by a rectangle, the sub-networks 406 and 408 can be represented by rectangles included within another drawn network element (e.g., network 404), the virtual machine 410 can be represented by a rectangle included within another drawn network element (e.g., sub-network 406), the virtual machine 412 can be represented by a pentagon included within another drawn network element (e.g., sub-network 408), and the database 418 can be represented by a circle. In some embodiments, these representations are realized based on a predefined correspondence between the shapes (and / or the orientation of the shapes, such as being included within another shape) and the specific network elements they represent.

[0031] In some embodiments, the shape can represent a specific network computing entity such as a database or a type of virtual machine. For example, a circle representing database 418 can correspond to an autonomous database. Other shapes may represent other types of databases. Also, a rectangle contained within another visual element can represent virtual machine 410 or a first type of virtual machine, and a pentagon contained within another visual element can represent virtual machine 412 or a second type of virtual machine. In some embodiments, the differences between virtual machines can range from the number of CPU cores provisioned to the various operating systems running on the virtual machine, depending on how the system is configured. For example, virtual machine 410 may be VM Standard2.4, and virtual machine 412 may be VM Standard2.8. In some embodiments, the virtual machine options are determined and / or provided by the cloud vendor. FIGS. 3 and 4 show a specific correspondence between shapes and network elements, but other suitable definitions / representations may be implemented.

[0032] In some embodiments, diagram 302 includes an arrangement of visual elements. For example, the arrangement of visual elements can represent architecture 400. The visual elements can indicate that load balancer 402 uses connections 414 and 416 to connect to sub-networks 406 and 408. This representation can indicate that load balancer 402 balances the load between sub-networks 406 and 408. In the illustrated embodiment, sub-network 406 includes virtual machine 410, and sub-network 408 includes virtual machine 412. Thus, in the illustrated embodiment, load balancer 402 effectively balances the load between virtual machine 410 and virtual machine 412.

[0033] Returning to FIG. 1, the image of diagram 302 can be captured using the image capture device 104. In some embodiments, the captured image can be provided / transmitted to the computer vision tool 102. Other suitable implementations for receiving the image of the diagram in the computer vision tool 102 can also be used.

[0034] FIG. 5 is a diagram showing an image of a computing architecture according to an exemplary embodiment. For example, the image 502 may be an image of the diagram 302 from FIG. 3, and this image may further represent the architecture 400 from FIG. 4. In other words, the image 502 may include a diagram that includes visual elements representing network computing elements and the arrangement of visual elements representing the network architecture. In some embodiments, the image 502 is received / transmitted to the computer vision tool 102 of FIG. 1.

[0035] In some embodiments, the computer vision tool 102 may be software that receives an image, recognizes objects within the image, and provisions computer resources. For example, the computer vision tool may be compiled code (e.g., executable) or script code that receives the image 502 of FIG. 5 as an input. In some embodiments, additional information can be received along with the image 502. FIG. 6 is a diagram showing an input for a computing architecture according to an exemplary embodiment.

[0036] In some embodiments, the input parameter 602 may include the number of network elements represented within the image 502. For example, the input parameter 502 can be broken down by network element type (e.g., network, sub-network, virtual machine instance, load balancer, connection, database, etc.). In the illustrated embodiment, the input parameter 602 for the image 502 is one network, two sub-networks, two virtual machine instances, one load balancer, two connections, and one database. Also, FIG. 6 shows the computer vision tool 102 as a script (e.g., a Python script) that accepts the input parameter 602. The computer vision tool 102 may be in other forms of software, hardware, or any combination thereof.

[0037] In some embodiments, when the image 502 and the input parameter 602 are received by the computer vision tool 102, the image 502 is processed. For example, one or more filters or masks can be applied to the image 502 to transform the image. In some embodiments, a convolution or cross-correlation function can be used to adjust the pixel values of the image 502 to generate a processed image. In some implementations, multiple processed images can be generated (e.g., based on multiple filters, masks, convolutions, cross-correlations, and other image processing techniques). Some embodiments implement image processing using one or more open source tools (e.g., OpenCV) or other image processing tools.

[0038] FIG. 7 is a diagram showing a processed image of a computing architecture according to an exemplary embodiment. For example, one or more processing techniques can be executed to generate the processed image 702 from the image 502. In the illustrated embodiment, the color scheme of the image 502 has been changed (e.g., black and white within the image have been inverted), and other image quality adjustments have been performed to reach the processed image 702.

[0039] In some embodiments, the image processing algorithm can convert the initial image to black and white and further cycle through a plurality of image processing parameters such as the filters, masks, and transforms described herein, or any suitable techniques, to generate a processed image. For example, for one or more of the processed images, an object recognition algorithm can be used to attempt to detect a desired architecture provided by the user (e.g., defined by user input). If / detects the object recognition algorithm detects the desired architecture, the associated processed image can be considered to have been correctly processed by the current set of image processing parameters. If the object recognition algorithm does not detect the desired architecture, the processing parameters can be repeatedly changed, the image can be reprocessed with a new configuration, and the object recognition algorithm can be executed on the newly processed image. In some embodiments, this iterative technique can be repeated until the desired architecture is detected by the object recognition algorithm.

[0040] In some embodiments, object recognition is performed on one or more processed images. For example, the computer vision tool 102 can execute an object recognition algorithm or function on the processed image 702 to detect objects within the image. Some embodiments execute one or more open source tools (e.g., OpenCV) to perform object recognition. For example, a shape object recognition algorithm or function (e.g., the findContours function of OpenCV) or other suitable recognition protocol can be used.

[0041] Figure 8 is a diagram showing objects recognized in an image of a computing architecture according to an exemplary embodiment. For example, Figure 8 includes a triangle 802, a rectangle 804, squares 806 and 808, a square 810, a pentagon 812, lines 814 and 816, and a circle 818. In some embodiments, after an edge detection algorithm or function for detecting the edges of an object, a shape object recognition algorithm or function for recognizing the detected object can be executed, followed by an object recognition algorithm or function. For example, the detected objects shown in Figure 8 are based on the visual elements within diagram 302 of Figure 3. This is because the processed image is based on the imported image of the diagram. In the shown embodiment, since the visual elements in diagram 302 include shapes, an object recognition algorithm that can easily identify and distinguish shapes can be executed. For example, the shapes in diagram 302 and the objects shown in Figure 8 can be distinguished based on the number of sides they have (e.g., 0 for a circle, 1 for a line, 3 for a triangle, 4 for a square and a rectangle, and 5 for a pentagon). Also, a square and a rectangle can be distinguished by comparing the lengths of the sides of the detected objects.

[0042] In some embodiments, the object recognition algorithm can detect whether an object is contained within another object by comparing the relative positions of the objects. For example, it can be determined that square 806 is within rectangle 804 and square 810 is within square 806. Similar determinations can also be made for square 808 and pentagon 812. Thus, the object recognition algorithm can use the number of sides of the object and whether it is contained within one or more other objects when recognizing / detecting the object.

[0043] In some embodiments, relative to the object based on the object's position identifier The position can be determined. For example, an image can be considered as a two-dimensional grid, and the first position identifier value can define the position in the first dimension of the grid (e.g., horizontal), and the second position identifier can define the position in the second dimension of the grid (e.g., vertical). Thus, the position within the image can be defined by a coordinate pair (e.g., (x, y)). The position of a given object can be defined using several different rules such as the center point of the object (e.g., defined by one point), the vertices of the object (e.g., defined by multiple points), or other suitable definitions. Also, an embodiment of the object recognition algorithm can detect the position of the detected object using, for example, one of these rules. In some embodiments, when using the vertices to define the position of an object (or when any other multiple-point definition indicating a region is used), the region within the object can be easily determined. Considering multiple detected objects and their positions, the object recognition algorithm can determine whether one object overlaps another object or contains another object.

[0044] In some embodiments, an object recognition algorithm or function can detect / recognize an object using an object definition. For example, a triangular object can be defined as a load balancer object, a circular object can be defined as a database object, etc., and the number of sides may be used in the definition. In some embodiments, the object definition may also include whether an object is within another object. For example, a rectangular object that is not contained within another object can be defined as a network. Also, a square object within a rectangular object (or an object recognized as a network) can be defined as a subnet. A square object or a pentagonal object within a square object (or an object recognized as a subnet) can be defined as a virtual machine instance. In another example, a square object or a pentagonal object within two objects (e.g., a network and a subnet) can be defined as a virtual machine instance. These are merely exemplary object definitions, and other suitable definitions may be implemented.

[0045] In some embodiments, a similar algorithm can be used to detect the arrangement of the detected objects. For example, a side of one object (e.g., a line) can be defined as a connection between objects. In some embodiments, when an object is recognized and the position of each object is recognized, these positions can be analyzed to determine the relative positions of the objects and whether any object is contained within another object (or within a plurality of objects).

[0046] In some embodiments, a hierarchy indicating the arrangement of recognized objects can be determined. For example, consider two squares, Square A and Square B (Square B is inside Square A). In the case of Square A, since it is the outermost shape, it can be determined that it has no parent. In this case, Square A can be given an index of 0 as it is the outermost shape, and the index of its parent can be -1. Next, Square B has a parent (e.g., Square A - index 0), and it can be determined that its parent has no parent (e.g., the index of the grandparent is -1). Using this, it can be determined that Square B is a child of Square A and is nested one level deep. Thus, a hierarchy indicating the nesting level of an object and its parent can be generated.

[0047] In some embodiments, using the input received from the user that defines the network architecture elements described with reference to FIG. 6, an object recognition algorithm can be configured. For example, the system can be notified that a computing instance should be detected, and the system can be further configured to recognize that such a computing instance would be located within a subnet, and that this subnet would be located within a network. Thus, it can be seen that objects nested at level 2 (within the object that is the network and the object that is the subnet) should be recognized. In some embodiments, the detected arrangement of the objects can be based on the detected hierarchy of the objects.

[0048] ​Embodiments also include the detection / recognition of connection objects such as lines. In some embodiments, a line may be expected to have two endpoints (e.g., not intersecting other lines). Lines such as those representing load balancer connections may be used to connect various other objects. In some embodiments, a line can be associated with a network element based on the proximity to the endpoints of the line (e.g., a load balancer can be connected to a virtual machine / subnet that receives traffic from the load balancer).

[0049] Similar to the recognition of the object hierarchy, the recognition of lines can be configured using the input received from the user that defines the network architecture elements. For example, if the input defines that there should be two load balancer connections (e.g., the load balancer is directing traffic to two different VMs), the object detection / recognition can be configured to look for a processed image that includes two lines having endpoints near 1) a load balancer object (e.g., an outer triangle) (e.g., right next to it) and 2) two different compute instance objects (e.g., two different nested squares) (e.g., right next to it). Similar to other object detection / recognition functions such as hierarchy determination, the relative positions of the objects can be used to detect which objects are close to the endpoints of the line. In some embodiments, one or more of these functions can be performed using libraries from open source tools (e.g., OpenCV). For example, findCountours() from OpenCV can be used to return a hierarchy matrix based on the detected shapes.

[0050] In some embodiments, based on these object definitions (e.g., defined as the number of sides and optionally whether the object is included in another object), each of the objects recognized in FIG. 8 can be associated with a network element. For example, triangle 802 can be associated with a load balancer, rectangle 804 can be associated with a network, circle 818 can be associated with a database. Also, squares 806 and 808 can be associated with a subnetwork (e.g., located within rectangle 804 associated with the network). Squares 810 and pentagon 812 can be associated with virtual machine instances (e.g., located within squares 806 and 808 associated with the subnetwork and within rectangle 804 associated with the network). Also, lines 814 and 816 can be recognized as connections between the load balancer represented by triangle 802 and the subnetwork represented by squares 806 and 808.

[0051] In some embodiments, object recognition is performed based on input parameters received with the image. For example, object recognition can be verified against the input parameters described with reference to FIG. 6. In some embodiments, the number of recognized objects associated with each network element type is verified against the number of each network element type defined in the input. For example, input parameter 602 in FIG. 6 includes the number of network elements represented in image 502 in FIG. 5. In the illustrated embodiment the input parameter 602 is one network, two subnets, two virtual machine instances, one load balancer, two connections and one database. By verifying the objects recognized in FIG. 8 and their associated network element types against these numbers, it can be confirmed that computer vision tool 102 in FIG. 1 has correctly determined the network architecture.

[0052] In some embodiments, other object definitions, recognized objects, and placements can be executed as well. Thus, based on the image processing, object definition, and object recognition functions, the network elements and network architecture represented in diagram 302 of FIG. 3 and image 502 of FIG. 5 can be detected by the computer vision tool 102 of FIG. 1.

[0053] In some embodiments, one or more configuration files can be generated for provisioning actual network elements using the recognized objects and recognized placements. For example, for each network element associated with a recognized object, one or more data files can be generated. FIGS. 9A-9B are diagrams showing configuration files for provisioning a computing architecture according to an exemplary embodiment. In some embodiments, data 902 and 904 may be part of a data file (e.g., a JavaScript Object Notation (“JSON”) file) that holds configuration information about the network elements to be processed and the relationships between them.

[0054] For example, data file 902 includes a database input "adbs" denoted as adb0 and a load balancer input "lbs" denoted as lb0, and the load balancer input "lbs" indicates connections to instance0 / subnet0 and instance1 / subnet1. The data file 902 also shows a virtual network input "vcns", which is further detailed by data file 904. For example, the virtual network defined in data file 904 has two subnets denoted as subnet1 and subnet0, each of which has virtual machine instances denoted as instance0 and instance1. Also, data file 904 shows the shapes of instance0 and instance1 as a rectangle and a pentagon respectively. Data file 904 indicates that the virtual network is denoted as vcn0.

[0055] The embodiment provides this configuration data file to a provisioning tool that provisions a network architecture defined in a file. For example, using Terraform by HashiCorp, computing resources can be provisioned in a cloud infrastructure (e.g., Oracle Cloud Infrastructure). Terraform (sometimes referred to as "Infrastructure as Code") can read in the state of a properly formatted file (e.g., one formatted according to JSON or HashiCorp's proprietary language, namely HashiCorp Configuration Language ("HCL")) and provision infrastructure based on the definitions within the file. For example, an execution plan can be generated that describes what functions will be executed to provision the computing resources defined in the configuration file, and then this plan can be executed to build the described infrastructure. In some embodiments, a graph of computing resources can be constructed and the provisioning of independent resources can be executed in parallel. The embodiment can also include a Terraform provisioner that can execute a set of scripts and / or commands to provision infrastructure. Thus, using the configuration data file, a set of processes, scripts, and / or commands (e.g., using Terraform) can be launched to provision the computing resources defined in the configuration file. Other suitable provisioning products, services, or functions can be executed similarly.

[0056] Figures 10-16 illustrate user interfaces related to provisioning a computing architecture according to an exemplary embodiment. For example, FIG. 10 illustrates a user interface 1002, which states that virtual machine instances 1004 and 1006 are being provisioned. FIG. 11 further shows information about virtual machine instance 1006, such as shape 1204, virtual cloud network 1206, and private IP address 1208.

[0057] FIG. 12 further shows information about virtual machine instance 1004, such as shape 1204, virtual cloud network 1206, and private IP address 1208. FIG. 12 also shows a public IP address 1210 and a subnet 1212 for virtual machine instance 1004, and user interface 1102 may include similar information about virtual machine instance 1006.

[0058] The user interface 1302 in FIG. 13 shows the provisioned available virtual network 1304. The user interface 1402 in FIG. 14 shows the provisioned available subnets within the virtual network 1304, such as subnets 1404 and 1406. Also, FIG. 14 shows the classless inter-domain routing blocks CIDR 1408 and 1410 for the subnets. Certain default settings can also be used to provision computing resources. For example, the default settings can be used to provision route tables 1412 and 1416 and security lists 1414 and 1418 for the subnets. In some embodiments, an Internet gateway can also be created by default, and this Internet gateway can provide an Internet connection to the provisioned computing resources. For example, the route tables (e.g., route tables 1412 and 1416) can be automatically configured to route traffic (e.g., outgoing traffic) through that Internet gateway.

[0059] FIG. 15 shows a user interface 1502 that describes the provisioned active load balancer 1504 and the public IP address 1506 for the load balancer. FIG. 16 shows a user interface 1602 that describes the provisioning database 1604. In some embodiments, network elements may require different times to be provisioned, and when the network elements are provisioned, the network architecture can become fully functional and available.

[0060] FIG. 17 is a flowchart for provisioning computing resources using computer vision according to an exemplary embodiment. In some embodiments, the functionality of FIG. 17 is performed by software stored in memory or other computer-readable or tangible media and executable by a processor. In other embodiments, each functionality may be performed by hardware (e.g., by using an application specific integrated circuit (ASIC), a programmable gate array (PGA), a field programmable gate array (FPGA), etc.), or any combination of hardware and software. In an embodiment, the functionality of FIG. 17 is executable by one or more elements of the system 200 of FIG. 2.

[0061] At 1702, an image including a plurality of visual elements can be received. For example, an image including a diagram of a network architecture can be received. This diagram may include visual elements representing network elements and the arrangement of visual elements representing the network architecture.

[0062] At 1704, an input associated with an image including some network elements of a network element type can be received. For example, input parameters corresponding to the visual elements / network elements represented in the diagram can be received. In some embodiments, the network element type may include one or more of a database, a load balancer, a network, a subnetwork, and / or a virtual machine instance. The input parameters may include the number of each network element type present in the diagram (e.g., one load balancer, two databases, two virtual machines, etc.).

[0063] In 1706, one or more transformations can be used to process an image. For example, to perform one or more transformations, one or more image processing techniques can be executed on the image. In some embodiments, the image processing technique can adjust the pixel values of the image. The image processing technique can include masks, filters, convolutions, cross-correlations, and other suitable image processing techniques.

[0064] In 1708, based on the above processing, visual elements within the image can be recognized as objects representing network elements, and the recognized objects represent network elements of a network element type. For example, the objects can be recognized based on a plurality of object definitions, and each object definition can be associated with at least one of the network element types.

[0065] In some embodiments, the object definition includes at least the number of detected sides. The object definition can also include shapes such as one or more of triangles, rectangles, squares, circles, or pentagons.

[0066] In some embodiments, object recognition is performed based on the input. For example, object recognition can be verified against the input such that the number of recognized objects associated with each network element type is verified against the number of each network element type defined in the input.

[0067] In some embodiments, the arrangement of recognized objects representing a network architecture is recognized. For example, a first portion of an image may include at least one first visual element that includes at least two second visual elements, and the network architecture for provisioned network elements based on the first portion of the image may include a network that includes two sub-networks. In this example, a first recognized object corresponding to the first visual element may be associated with the network, a second recognized object corresponding to the second visual element may be associated with the sub-network, and the recognized arrangement between the first object and the second object may be associated with the network comprising the sub-network.

[0068] In another example, a second portion of an image may include at least a third visual element visually connected to the second visual element, and the network architecture for provisioned network elements based on the second portion of the image may include a load balancer that balances the load between two sub-networks. In this example, a third recognized object corresponding to the third visual element may be associated with the load balancer, and the recognized arrangement between the third object and the second object may be associated with the connection between the load balancer and the two sub-networks. In 1710, a plurality of network element configuration files can be generated based on the recognized objects, and these configuration files can be used to provision network elements associated with the recognized objects. For example, the configuration file may be a data file generated according to a predetermined protocol. In some embodiments, the provisioning tool may be configured to receive the data file and execute the provisioning of network elements.

[0069]

[0070] In some embodiments, one or more of the network element configuration files are defined based on the recognized placement of the recognized objects. For example, the relationships between network elements corresponding to the recognized objects can be represented by the placement of the recognized objects, and these relationships can be defined in the generated data file.

[0071] In 1712, computing resources can be provisioned corresponding to the recognized objects by provisioning network elements that include one or more of the network element types. For example, the provisioned network elements can correspond to the recognized objects such that the number of each provisioned network element type corresponds to the number of recognized objects associated with each network element type.

[0072] In some embodiments, provisioning includes generating a network architecture for the provisioned network elements based on the recognized placement. For example, the recognized placement of the recognized objects can represent sub-networks within a network, networks or virtual machines within a sub-network, connections between a load balancer and a network, sub-network, virtual machine or other load balancer, etc. Provisioning can include provisioning network elements according to their relationships with each other (e.g., as represented by the recognized placement). In some embodiments, provisioning computing resources includes provisioning network elements according to the network architecture within a cloud infrastructure.

[0073] The features, structures, or characteristics of the present disclosure described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, the use throughout this specification of "one embodiment," "some embodiments," "certain embodiments," "certain multiple embodiments," or other similar expressions refers to the fact that the particular features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of the present disclosure. Accordingly, the appearances of the phrases "one embodiment," "some embodiments," "certain embodiments," "certain multiple embodiments," and the like, or other similar expressions throughout this specification do not necessarily all refer to the same group of embodiments, and the features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0074] Those skilled in the art will readily understand that the above embodiments may be implemented in different orders of steps and / or with elements of configurations different from the disclosed configurations. Accordingly, while the present disclosure contemplates the embodiments outlined, it will be apparent to those skilled in the art that specific variations, modifications, and alternative structures are apparent while staying within the spirit and scope of the present disclosure. It will be apparent to those skilled in the art that, accordingly, reference should be made to the appended claims to determine the bounds of the present disclosure.

Claims

1. 1. A method for provisioning computing resources using computer vision, comprising: receiving an image comprising a plurality of visual elements; recognizing the visual elements in the image as objects representing network elements; the recognized object represents a network element of a network element type, the network element type comprising one or more of a database, a load balancer, a sub-network, or a virtual machine instance; A placement of the recognized objects representing a network architecture is recognized; The method further comprises: provisioning computing resources corresponding to the recognized objects by provisioning network elements comprising one or more of the network element types, the provisioning comprising generating a network architecture for the provisioned network elements based on the recognized placement; The method further comprises: A method comprising receiving an input associated with the image comprising several network elements of each network element type, wherein the object recognition is performed based on the input.

2. The method of claim 1 , wherein the object is recognized based on a plurality of object definitions, each object definition being associated with at least one of the network element types.

3. The method of claim 2 , wherein the object definition comprises at least a number of detected edges.

4. The method of claim 3 , wherein the object definition comprises one or more of a triangle, a rectangle, a square, a circle, or a pentagon.

5. 3. The method of claim 2, wherein the provisioned network elements correspond to the recognized objects such that the number of provisioned network element types corresponds to the number of recognized objects associated with each network element type.

6. 6. The method of claim 5, wherein the first portion of the image comprises at least one first visual element that includes at least two second visual elements, and the network architecture for the provisioned network element based on the first portion of the image comprises a network including two sub-networks.

7. 7. The method of claim 6, wherein a first recognized object corresponding to the first visual element is associated with a network, a second recognized object corresponding to the second visual element is associated with a sub-network, and a recognized arrangement between the first object and the second object is associated with a network comprising the sub-network.

8. 8. The method of claim 7, wherein the second portion of the image comprises at least a third visual element visually connected to the second visual element, and the network architecture for the provisioned network element based on the second portion of the image comprises a load balancer that balances load between the two sub-networks.

9. 9. The method of claim 8, wherein a third recognized object corresponding to the third visual element is associated with a load balancer, and a recognized arrangement between the third object and the second object is associated with a connection between the load balancer and the two sub-networks.

10. The method of claim 2 , wherein provisioning the computing resources comprises provisioning the network elements according to the network architecture in a cloud infrastructure.

11. 3. The method of claim 2, further comprising generating a plurality of network element configuration files based on the recognized objects, the configuration files being used to provision the network elements associated with the recognized objects, one or more of the network element configuration files being defined based on the recognized placement of the recognized objects.

12. 2. The method of claim 1, wherein the object recognition is validated against the input such that a number of recognized objects associated with each network element type is validated against a number of each network element type defined in the input.

13. The method of claim 2 , further comprising processing the image using one or more transforms, and wherein the object recognition is performed based on the processed image.

14. A program for causing a computer to execute the method according to any one of claims 1 to 13.

15. a memory storing the program according to claim 14; a processor for executing the program.

Images