A system for provisioning of remote processing

Abstract

A system for provisioning of remote processing comprising: a controller, a plurality of processing units, PUs; and wherein the controller is configured to: receive a request for processing capacity from a user, wherein the request for processing capacity includes a processing requirement demand; the controller is configured to: determine which of the PUs are available for processing; and determine which combination of available PUs is capable of meeting the processing requirement demand based on predetermined capability information; and assign PUs to provide processing for the request for processing capacity based on the determination of which combination of available PUs is capable of meeting the processing requirement demand; and provide for the provisioning of remote processing by providing for communication between the user and the assigned combination of PUs to enable the user to perform remote processing using the combination of assigned PUs.

Description

[0001]A SYSTEM FOR PROVISIONING OF REMOTE PROCESSING Field The present disclosure relates to a system for the provisioning of remote processing to a user. Developers are often forced to spend significant time and resources setting up an infrastructure tailored for new machine learning and artificial intelligence development. Further, as projects grow or data is increased, scaling of infrastructure becomes a bottleneck for machine learning developers, requiring manual intervention and hardware upgrades. Even once a trained model is read for deployment, translating that model into a production-ready solution is not straightforward in most working environments. The present disclosure provides a solution in which these problems, and others, are addressed by providing for provisioning of remote processing to a user. According to a first aspect of the present disclosure, there is provided a system for provisioning of remote processing comprising: a controller;a plurality of processing units, PUs; and wherein the controller is configured to: receive a request for processing capacity from a user, wherein the request for processing capacity includes a processing requirement demand; the controller is further configured to, in response to the receipt of the request for the processing capacity: determine which of the PUs are available for processing; and determine which combination of available PUs is capable of meeting the processing requirement demand based on predetermined capability information associated with each processing unit; and the controller is further configured to: assign processing units to provide processing for the request for processing capacity based on the determination of which combination of available PUs is capable of meeting the processing requirement demand; and provide for the provisioning of remote processing by providing for communication between the user and the assigned combination of processing units to enable the user to perform remote processing using the combination of assigned processing units. In one or more embodiments, the controller may be configured to: receive a request for suspension of the provisioning of remote processing wherein the PUs used in the provisioning of remote processing comprise data thereon being used by the user; store at least a subset of the data on the PUs to a memory in response to the request for suspension; and after storing the subset of data, remove the communication between the combination of PUs and the user. In one or more embodiments, each PU may be one of an intelligence processing unit, IPU, a graphics processing unit, GPU, and a computer processing unit, CPU. In one or more embodiments, the request for processing capacity may further comprise a request for one or more of: a PU type requirement, a computing architecture requirement, a computing time requirement, a memory type requirement, a memory size requirement, a speed of processing requirement, a speed of data transfer requirement, a software requirement, a compiler type requirement, an encryption protocol requirement, a geographical location requirement, a user access requirement, a user identity requirement, a storage capacity requirement, a cluster size requirement, a software development kit requirement, and access to one or more additional machines requirement. In one or more embodiments, the controller may be configured to provide access to the processing units which provide for the provisioning of remote processing to the user or one or more additional users which have been provided with access permission by the user. In one or more embodiments, the virtual provisioning of remote processing may be implemented by a plurality of PUs in a single processing on demand unit, POD. In one or more embodiments, there is provided the system wherein, during the provisioning of remote processing to the user by providing for access to the combination of PUs as a result of a processing capacity request, the controller may be configured to receive an updated processing capacity request from the user wherein the updated processing capacity request comprises an updated processing requirement demand different from the processing requirement demand of the initial processing requirement request and wherein the controller is configured to, in response to the updated processing capacity request: determine which of the PUs are available for processing and wherein, additionally, the PUs that are being currently in use for the provisioning of remote to the user are considered to be available for processing for the purpose of providing an updated provisioning of remote processing to the user; and determine which combination of PUs is, capable of meeting the updated processing requirement demand based on predetermined capability information associated with each processing unit; and the controller is further configured to: assign the combination of processing units to provide processing for the updated request for processing capacity based on the determination of which combination of available PUs is capable of meeting the updated processing requirement demand; and provide for the updated provisioning of remote processing by providing for communication between the user and the assigned combination of processing units to enable the user to perform remote processing using the combination of assigned processing units. In one or more embodiments, the provisioning of remote processing may comprise initiating and providing for access to a virtual processing environment wherein the virtual processing environment is implemented using the assigned combination of processing units. In one or more embodiments, the system may be configured to, upon initiating the virtual processing environment, installing one or more software programs in the virtual processing environment. One or more embodiments will now be described by way of example only with reference to the accompanying drawings in which: Figure 1 shows an example embodiment of a system of the present disclosure. The present disclosure provides for a system configured to provide for the provisioning of remote processing to a user. By providing for remote provisioning as disclosed, a user is provided with the opportunity to operate in a virtual processing environment which can provide the resources necessary for seamless development. Even without the use of the virtual processing environment itself, the user is provided with the remote processing capacity to carry out necessary tasks based on a user’s request for processing capacity. The remote processing capacity is managed dynamically by a controller of the system and, as such, the system is able to make efficient use of a plurality of individual processing units. This autonomous management of the plurality of individual processing units allows different demands to be met without relying on pre-partitioned sets of processing units. By moving away from pre- partitioned sets of processing units, fewer processing units can be used to meet the demand of a plurality of users, thereby saving space and energy. This approach also allows the system to react to a request for increased processing capacity from a user, thereby more efficiently managing a development project as it grows from conception to prototype through to final product. Figure 1 shows an example system 100 for the provisioning of remote processing to a user 102. The system 100 comprises a controller 102 configured to receive a request for processing capacity from a user 101. The request for processing capacity includes a processing requirement demand. The controller 102 may be any suitable controller 102 which is capable of receiving a request from a user 101 and interacting with a plurality of processing units 103 based on that request to put into place the provisioning of remote processing. The controller 102 may be implemented in software or hardware. In one or more embodiments, the controller 102 may be a server such as a management server. In other embodiments, the controller 102 may be a collection of units which work together to provide the functionality of the controller 102 described herein. The system 100 further comprises a plurality of processing units (PUs) 103. The processing units 103 may be any suitable processing units 103 for providing remote provisioning to a user 101. For example, the processing units 103 may be central processing units (CPUs), graphical processing units (GPUs) or intelligent processing units (IPUs). In one or more embodiments, the processing units 103 may specifically be IPUs. IPUs may be particularly beneficial, particularly in the context of AI development projects, as IPUs are specifically designed to handle tasks common in AI projects. The request for processing capacity from a user 101 may be received in any form suitable for interpretation by the controller 102. The user 101 may be a user who is located remotely from both the controller and the PUs. For example, the user 101 may be a user working at home or in an office while the controller 102 may be located at a processing centre located in the same or even a different country. The processing units 103 may be co-located with the controller 102 or they may be located at a separate location. In one or more embodiments, the processing units 103 may be distributed between a plurality of different geographic locations. The request for processing capacity includes a processing requirement demand which indicates the amount of processing capacity required by the user 101. The processing requirement demand may be in the form of an amount of required memory or any other suitable way of defining a required processing capacity. The request for processing capacity may include additional requests in addition to the processing requirement demand. For example, the request for processing capacity may include a PU type requirement in which the user specifies the type of processing unit they want from a predetermined list of available processor types such as CPUs, GPUs or IPUs. The request for processing capacity may comprise a computing architecture requirement that generally defines one or more hardware, software requirements and interconnection requirements between the hardware and software. The request for processing capacity may comprise a computing time requirement which defines the period over which access to the provisioning for remote processing is provided to the user 101. The request for processing capacity may comprise any one or more of a memory type requirement, a memory size requirement, a speed of processing requirement or a speed of data transfer requirement. The request for processing capacity may comprise a software requirement which defines one or more types of software which are made available to the user 101 via the provisioning for remote processing. The request for processing capacity may further comprise a software development kit requirement wherein the software development kit comprises a predetermined set of software, APIs, tools, libraries and documentation that is configured to operate efficiently together. By providing for the option of predetermined SDKs, time may be saved by the user 101 in setting up an environment in which they can pursue development, thereby allowing for development to get started sooner. The request for processing capacity may comprise a compiler type requirement. The request for processing capacity may comprise an encryption protocol requirement. The request for processing capacity may comprise a geographical location requirement. For example, a user 101 may request that the PUs 103 that provide for the provisioning of remote processing are provided from a particular country, city or some other defined geographical location. This may be done in order to reduce latency or for other reasons specific to the user. In one or more alternative embodiments, the user 101 may not be able to select a geographical location of the PUs and, instead, the controller 102 may be configured to assign the geographical location from which the PUs 103 are to be provided based on one or more factors, such as for reduced latency time or based on instantaneous capacity of the PUs 103 from across the plurality of PUs 103. The request for processing capacity may comprise a user access requirement. For example, the user access requirement may define whether access to the provisioning for remote processing should be provided as a public network or as a private network. The request for processing capacity may comprise a user identity requirement. For example, the user identity requirement may include a user access verification requirement to ensure that the user 101 accessing provisioning of remote processing is a designated user or a permitted user from a plurality of permitted users. The user identity requirement may further include credential information so that the controller can properly identify that a user 101 attempting to access the provisioning for remote processing is the designated user or one of the permitted users. The controller 102 may further be configured to provide access to the processing units 103 which provide for provisioning of the remote processing to the user 101 or one or more additional users which have been provided with access permission by the user 101. The request for processing capacity may comprise a storage capacity requirement. The storage capacity requirement may define the amount of non- volatile memory required by the user 101. The request for processing capacity may comprise an access to additional machines requirements such as a type of configuration and a software development kit (SDK) version. In response to the receipt of the request for processing capacity, the controller 102 is configured to determine which of the PUs 103 are available for processing. This may be performed by contacting and awaiting a response from each of the PUs 103 in order to identify which PUs 103 are available for processing. In other embodiments, the controller 102 may be configured to maintain an availability record wherein the availability record stores, for each PU 103, an indication of whether the PU 103 is available for processing or unavailable for processing. In the embodiments where an availability record is maintained, the controller 102 may be configured to update the availability record whenever one or more PUs 103 are assigned to a user 101. The controller 102 may be further configured to update the availability record whenever a user session with a plurality of assigned PUs 103 is ended. Once the available PUs 103 have been determined, the controller is further configured to determine which combination 104 of available PUs is capable of meeting the processing requirement demand based on predetermined capability information associated with each processing unit 103. This capability information may be information about the processing capacity of each respective processing unit 103. In some embodiments, all of the processing units 103 may have the same, or substantially the same, processing capacity. In other embodiments, the PUs 103 might have different processing capacities and it may be possible to select a combination 104 of PUs which accurately meet the processing requirement demand. The controller 102 may further be configured to meet one or more other parameters of the request for processing capacity, such as those parameters already listed above. For example, the controller 102 may further be configured to selecting processing units 103 that are all in the same geographical location. In one or more embodiments, the processing units 103 selected by the controller 102 may be within a same processing on demand, PoD, unit. A PoD may be a bare metal unit which holds a plurality of PUs 103. The provision of remote processing by the selected PUs 103 within the same PoD may be provided based on a specific request in the request for processing capacity provided by the user 101. In other embodiments, the default approach taken by the controller 102 may be to prioritise or ensure that the selected processing units are physically located within the same PoD. Computational efficiency may be improved by way of selecting PUs 103 within the same PoD. Where it is not possible to provide all of the PUs 103 for the provision of remote processing from a single PoD, the controller 102 may be configured to select PUs 103 from PoDs that are in close proximity to one-another, such as those in the same geographical location and, preferably, an adjacent PoD in an area with a plurality of PoDs. Thus, one or more parameters that the controller 102 utilises to select the combination of PUs 103 may be set by one of the request for processing capacity from the user 101 or the one or more parameters may be predetermined by the controller 102 as a default approach. The controller 102 is further configured to assign processing units 103 to provide processing for the request for processing capacity based on the determination of which combination 104 of available PUs is capable of meeting the processing requirement demand. Assigning the combination 104 of processing units may comprise partitioning the processing units 103 such that they are only in communication with each other and not with any other processing units of the plurality of processing units 103. The partition may be a virtual partition. Assigning the combination 104 of processing units may also comprise adjusting an availability record to indicate that the selected PUs 103 are no longer available for processing. Assigning the combination 104 of processing units may comprise initiating a virtual processing environment using the PUs 103. The virtual processing environment may be configured to emulate the hardware and software resources of a physical computer. Initiating the virtual processing environment may comprise installing one or more software programs on the PUs to form the virtual processing environment. Further, initiating the virtual processing environment may comprise installing one or more software development kits in the virtual processing environment. Providing for this pre-installation prior to providing access to the user 101 may ensure that the user 101 is able to access the virtual processing environment and get working on development as quickly as possible without the need to do set-up themselves. Once the combination 104 of processing units has been assigned and, where necessary, any set up has been performed, the controller 102 is configured to provide for the provisioning of the remote processing to the user 101 by providing for communication between the user 101 and the assigned combination 104 of processing units. This is performed to enable the user 101 to perform remote processing using the combination of assigned processing units. In one or more embodiments, one or more pieces of software or parts of an SDK may be installed in the virtual processing environment after the provisioning for remote processing has been provided to the user 101. For example, certain features may not be essential to the user 101 getting started and so their installation may be delayed in preference to providing the user with access to the provisioning for remote processing sooner. In one or more embodiments, the controller 102 may be further configured to receive a request for suspension of the provisioning of remote processing from the user 101. Suspension of the provisioning of remote processing may entail temporarily disabling the user’s access to remote processing and, subsequently, being able to re-enable that access. When the provisioning of remote processing is re-enabled, the user 101 may be able to continue where they left off but, unbeknownst to them, they may be operating with a different combination 104 of PUs. During the provisioning of remote processing, data is stored on the PUs 103 in order for the user to perform tasks. The controller 102 is configured to, in response to receipt of the request for suspension of the provisioning of remote processing, store at least a subset of the data on the PUs 103 to a memory in response to the request for suspension. The memory may be a transient or non-transient memory suitable to store the data for the duration of the suspension. It will be appreciated that only a subset of the data may be helpful or relevant for re-establishment of the user’s access to provisioning of remote processing and, as such, only a subset of the data may be stored. In one or more alternative embodiments, the controller 102 may be configured to store all of the data on the PUs 103 to the memory. After storing at least the subset of the data, the controller 102 may be configured to remove the communication between the combination 104 of PUs and the user 101. Where an availability record is maintained by the controller 102, the controller 102 may also be configured to designate the PUs 103 of the combination 104 of PUs as available for processing. Where an availability record is not maintained, the PUs 103 will be able to indicate that they are available for processing in response to any of the controller’s 102 requests about availability after the communication between the user and the PUs 103 of the combination 104 of PUs has been removed. Thus, in this way, a user 101 can request suspension of a session and releasing the PUs 103 can allow the PUs 103 to be returned to the pool of available PUs 103 for other users. The controller 102 is further configured to receive a request for resumption of access to processing capacity by a user 101 who has suspended their session. Upon receiving the resumption of access request from the user 101, the controller 102 may be configured to perform the same or similar steps to those used to initiate the provisioning of remote processing to the user 101. In particular, the controller 102 may determine which of the PUs 103 are available for processing; determine which combination 104 of available PUs is capable of meeting the processing requirement demand of the request for resumption based on predetermined capability information associated with each processing unit 103; assign processing units 103 to provide processing for the request for processing capacity based on the determination of which combination 104 of available PUs is capable of meeting the processing requirement demand and provide for the provisioning of remote processing by providing for communication between the user 101 and the assigned combination 104 of processing units to enable the user 101 to perform remote processing using the combination 104 of assigned processing units. Upon resumption of the provisioning of remote processing to the user, before or after the user is put in communication with the combination of PUs, the controller may be configured to retrieve the stored subset of data and load it onto the combination of PUs for use by the user. This may allow for a smooth reinitiation of the user’s suspended session. In some embodiments, in the case of a request for resumption of access to processing capacity 103, the processing requirement demand may be, by default, the same as the pre-suspension processing requirement demand made when the request for processing capacity was made. In other embodiments, the user 101 may be able to specify a new processing requirement demand which may be greater or less than the processing requirement demand of the initial processing requirement demand. In this way, the user 101 may be able to easily take a break from development and then, upon resumption of development, either scale up or scale down the required processing resources. Similarly to the case of suspension and resumption, during the provisioning of remote processing to the user 101 by providing for access to the combination 104 of PUs as a result of a processing capacity request, the controller 102 may be configured to receive an updated processing capacity request from the user 101. The updated processing capacity request may comprise an updated processing requirement demand which is different from the processing requirement demand of the initial processing requirement request. In response to receiving the updated processing capacity request, the controller 102 may be configured to determine which of the PUs 103 are available for processing and also which PUs 103 are currently in use for the provisioning of remote processing to the user 101. The PUs 103 which are either available or currently in use by the user 101 may be considered to be available for processing for the purposes of the updated processing capacity request, since those currently being used by the user 101 can remain assigned to that user 101. The controller 102 may further be configured to determine which combination 104 of available PUs is capable of meeting the updated processing requirement demand based on predetermined capability information associated with each processing unit 103. The predetermined capability information will generally be the capability information that was previously used for the original request for the provisioning of remote processing. The controller 102 may further be configured to assign a combination 104 of the processing units to provide processing for the updated request for processing capacity based on the determination of which combination 104 of available PUs is capable of meeting the updated processing requirement demand. Finally, the controller 102 may be configured to provide for the updated provisioning of remote processing by providing for communication between the user 101 and the assigned combination 104 of processing units to enable the user 101 to perform remote processing using the combination 104 of assigned processing units. In this way, the user 101 is able to request an increase or decrease in the processing capacity that they have available to them during a session and have that processing capacity updated in a smooth manner. Transitioning to the updated provisioning of processing capacity may further involve data transfer from one or more PUs 103 of the first combination 104 of PUs to the second combination of PUs. This may be particularly necessary where at least a subset of the PUs selected for the updated combination of PUs are not the same as at least a subset of the PUs of the original combination 104 of PUs. Simultaneously or separately, the controller 102 may be configured to receive an updated processing capacity request which comprises an update to one or more of the other parameters which have been discussed herein.

Claims

CLAIMS 1. A system for provisioning of remote processing comprising: a controller; a plurality of processing units, PUs; and wherein the controller is configured to: receive a request for processing capacity from a user, wherein the request for processing capacity includes a processing requirement demand; the controller is further configured to, in response to the receipt of the request for the processing capacity: determine which of the PUs are available for processing; and determine which combination of available PUs is capable of meeting the processing requirement demand based on predetermined capability information associated with each processing unit; and the controller is further configured to: assign processing units to provide processing for the request for processing capacity based on the determination of which combination of available PUs is capable of meeting the processing requirement demand; and provide for the provisioning of remote processing by providing for communication between the user and the assigned combination of processing units to enable the user to perform remote processing using the combination of assigned processing units.

2. The system of claim 1 wherein the controller is configured to: receive a request for suspension of the provisioning of remote processing wherein the PUs used in the provisioning of remote processing comprise data thereon being used by the user; store at least a subset of the data on the PUs to a memory in response to the request for suspension; and after storing the subset of data, remove the communication between the combination of PUs and the user.

3. The system of any preceding claim, wherein each PU is one of an intelligence processing unit, IPU, a graphics processing unit, GPU, and a computer processing unit, CPU.

5. The system of any preceding claim wherein the request for processing capacity further comprises a request for one or more of: a PU type requirement, a computing architecture requirement, a computing time requirement, a memory type requirement, a memory size requirement, a speed of processing requirement, a speed of data transfer requirement, a software requirement, a compiler type requirement, an encryption protocol requirement, a geographical location requirement, a user access requirement, a user identity requirement, a storage capacity requirement, a cluster size requirement, a software development kit requirement, and access to one or more additional machines requirement.

6. The system of any preceding claim wherein the controller is configured to provide access to the processing units which provide for the provisioning of remote processing to the user or one or more additional users which have been provided with access permission by the user.

7. The system of any preceding claim wherein the virtual provisioning of remote processing is implemented by a plurality of PUs in a single processing on demand unit, POD.

8. The system of any preceding claim wherein, during the provisioning of remote processing to the user by providing for access to the combination of PUs as a result of a processing capacity request, the controller is configured to receive an updated processing capacity request from the user wherein the updated processing capacity request comprises an updated processing requirement demand different from the processing requirement demand of the initial processing requirement request and wherein the controller is configured to, in response to the updated processing capacity request: determine which of the PUs are available for processing and wherein, additionally, the PUs that are being currently in use for the provisioning of remote to the user are considered to be available for processing for the purpose of providing an updated provisioning of remote processing to the user; and determine which combination of PUs is, capable of meeting the updated processing requirement demand based on predetermined capability information associated with each processing unit; and the controller is further configured to: assign the combination of processing units to provide processing for the updated request for processing capacity based on the determination of whichcombination of available PUs is capable of meeting the updated processing requirement demand; and provide for the updated provisioning of remote processing by providing for communication between the user and the assigned combination of processing units to enable the user to perform remote processing using the combination of assigned processing units.

9. The system of any preceding claim wherein the provisioning of remote processing comprises initiating and providing for access to a virtual processing environment wherein the virtual processing environment is implemented using the assigned combination of processing units.

10. The system of claim 9 wherein the system is configured to, upon initiating the virtual processing environment, installing one or more software programs in the virtual processing environment.

Images