Shared Compilation Cache Verification System

Abstract

Exemplary embodiments of the present disclosure, in one exemplary aspect, provide an exemplary computer-implemented method for validating a shared cache. The exemplary method can include obtaining a pre-compiled shared cache entry corresponding to a shared cache key, the shared cache key associated with an operation request. The exemplary method can include obtaining a directly compiled resource associated with the operation request. The exemplary operation can include authenticating one or more portions of the shared cache based at least in part on a comparison of the pre-compiled shared cache entry and the directly compiled resource.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates generally to data caching and, more particularly, to validating shared cache entries. [Background technology]

[0002] Compute nodes can be operated in a run-time compilation manner where code to perform a requested operation can be compiled on demand. However, to avoid redundant compilation, a cache of precompiled binaries can be made available on the compute nodes. Then, when an operation is requested for the second or subsequent times, the code can be retrieved from the cache and executed on the compute node. Summary of the Invention

[0003] Aspects and advantages of embodiments of the disclosure will be set forth in part in the description that follows, or may be learned from the description, or may be learned by practice of the embodiments.

[0004] In one aspect, the present disclosure provides an exemplary computer-implemented method for authenticating a shared cache. The exemplary method includes obtaining, by a computing system including one or more processors, a pre-compiled shared cache entry corresponding to a shared cache key, the shared cache key associated with the operation request. The exemplary method includes obtaining, by the computing system, a directly compiled resource associated with the operation request. The exemplary method includes authenticating, by the computing system, one or more portions of the shared cache based at least in part on a comparison of the pre-compiled shared cache entry and the directly compiled resource.

[0005] In another aspect, the present disclosure provides an exemplary non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform exemplary operations. The exemplary operations include obtaining a pre-compiled shared cache entry corresponding to a shared cache key, the shared cache key associated with the operation request. The exemplary operations include obtaining a directly compiled resource associated with the operation request. The exemplary operations include authenticating one or more portions of the shared cache based at least in part on a comparison of the pre-compiled shared cache entry and the directly compiled resource.

[0006] In another aspect, the present disclosure provides an exemplary shared cache validation system. The exemplary system includes one or more processors. The exemplary system includes one or more non-transitory computer-readable media including instructions that, when executed, cause the one or more processors to perform exemplary operations. The exemplary operations include receiving a request for a pre-compiled shared cache entry from a shared cache from a computation node that processes the requested operation. The exemplary operations include obtaining an authentication status for the shared cache based at least in part on a comparison of (i) a direct compiled resource compiled by the computation node to perform the requested operation and (ii) the pre-compiled shared cache entry, the authentication status indicating incompatibility of the pre-compiled shared cache entry with the computation node. The exemplary operations include deactivating the shared cache for one or more future requests from the computation node.

[0007] Other aspects of the present disclosure are directed to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.

[0008] These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain associated principles.

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A detailed description, directed to one skilled in the art, is set forth herein with reference to the accompanying drawings. [Brief description of the drawings]

[0010] [Figure 1A] 1 illustrates a block diagram of an exemplary computing system that performs shared compilation cache validation in accordance with an exemplary embodiment of the present disclosure. [Figure 1B] 1 illustrates a block diagram of an exemplary computing device that performs shared compilation cache validation in accordance with an exemplary embodiment of the present disclosure. [Figure 1C] 1 illustrates a block diagram of an exemplary computing device that performs shared compilation cache validation in accordance with an exemplary embodiment of the present disclosure. [Diagram 2] 1 illustrates a block diagram of additional aspects of an exemplary shared compilation cache validation system, in accordance with an exemplary embodiment of the present disclosure. [Diagram 3] 1 illustrates a block diagram of an example data flow for aspects of an example shared compilation cache validation system, in accordance with an example embodiment of the present disclosure. [Figure 4] 1 illustrates a block diagram of an exemplary state-based scheme of an exemplary shared compilation cache validation system, in accordance with an exemplary embodiment of the present disclosure; [Diagram 5] 1 illustrates a block diagram of an exemplary data flow of an exemplary shared compilation cache verification system using a non-authenticating cache, in accordance with an exemplary embodiment of the present disclosure. [Figure 6]1 illustrates a block diagram of an exemplary data flow using an authentication cache of an exemplary shared compilation cache verification system, in accordance with an exemplary embodiment of the present disclosure. [Figure 7] 1 illustrates a block diagram of an exemplary data flow using an invalidation cache of an exemplary shared compilation cache verification system, in accordance with an exemplary embodiment of the present disclosure. [Figure 8] 2 illustrates a flowchart diagram of an exemplary method for performing shared cache validation, according to an exemplary embodiment of the present disclosure. [Figure 9] 1 illustrates a flowchart diagram of another exemplary method for performing shared cache validation, according to an exemplary embodiment of the present disclosure.

[0011] Reference numbers repeated among the drawings are intended to identify like features in the various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] overview In general, the present disclosure is directed to systems and methods for shared cache validation. For example, exemplary embodiments according to aspects of the present disclosure relate to techniques that may be used to manage a shared cache of compiled resources. For example, in some embodiments, when multiple computing nodes are configured to be used to perform the same or similar operations, a shared cache may be made available to these nodes, so that resources already compiled by one node may be reused by another node for performing the same or similar operations. However, in some scenarios, placement of the shared cache may be associated with a risk of incompatibility. For example, a shared cache entry compiled on one node may not be fully compatible or otherwise valid for execution on another node (e.g., due to hardware differences, firmware differences, software differences, configuration differences, etc.). Resource incompatibility may be associated with a risk of system failure, data loss, downtime, vulnerability, etc.

[0013] Advantageously, systems and methods according to example aspects of the present disclosure may provide improved validation of a shared cache. For example, in some embodiments, a shared cache may be authenticated in accordance with aspects of the present disclosure before it is trusted to serve cached resources among multiple endpoints. For example, in some embodiments, a shared cache may be assigned an authentication status (e.g., with a state including valid / authenticated, invalid, not authenticated, etc.). The authentication status may be maintained, updated, and / or reset based on a state-based approach.

[0014] The authentication status may, in some embodiments, be determined by validating one or more pre-compiled shared cache entries in the shared cache against one or more resources that have been directly compiled (e.g., compiled locally, compiled by a runtime compiler, etc.) for the desired endpoint. For example, a directly compiled counterpart compiled for or by a node may be referenced as an example of a resource that is compatible with that node. In this manner, for example, a pre-compiled shared cache entry may be compared to known compatible resources to verify that it is compatible.

[0015] For example, in some embodiments, a compute node may receive a request to perform an operation. The compute node may query a shared cache to obtain pre-compiled resources associated with the requested operation(s). In some cases, the compute node may also obtain resources directly compiled (e.g., compiled locally by the node, compiled for the node by a runtime compiler, etc.) for the requested operation(s) and compare the result(s) with the pre-compiled resources obtained from the shared cache. In some embodiments, based on the comparison (e.g., a positive comparison such as a match), the pre-compiled resources may be validated (e.g., indicated as compatible). In this manner, for example, the pre-compiled resources in the shared cache may be determined to be compatible and thus may be utilized to obtain improved performance across one or more compute nodes. However, in some embodiments, based on obtaining a negative comparison (e.g., an incompatibility is detected, etc.), the shared cache may be deactivated in whole or in part to avoid or reduce the risk of running incompatible resources on one or more connected compute node(s).

[0016] In some embodiments according to example aspects of the present disclosure, a pre-compiled cache entry may be stored in a shared cache in association with a shared cache key. The shared cache key may be configured as a means for retrieving the corresponding pre-compiled cache entry. For example, in some examples, a pre-compiled shared cache resource may be initialized and assigned a shared cache key that encodes one or more identifiers of the resource. In some embodiments, the identifier may link a request (e.g., a request for processing by a compute node) to a resource for performing the requested operation on the compute node.

[0017] For example, in some embodiments, the shared cache key may be configured to distinguish a target shared cache entry for performing a requested operation on a compute node from other entries (e.g., other resources that are not compatible with the compute node). In some embodiments, the shared cache key may include an abbreviated representation of the shared cache entry or a pointer to the shared cache entry. In some embodiments, for example, the abbreviated representation of the shared cache entry may include a hash or other encoding of one or more aspects or portions of the shared cache entry (e.g., a hash or other encoding of the entry's tag, label, title, structure, content, locator, etc.). In some embodiments, the abbreviated representation of the shared cache entry may include a hash or other encoding of one or more aspects or portions of a request for an operation to be performed by the shared cache entry. For example, in some embodiments, the shared cache key may be generated from a request for an operation on a compute node, and the key may be matched to a previously stored key associated with a pre-compiled shared cache entry that was previously compiled on the same or a similar (e.g., mutually compatible) compute node. In this manner, for example, given a request for an operation (e.g., a function call, etc.), a shared cache key may be obtained that is operable to obtain a resource from the shared cache that is configured to perform the requested operation. And in this way, for example, a shared cache key can be constructed to retrieve a compatible entry in the shared cache.

[0018] To help ensure that a shared cache key properly retrieves a compatible shared cache entry (e.g., configured to properly distinguish incompatibilities between nodes), systems and methods according to example aspects of the present disclosure may provide a shared cache credential. A shared cache may be associated with an authentication status indicator, for example, determined based on validation of one or more entries in the shared cache, or other credential.

[0019] For example, a shared cache entry obtained to perform an operation associated with a request may be compared to a directly compiled resource associated with the request. In some embodiments, a directly compiled resource may include a resource compiled locally by a node requesting a cached resource. In some embodiments, a directly compiled resource may be compiled by a runtime compiler (e.g., on the node, on a server servicing the node, etc.). In some examples, for example, a directly compiled resource may be compiled by a compiler configured to compile a resource compatible with the target node.

[0020] In some embodiments, a directly compiled resource may include, for example, executable code compiled to perform an operation associated with a request. A shared cache entry may include, for example, pre-compiled executable code that is also configured to perform an operation associated with a request.

[0021] The comparison of the directly compiled resource to the shared cache entry may include, for example, a comparison of fingerprints (e.g., hashes or other encodings that provide an identification) of the directly compiled resource and the shared cache entry, respectively. For example, in some embodiments, if one or more portions of the directly compiled resource and the shared cache entry correspond to equivalent (e.g., identical) hash values, the shared cache entry may be determined to be sufficiently interchangeable with the directly compiled resource such that the shared cache entry may be trusted for compatible execution on the compute node. In this manner, for example, one or more shared cache entries (e.g., and / or keys associated therewith) may be verified by a successful comparison (e.g., a comparison that satisfies one or more compatibility criteria) with one or more corresponding directly compiled resources.

[0022] In some embodiments, the fingerprints used in validating one or more shared cache entries can be configured to reveal differences between one or more shared cache entries and their directly compiled counterparts. For example, in some embodiments, the fingerprints can be configured to reflect the content of the resources (e.g., the shared cache entries, the directly compiled resources, etc.) in a granular manner. For example, the fingerprints can include a hash value or other encoding generated from the resources (e.g., the entire content of the resource, the operational content of the resource, etc.). In this manner, the fingerprints can advantageously reveal potential incompatibilities between the directly compiled resources and the shared cache entries, even if, for example, the shared cache entries were obtained by a shared cache key associated with both the directly compiled resources and the shared cache entries.

[0023] In some embodiments, for example, the fingerprint can be used to verify the authenticity of the composition of a shared cache key. For example, in the simplest example, consider a request for a compute node to perform operation A. The key can be some encoding of "operation A," and the compute node can query the shared cache with the key to retrieve a pre-compiled shared cache entry tagged with a key indicating "operation A." To test whether the pre-compiled shared cache entry is compatible with the compute node, the compute node can retrieve a directly compiled resource to perform the requested operation A. The directly compiled resource can be compared to the retrieved pre-compiled shared cache entry to verify. However, a comparison (e.g., hash value, other encoding, etc.) may reveal that the retrieved resource is different from the directly compiled resource, such that it is an invalid resource for the requesting node. For example, the retrieved pre-compiled resource may have been compiled by a node of type 1 to perform operation A, but the requesting node is of type 2, which is incompatible with the resource compiled by the node of type 1. Because the shared cache keying scheme failed to distinguish between resources compiled for type 1 nodes and resources compiled for type 2 nodes, one or more portions of the shared cache may be deauthenticated or otherwise invalidated to prevent invalid cached resources from being served.

[0024] Exemplary embodiments according to exemplary aspects of the present disclosure provide numerous technical effects and advantages. By providing improved validation of shared cache implementations, exemplary embodiments may utilize pre-compiled code across one or more individual nodes to provide improved performance of a computing system such that individual nodes are not unnecessarily burdened with duplicate compilation. For example, exemplary embodiments may provide increased speed, reduced latency, reduced network bandwidth requirements, reduced power consumption, reduced memory usage, etc., for one or more computing nodes (e.g., individually and / or collectively) by providing a secure and reliable means of deploying a shared cache system. For example, traditionally, when a new computing node comes online, that computing node may lack a pre-populated local cache, so operations performed on the new node may need to compile all operations on demand and may not generally benefit from improved processing of cached resources. However, according to exemplary embodiments of the present disclosure, an authenticated shared cache may be used to more efficiently populate the local cache of the new node, advantageously reducing the processing requirements (and associated lead times, etc.) for bringing the new node online. In another example, for example, an existing node may encounter a new task, and instead of having to directly compile resources to complete the task, an exemplary embodiment of a shared cache system according to the present disclosure can provide for supplying pre-compiled resources to the node to process the new task with improved speed and efficiency.

[0025] Another additional benefit, in some example embodiments, includes improved robustness against execution failures and / or other consequences of incompatible or unauthorized code. For example, in some examples, improved verification techniques according to aspects of the present disclosure may increase the reliability and / or security of a system of one or more computing nodes, resulting in increased uptime, fewer mis-execution events, reduced maintenance costs (e.g., computational costs of debugging, time and labor costs, etc.). In this manner, for example, fewer computing resources may be consumed to provide a target level of uptime, reliability, and / or security. Alternatively, or additionally, in some embodiments, improved uptime, reliability, and / or security may be achieved by more efficient use of computing resources (e.g., reduced network bandwidth requirements, reduced power consumption, reduced memory usage, etc.). In this manner, for example, functionality of example embodiments of the computing system itself may be improved by reducing the risk of failure while resulting in improvements in processing, latency, etc.

[0026] Referring now to the figures, exemplary embodiments of the present disclosure will be described in further detail.

[0027] Exemplary Devices and Systems 1A illustrates a block diagram of an exemplary computing system 100 for performing shared cache validation, in accordance with an exemplary embodiment of the present disclosure. The system 100 includes a computing device 102, a server computing system 130, and a training computing system 150, communicatively coupled via a network 180.

[0028] Computing device 102 may be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, a workstation, a computing endpoint of a distributed system, a server computing device, a host device for one or more virtual machines, or any other type of computing device.

[0029] The computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operably connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the computing device 102 to perform operations.

[0030] The memory 114 may also include a local cache 120 for storing compiled instructions (e.g., compiled instructions 118) for execution on the computing device 102. For example, a local compiler 122 may compile the instructions 118 (e.g., to perform one or more operations of the machine-learned model(s) 124). The one or more compiled instructions may be stored in the local cache 120 to provide reduced processing requirements and lower latency for one or more future executions of the instructions 118.

[0031] In some implementations, the computing device 102 may store or include one or more machine-learned models 124. For example, the machine-learned models 124 may be or may otherwise include various machine-learned models, such as neural networks (e.g., deep neural networks), or other types of machine-learned models including nonlinear and / or linear models. The neural networks may include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. Some exemplary machine-learned models may utilize attention mechanisms, such as self-attention. For example, some exemplary machine-learned models may include multi-headed self-attention models (e.g., transformer models).

[0032] In some implementations, one or more machine-learned models 124 may be received from the server computing system 130 over the network 180 and stored in the computing device memory 114 and then used or otherwise executed by one or more processors 112. In some implementations, the computing device 102 may implement multiple parallel instances of a single machine-learned model 124.

[0033] Additionally or alternatively, the one or more machine-learned models 144 may be included in or otherwise stored and implemented on a server computing system 130 that communicates with the computing device 102 according to a client-server relationship. For example, the machine-learned models 144 may be implemented by the server computing system 140 as part of a web service. Thus, the one or more models 124 may be stored and implemented on the computing device 102 and / or the one or more models 144 may be stored and implemented on the server computing system 130.

[0034] The server computing system 130 includes one or more processors 132 and a memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operably connected. The memory 134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0035] The memory 134 may also include a shared cache 140 for storing compiled instructions (e.g., compiled instructions 118, compiled instructions 138) for execution on the computing device 102, the server computing system 130, and / or the training computing system 150. For example, the server compiler 142 may compile the instructions 118 and / or the instructions 138 (e.g., to perform one or more operations of the machine-learned model(s) 124 and / or the machine-learned model(s) 144). The one or more compiled instructions may be stored in the shared cache 140 to provide reduced processing requirements and lower latency for one or more future executions of the instructions 118 and / or the instructions 138. In some embodiments, the shared cache 140 may be configured to provide one or more entries (e.g., resources, such as pre-compiled resources) to one or more other systems and devices via the network 180, as described herein. In some embodiments, the multiple computing devices 102 are each in communication with the server computing system 130 and are configured to receive pre-compiled resources from the shared cache 140. For example, system 100, in some embodiments, may coordinate compilation tasks among computing device 102, training computing system 150, and server computing system 130 to take advantage of shared cache 140 (and in some embodiments, a more powerful processor 132 and / or compiler 142).

[0036] In some implementations, server computing system 130 includes or is otherwise implemented by one or more server computing devices. When server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0037] As described above, the server computing system 130 may store or otherwise include one or more machine-learned models 144. For example, the models 144 may be or otherwise include a variety of machine-learned models. Exemplary machine-learned models include neural networks or other multi-layer nonlinear models. Exemplary neural networks include feed-forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some exemplary machine-learned models may utilize attention mechanisms such as self-attention. For example, some exemplary machine-learned models may include a multi-headed self-attention model (e.g., a transformer model).

[0038] The computing device 102 and / or the server computing system 130 may train the models 124 and / or 144 through interaction with a training computing system 150 that is communicatively connected via a network 180. The training computing system 150 may be separate from the server computing system 130 or may be part of the server computing system 130.

[0039] The training computing system 150 includes one or more processors 152 and memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operably connected. The memory 154 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some implementations, the training computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0040] The memory 154 may also include a local cache 160 for storing compiled instructions (e.g., compiled instructions 158) for execution by the training computing system 150. For example, a local compiler 162 may compile the instructions 158 (e.g., to perform one or more operations of the model trainer 164). One or more compiled instructions may be stored in the local cache 160 to provide reduced processing requirements and lower latency for one or more future executions of the instructions 158.

[0041] The training computing system 150 may include a model trainer 164 that trains the machine-learned models 124 and / or 144 stored on the computing device 102 and / or the server computing system 130 using various training or learning techniques, such as, for example, backpropagation. For example, a loss function may be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy error, hinge loss, and / or various other loss functions. Gradient descent may be used to iteratively update the parameters over a number of training iterations.

[0042] In some implementations, performing backpropagation may include performing truncated backpropagation through time. The model trainer 164 may perform some generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the trained model. In particular, the model trainer 164 may train the machine-learned models 124 and / or 144 based on the training dataset.

[0043] In some implementations, if consent is provided, the training examples may be provided by the computing device 102. Thus, in such implementations, the model 124 provided to the computing device 102 may be trained by the training computing system 150 with device-specific data received from the computing device 102. In some examples, this process may be referred to as personalizing the model.

[0044] Model trainer 164 includes computer logic utilized to provide desired functionality. Model trainer 164 may be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in some implementations, model trainer 164 includes program files stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, model trainer 164 includes one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium, such as RAM, a hard disk, or an optical or magnetic medium.

[0045] Network 180 may be any type of communications network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications over network 180 may occur over any type of wired and / or wireless connections, using a wide variety of communications protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0046] 1A illustrates one exemplary computing system 100 that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some implementations, the computing device 102 can include a model trainer 164 and a training dataset. In such implementations, the model 124 can be trained or used locally on the computing device 102. In some such implementations, the computing device 102 can implement a model trainer 164 that personalizes the model 124 based on device-specific data.

[0047] 1B illustrates a block diagram of an exemplary computing device 10 for executing, according to an exemplary embodiment of the present disclosure. In various embodiments, the computing device 10 may be a client computing device or a server computing device. For example, the computing device 10 may correspond to a computing device 102. In some embodiments, the computing device 10 (e.g., the computing device 102) may provide operations of machine learning models to one or more client computing devices over a network 180.

[0048] Computing device 10 includes several applications (e.g., applications 1-N). Each application includes its own machine learning library and machine-learned model(s). For example, each application may include a machine-learned model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc.

[0049] 1B, each application may communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application may communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0050] 1C illustrates a block diagram of an exemplary computing device 50 for executing, according to an exemplary embodiment of the present disclosure, a client computing device or a server computing device.

[0051] Computing device 50 includes several applications (e.g., applications 1-N). Each application communicates with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and the model(s) stored therein) using an API (e.g., an API common to all applications).

[0052] The central intelligence layer includes several machine-learned models. For example, as shown in FIG. 1C, each machine-learned model may be provided for each application and managed by the central intelligence layer. In other embodiments, two or more applications may share a single machine-learned model. For example, in some embodiments, the central intelligence layer may provide a single model to all applications. In some embodiments, the central intelligence layer is included within or otherwise implemented by the operating system of the computing device 50.

[0053] The central intelligence layer can communicate with a data layer of a central device, which can be a centralized repository of data for the computing device 50. As shown in FIG. 1C, the data layer of the central device can communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the data layer of the central device can communicate with each device component using an API (e.g., a private API).

[0054] 2 illustrates several computing devices 102-1, 102-2, ..., 102-n serviced by a shared cache 210. As illustrated, each of the computing devices 102-1, 102-2, ..., 102-n includes one or more processors 112-1, 112-2, ..., 112-n, and memory 114-1, 114-2, ..., 114-n. The one or more processors may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operatively connected. The memory may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memories 114-1, 114-2, ..., 114-n may store data 116-1, 116-2, ..., 116-n and instructions 118-1, 118-2, ..., 118-n, respectively, that are executed by the processor to cause the respective computing device to perform operations.

[0055] The memories 114-1, 114-2, ..., 114-n may also include local caches 120-1, 120-2, ..., 120-n that store compiled instructions for execution on the respective computing device, each compiled by a local compiler 122-1, 122-2, ..., 122-n, respectively. To provide reduced processing requirements and lower latency for one or more future executions of the instructions, one or more compiled instructions may be stored in the local cache of the respective computing device.

[0056] However, in some embodiments, one or more of computing devices 102-1, 102-2, ..., 102-n may be requested to perform a task for which their respective local caches do not contain precompiled resources. For example, computing device 102-n may be initialized and added to an array of computing devices, and computing device 102-n may not yet contain any precompiled resources in its local cache 120-n. In one example, local cache 120-1 of computing device 102-1 may store the precompiled resources in shared cache 210 as precompiled shared cache entries 212 associated with shared cache key 220. For example, computing device 102-1 may provide precompiled shared cache entries 212 (e.g., resources compiled by compiler 122-1) stored and associated with shared cache key 220 (e.g., a key configured to enable retrieval of precompiled shared cache entries 212). Thus, computing device 102-n may receive a request to perform an operation associated with shared cache key 220 in some embodiments. Thus, computing device 102-n may query shared cache 210 with shared cache key 220 to obtain a pre-compiled shared cache entry 212 for performing the requested operation. If shared cache 210 has a valid certificate 214 (e.g., a certificate indicating that shared cache 210 is compatible with computing device 102-n), the pre-compiled shared cache entry 212 may be used by computing device 102-n to perform the requested operation.Similarly, in some embodiments, if the shared cache 210 has a positive certificate 214, one or more entries of the shared cache 210 may be populated within the cache 120-n of the computing device 102-n (e.g., automatically, prior to receiving an associated request, etc.).

[0057] 3 illustrates an exemplary state-based approach for determining authentication status (e.g., for renewing a certificate 214). The initial state 300 of the shared cache may be labeled "not authenticated." The state may be updated in a validation update cycle 310 and / or in a certificate update cycle 320. The validation update cycle 310 includes determining the validation status of one or more entries of the shared cache (e.g., precompiled shared cache entries 212 of the shared cache 210) and / or their associated shared cache keys in a validity check 312. If the validity cannot be checked, an invalid status state update 314 is triggered.

[0058] An exemplary approach for performing the validation check 312 is illustrated in the block diagram of FIG. 4. A source 410 (e.g., selected or otherwise determined based on a request to perform an operation) may be provided to each of the two processing streams. In one aspect, the source 410 may be provided to the local compiler 420 (e.g., to the local compiler 122, such as from the memory 114 to the local compiler 122). The compiler 420 may output a compiled resource 430 (e.g., the compiled source 410) directly, which may be processed to obtain a fingerprint 440 that identifies the content of the compiled resource 430. In another aspect, a shared cache key 450 may be associated with the source 410 (e.g., generated based on a request of the source 410, such as a request to perform an operation performed by the source 410), and the shared cache key 450 may be used to query a shared cache (e.g., the shared cache 210) to obtain a shared cache hit 460. A shared cache hit 460 may indicate a retrieval of a precompiled resource 470 from a shared cache based on a shared cache key 450 (e.g., the shared cache key 450 is a means for retrieving the precompiled resource 470). A fingerprint 480 may be obtained based on the precompiled resource 470 that identifies the content of the precompiled resource 470.

[0059] In some embodiments, the fingerprints 440, 480 may be processed in a comparison 490 to determine whether the precompiled resource 470 is comparable (e.g., interchangeable, compatible, identical, etc.) to the compiled resource 430. For example, in some embodiments, the fingerprints 440, 480 may be configured to represent or otherwise identify (e.g., uniquely identify) the content of each resource (e.g., compiled resource 430, precompiled resource 470, etc.) with sufficient granularity to capture differences between the resources that may cause incompatibility. For example, in some embodiments, the fingerprints 440, 480 may include a hash value or other encoding of the content of the respective resource (e.g., some, some, or all of the resource, etc.).

[0060] In some embodiments, the output of comparison 490 may include an indication of the validity of precompiled resource 470. For example, the determination of "valid" may be based on functional equivalence or interoperability with compiled resource 430. In some embodiments, a match (e.g., an exact match, etc.) between fingerprints 440 and 480 may indicate the validity of precompiled resource 470.

[0061] Referring again to FIG. 3, in some embodiments, if the cache entry is valid (e.g., unauthenticated), the state of the shared cache is maintained. In the certification cycle 320, the shared cache is evaluated at eligibility check 322 to determine if the shared cache is sufficiently valid. For example, in some embodiments, one valid entry may qualify the shared cache. However, in some embodiments, an eligibility threshold may require multiple shared cache entries to be verified before the shared cache state is set to authenticated at 324.

[0062] In some embodiments, once authenticated, the shared cache may be periodically re-authenticated. For example, in some embodiments, once authenticated, the shared cache may periodically (e.g., for queries selected by the cache) recheck the validation of one or more cache entries at 312, invalidating the set shared cache status if validation is not confirmed. In some embodiments, the shared cache status may be periodically reset to unauthenticated. For example, the shared cache status may be reset to unauthenticated such that validation of one or more (e.g., multiple) shared cache entries at 312 may trigger a requalification of the cache at 322. In this manner, for example, in some embodiments, a credential (e.g., credential 214) may be maintained for the shared cache.

[0063] In some embodiments, the credential of the shared cache may be updated based on one or more triggering events. For example, the shared cache credential may be updated (e.g., reset, revalidated, etc.) after a set period of time, a set number of cache hits, exceeding a hit rate threshold, etc. In some embodiments, the shared cache credential may be updated when a node newly served by the shared cache queries the cache. For example, one or more entries of the shared cache may be verified to the newly served node (e.g., verified in accordance with this disclosure, such as by the techniques described with respect to FIG. 4, etc.) such that the shared cache can be authenticated to the newly served node and trusted to provide a shared cache entry to the node. In some embodiments, for example, the shared cache may include multiple entries; i.e., a subset may be used to authenticate (or re-authenticate) the shared cache, and some or all of the remainder may be dedicated to the receiving node (e.g., to populate the node's local cache, etc.).

[0064] In some embodiments, if the validity check 312 fails to validate the cache entry, the status of the shared cache may be set to invalid (e.g., at 314). An invalid status or certificate may, for example, cause one or more portions of the shared cache to be deactivated. For example, in some embodiments, the invalidated shared cache stops serving entries to one or more nodes (e.g., all nodes). In some embodiments, the shared cache may revert to its last known authentication state. That is, for example, the shared cache may drop newly added entries and / or stop serving newly serving nodes that failed to validate the cache entry (e.g., provide a safe mode of operation until further debugging). In some embodiments, an update of the invalid status may trigger an update to the configuration of the shared cache keys. For example, if a shared cache key retrieves an incompatible resource from a shared cache (e.g., leading to a validation failure), in some embodiments it may be determined that the shared cache key failed to identify the cached resource with sufficient specificity to capture and respond to the incompatibility between the nodes, and the configuration schema for the shared cache key may be updated to, for example, include a more detailed identifier (e.g., increasing the bit depth of the key encoding, changing the generation of the key encoding to capture additional or different information, or otherwise modified to reduce collisions).

[0065] FIG. 5 illustrates an example extension of the state-based approach of FIG. 3 when the shared cache is in an unauthenticated state. The decision blocks of FIG. 5 are shown arranged in the drawing, but it should be understood that additional and alternative arrangements are contemplated and within the scope of the present disclosure. In some embodiments, a request for an operation to be performed may be made at 502. In some embodiments, for example, a request may be made to a computational node to perform the operation (e.g., by invoking a function of the computational node, such as a machine-learned function or operation). Based on the request 502, it may be determined that a resource is needed to perform the requested operation. In some embodiments, the resource is a computer code resource. In some embodiments, the resource requires or otherwise benefits from compilation for execution. At 504, a query may be performed against a local cache to determine whether the required resource has been previously compiled and is locally available for execution. A hit in the local cache (e.g., indicating a matching entry corresponding to the query) may be provided at 506 to process the request 502 using the local cache entry, and the data flow may end the current cycle or iteration at 599. A miss in the local cache (e.g., indicating that an entry corresponding to the query is not found) may be provided at 508 for compilation of the required resource on the compute node (e.g., locally, on a server, etc.) since the shared cache is in an unauthenticated state using the compiled resource at 510. The shared cache may be queried at 512 (e.g., using the shared cache key) for an entry corresponding to the required resource. If the query misses, the shared cache may be populated at 514 using the resource compiled at 508. If the query is a hit and an entry is retrieved from the cache, the retrieved resource may be verified at 516 (e.g., verified by comparing the compiled resource with the resource retrieved from the cache).If invalid, the authentication state or credentials of the shared cache may be updated from unauthenticated to invalid at 518. If the retrieved resource is valid (e.g., match with compiled resource, compatible, interoperable, etc.), it may be determined at 520 whether the cache is sufficiently valid to obtain a credential. If not, the cycle or iteration ends, optionally at 516, while incrementing a validation count or other indicator of validation. If the cache qualifies (e.g., based on a fitness metric such as a threshold number of retrieved valid entries), the authentication state may be updated at 522 from unauthenticated to authenticated.

[0066] FIG. 6 illustrates an example extension of the state-based approach of FIG. 3 when the shared cache is in an authenticated state. The decision blocks of FIG. 6 are shown arranged in the drawing, but it should be understood that additional and alternative arrangements are contemplated and are within the scope of the present disclosure. In some embodiments, a request for an operation may be obtained at 602. Based on the request 602, the local cache may be queried at 604 for an entry corresponding to a resource required to perform the requested operation. If the query returns a hit, the obtained local cache entry may be used at 606 to perform the requested operation, and the cycle may end at 699. If the local cache query misses, a query may be performed against the authenticated shared cache at 608. If the shared cache also misses, the required resource may be compiled (e.g., locally, at the server, etc.) and used at 610. However, if the shared cache query returns a hit (e.g., using a shared cache key as a means to obtain a shared cache entry), the obtained shared cache entry may be used at 612.

[0067] FIG. 7 illustrates an example extension of the state-based approach of FIG. 3 when the shared cache is in an invalid state. The decision blocks of FIG. 7 are shown arranged in the drawing, but it should be understood that additional and alternative arrangements are contemplated and are within the scope of the present disclosure. In some embodiments, a request for an operation may be obtained, at 702. Based on the request 702, the local cache may be queried, at 704, for an entry corresponding to a resource required to perform the requested operation. If the query returns a hit, the obtained local cache entry may be used, at 706, to perform the requested operation, and the cycle may end, at 799. If the local cache query misses, the required resource may be compiled (e.g., locally, at the server, etc.) and used, at 708.

[0068] Exemplary Methods 8 illustrates a flow chart diagram of an exemplary method 800 performed according to an exemplary embodiment of the present disclosure. Although FIG. 8 illustrates steps performed in a particular order for purposes of illustration and description, the method of the present disclosure is not limited to the specifically shown order or arrangement. Various steps of method 800 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0069] In some exemplary aspects, the exemplary method 800 provides, in some embodiments, a computer-implemented method for authenticating a shared cache. For example, a computational node or device (e.g., of a computing system) may receive a request to perform an operation. For example, the request may include substantially any processing task, such as a function call, a queued task, a distributed process, etc. Performing the operation may use, execute, or otherwise implement one or more resources. In some embodiments, the resources may be compiled to perform the operation (e.g., need to be compiled to function, function better when compiled, etc.). In some embodiments, for example, performing the requested operation may include executing compiled computer code.

[0070] In some embodiments, pre-compiled resources are desirable for performing an operation. Caching of previously used or retrieved resources can, in some embodiments, improve the speed of performing an operation using the cached resources (e.g., by providing faster retrieval than recompiling the resources). However, in some embodiments, the example method 800 includes determining, by the computing system, that there are no locally cached pre-compiled resources associated with the operation request. For example, the computing node or device can query a first cache (e.g., a local cache) to retrieve a cached copy of the pre-compiled resource. However, in some cases, the first cache (e.g., a local cache) does not contain the requested resource.

[0071] At 802, the example method 800 may include obtaining a pre-compiled shared cache entry corresponding to the shared cache key. The shared cache key, in some embodiments, may be associated with an operation request. For example, the operation request may indicate an operation to be performed and may indicate an operation to be associated with the shared cache key to obtain one or more resources for performing the operation. For example, in some embodiments, the shared cache key may be generated based at least in part on the operation request and configured to identify one or more resource(s).

[0072] At 804, the example method 800 may include obtaining (e.g., from a runtime compiler) a direct compiled resource associated with the operation request. For example, in some embodiments, a computing system may both obtain a pre-compiled shared cache entry and generate or otherwise obtain a direct compiled resource that corresponds to the pre-compiled shared cache entry.

[0073] At 806, example method 800 may include authenticating one or more portions of the shared cache based at least in part on a comparison of the pre-compiled shared cache entry to the directly compiled resource (e.g., as described above with respect to FIGS. 3-7). In some embodiments, for example, the comparison includes comparing a fingerprint (e.g., a hash, etc.) of the directly compiled resource to a fingerprint of the pre-compiled shared cache entry. In some embodiments, the pre-compiled shared cache entry was compiled for or by the same or an equivalent computing system as the directly compiled resource, such that the pre-compiled shared cache entry and the directly compiled resource are substantially equivalent (e.g., interoperable, interchangeable, etc.) and / or identical.

[0074] If the comparison does not indicate a match, it may indicate, for example, that the pre-compiled shared cache entry is not a valid equivalent to the directly compiled resource and, therefore, was invalidly obtained by the shared cache key. For example, the example method 800 may include deactivating, by the computing system, one or more portions of the shared cache based at least in part on determining that the pre-compiled shared cache entry is invalid. Deactivating One or More Portions of the Shared Cache

[0075] If the comparison indicates a match, it may indicate, for example, that the pre-compiled shared cache entry is valid and was validly obtained by the shared cache key. In this manner, it may be determined, for example, that the shared cache does indeed provide compatible cached resources and may be used to provide the pre-compiled resource to perform the requested operation. In some embodiments, an eligibility threshold may be used to determine the number of entries that are confirmed as valid before authenticating one or more portions of the shared cache as a whole. For example, the example method 800 may include updating, by the computing system, a plurality of comparisons indicative of validity of one or more shared cache entries of the shared cache, the plurality of comparisons corresponding to an eligibility threshold, and determining, by the computing system, to authenticate the shared cache based at least in part on the plurality of comparisons satisfying the eligibility threshold.

[0076] In some embodiments, the exemplary method 800 may include maintaining, by the computing system, an authentication state for one or more portions of the shared cache. For example, in some embodiments (e.g., as described above with respect to FIGS. 3-7), a certificate for the shared cache (e.g., or one or more portions thereof) may indicate an authentication state of unauthenticated, authenticated, invalid, etc. The exemplary method 800 may include updating, by the computing system, the authentication state based at least in part on the comparison. In some embodiments, the exemplary method 800 includes periodically and by the computing system resetting the authentication state of the shared cache. For example, it may be desirable to periodically reauthenticate one or more portions of the shared cache (e.g., for reliability, security, maintenance, etc.).

[0077] 9 illustrates a flow chart diagram of an exemplary method 900 performed according to an exemplary embodiment of the present disclosure. Although FIG. 9 illustrates steps performed in a particular order for purposes of illustration and description, the method of the present disclosure is not limited to the specifically shown order or arrangement. Various steps of method 900 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0078] At 902, the example method 900 includes receiving a request for a pre-compiled shared cache entry from a shared cache from a compute node that processes the requested operation. In some embodiments, the shared cache entry is obtained for the compute node using a shared cache key.

[0079] At 904, the example method 900 may include obtaining an authentication status of the shared cache based at least in part on a comparison of (i) a directly compiled resource (e.g., compiled by the compute node for the compute node, such as by a runtime compiler) for performing the requested operation and (ii) a pre-compiled shared cache entry (e.g., as described above with respect to Figures 3-7), where the authentication status indicates an incompatibility of the pre-compiled shared cache entry. In some embodiments, the comparison may include a comparison of a fingerprint of the directly compiled resource and a fingerprint of the pre-compiled shared cache entry.

[0080] At 906, the example method 900 can include deactivating the shared cache for one or more future requests from the compute node. For example, the shared cache can be taken offline entirely. For example, if a shared cache key obtains a shared cache entry that is incompatible with the querying node, the shared cache can be taken offline (e.g., taking the cached entry out of service) to prevent further serving of the incompatible entry in the shared cache key scheme. For example, a validation failure can, in some embodiments, indicate a failure of the shared cache key scheme to accurately identify resources that are compatible with the requesting computing system / device and distinguish between incompatible resources. In some embodiments, the shared cache can be reset to a last known secure configuration (e.g., a state at the time of the last validation, etc.) and / or one or more previously validated and authenticated portions of the shared cache can be retained while one or more other portions can be taken out of service.

[0081] Additional Disclosures The technology described herein refers to servers, databases, software applications, and other computer-based systems, as well as actions performed to and from such systems and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functions among components. For example, the processes described herein may be performed using a single device or component, or multiple devices or components operating in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate serially or in parallel.

[0082] Although the present subject matter has been described in detail with respect to various specific and exemplary embodiments thereof, each example is provided for the purpose of illustration and not for the purpose of limiting the present disclosure. Those skilled in the art, upon understanding the above content, can easily make modifications, variations, and equivalents of such embodiments. Thus, the present disclosure does not exclude the inclusion of such modifications, alterations, and / or additions to the present subject matter as would be readily apparent to those skilled in the art. For example, features illustrated or described as part of one embodiment can be used with another embodiment to produce yet another embodiment. Thus, the present disclosure is intended to cover such modifications, alterations, and equivalents.

Claims

1. 1. A computer-implemented method for authenticating a shared cache, comprising: obtaining, by a computing system including one or more processors, a pre-compiled shared cache entry corresponding to a shared cache key, the shared cache key being associated with an operation request; obtaining, by the computing system, a directly compiled resource associated with the operation request; and authenticating, by the computing system, one or more portions of the shared cache based at least in part on a comparison of the pre-compiled shared cache entry and the directly compiled resource.

2. The computer-implemented method of claim 1 , further comprising determining, by the computing system, that there is no locally cached pre-compiled resource associated with the operation request.

3. Retrieving, by the computing system, a directly compiled resource associated with the operation request, The computer-implemented method of claim 1 , comprising obtaining the directly compiled resource by the computing system and from a run-time compiler.

4. The computer-implemented method of claim 1 , wherein the comparison comprises comparing a fingerprint of the direct compiled resource with a fingerprint of the pre-compiled shared cache entry.

5. authenticating, by the computing system, one or more portions of the shared cache based at least in part on the comparison, 2. The computer-implemented method of claim 1, further comprising updating, by the computing system, a plurality of comparisons indicative of validity of one or more shared cache entries of the shared cache, the plurality of comparisons corresponding to an eligibility threshold.

6. The computer-implemented method of claim 5 , further comprising determining, by the computing system, to authorize the shared cache based at least in part on the plurality of comparisons satisfying the eligibility threshold.

7. authenticating, by the computing system, one or more portions of the shared cache based at least in part on the comparison, 2. The computer-implemented method of claim 1, further comprising deactivating, by the computing system, the one or more portions of the shared cache based at least in part on determining that the pre-compiled shared cache entry is invalid.

8. maintaining, by the computing system, an authentication state for the one or more portions of the shared cache; The computer-implemented method of claim 1 , further comprising updating, by the computing system, the authentication state based at least in part on the comparison.

9. The computer-implemented method of claim 8 , further comprising periodically and by the computing system, resetting the authentication state of the shared cache.

10. A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations, the operations including: Retrieving a pre-compiled shared cache entry corresponding to a shared cache key, the shared cache key being associated with an operation request; Obtaining a directly compiled resource associated with the operation request; and and authenticating one or more portions of the shared cache based at least in part on a comparison of the pre-compiled shared cache entries and the directly compiled resources.

11. The operation is The computer-readable medium of claim 10 , further comprising determining that there is no locally cached pre-compiled resource associated with the operation request.

12. Retrieving a directly compiled resource associated with the operation request includes: The computer-readable medium of claim 10 , further comprising obtaining the directly compiled resource from a run-time compiler.

13. 11. The computer-readable medium of claim 10, wherein the comparison comprises comparing a fingerprint of the direct compiled resource with a fingerprint of the pre-compiled shared cache entry.

14. authenticating the shared cache based at least in part on the comparison, 11. The computer-readable medium of claim 10, further comprising updating a plurality of comparisons indicative of validity of one or more shared cache entries of the shared cache, the plurality of comparisons corresponding to an eligibility threshold.

15. The operation is The computer-readable medium of claim 14 , further comprising determining to authorize the shared cache based at least in part on the plurality of comparisons satisfying the eligibility threshold.

16. authenticating the shared cache based at least in part on the comparison, 11. The computer-readable medium of claim 10, further comprising deactivating the one or more portions of the shared cache based at least in part on determining that the pre-compiled shared cache entry is invalid.

17. The operation is maintaining an authentication state for the one or more portions of the shared cache; and updating the authentication state based at least in part on the comparison.

18. one or more processors; and one or more non-transitory computer-readable media comprising instructions that, when executed, cause the one or more processors to perform operations, the operations including: receiving a request for a precompiled shared cache entry from the shared cache from a compute node that processes the requested operation; obtaining an authentication status of the shared cache based at least in part on a comparison of (i) a directly compiled resource for performing the requested operation on the compute node and (ii) the precompiled shared cache entry, the authentication status indicating an incompatibility of the precompiled shared cache entry with the compute node; and deactivating the shared cache for one or more future requests from the computing node based at least in part on the authentication status.

19. The system of claim 18 , wherein the directly compiled resources are compiled for the compute node.

20. 20. The system of claim 19, wherein the comparison comprises comparing a fingerprint of the direct compiled resource with a fingerprint of the pre-compiled shared cache entry.

Images