Low-latency file-based ordered message delivery at scale

Abstract

Example embodiments of the present disclosure provide for an example method including accessing message data indicative of an update. The method includes transmitting, from a source cluster to a destination cluster, a first replica of the message data via a nonvolatile storage and, in parallel, a second replica of the message data via a data cache by splitting the message data into a plurality of data chunks including at least a first data chunk, writing, by a writer of the destination cluster, the first data chunk and first metadata to a memory of the data cache, and reading, by a reader of the destination cluster, the first metadata and the first data chunk. The method includes serving, by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster.

Description

LOW-LATENCY FILE-BASED ORDERED MESSAGE DELIVERY AT SCALEPRIORITY

[0001] The present application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 722,290, filed on November 19, 2024, which is incorporated by reference herein.FIELD

[0002] The present disclosure relates generally to systems and methods for low-latency filebased ordered message delivery at scale.BACKGROUND

[0003] Message delivery is utilized in real-time systems such as fraud detection, activity monitoring, online matching, retail inventory management, advertising, and dynamic pricing. These systems can include real-time extractors that can rely on message delivery subsystems to transport messages containing extracted data to real-time systems for delivery. The systems must provide for in-order delivery and at-least once delivery to systems in the network to support stateful replicated systems. In-order delivery and at-least-once delivery subsumes unordered and at-most-once delivery respectively.SUMMARY

[0004] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.

[0005] In one example aspect, the present disclosure provides for an example system for low- latency filed-based ordered message delivery at scale, including one or more processors and one or more memory device storing instructions that are executable to cause the one or more processors to perform operations. In some implementations, the one or more memory devices can include one or more transitory or non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations. In the example system, the operations can include accessing message data indicative of an update. In the example system, the operations can include transmitting, from a source cluster, a first replica of the message data via a nonvolatile storage of the source cluster to a nonvolatile storage of a destination cluster. In the example system, the operations can include transmitting, from the source cluster, in parallel to transmitting the first replica, a second replica of the message data via a data cache of the source cluster to a data cache of thedestination cluster by: splitting the second replica of the message data into a plurality of data chunks comprising at least a first data chunk; writing, by a writer of the destination cluster, the first data chunk and first metadata to a memory of the data cache; and reading, by a reader of the destination cluster, the first metadata and the first data chunk. In the example system, the operations can include and serving, by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster.

[0006] In an example aspect, the present disclosure provides for an example computer- implemented method. The example method includes accessing message data indicative of an update. The example method includes transmitting, from a source cluster, a first replica of the message data via a nonvolatile storage of the source cluster to a nonvolatile storage of a destination cluster. The example method includes transmitting, from the source cluster, in parallel to transmitting the first replica, a second replica of the message data via a data cache of the source cluster to a data cache of the destination cluster by: splitting the second replica of the message data into a plurality of data chunks comprising at least a first data chunk; writing, by a writer of the destination cluster, the first data chunk to a memory of the data cache; and reading, by a reader of the destination cluster, the first data chunk. The example method includes serving, by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster. The example method includes determining that a second data chunk is missing from the data cache. The example method includes responsive to determining that the second data chunk is missing from the data cache, reading the first replica of the message data from the nonvolatile storage of the destination cluster.

[0007] In an example aspect, the present disclosure provides for an example transitory or non-transitory computer readable medium embodied in a computer-readable storage device and storing instructions that, when executed by a processor, cause the processor to perform operations. In the example transitory or non-transitory computer readable medium, the operations include accessing message data indicative of an update. In the example transitory or non-transitory computer readable medium, the operations include transmitting, from a source cluster, a first replica of the message data via a nonvolatile storage of the source cluster to a nonvolatile storage of a destination cluster. In the example transitory or non- transitory computer readable medium, the operations include transmitting, from the source cluster, in parallel to transmitting the first replica, a second replica of the message data via a data cache of the source cluster to a data cache of the destination cluster by: splitting the second replica of the message data into a plurality of data chunks comprising at least a firstdata chunk; writing, by a writer of the destination cluster, the first data chunk and first metadata to a memory of the data cache; and reading, by a reader of the destination cluster, the first metadata and the first data chunk. In the example transitory or non-transitory computer readable medium, the operations include and serving, by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Detailed discussion of embodiments directed to one of ordinary skill in the art is set forth in the specification, which makes reference to the appended figures, in which:

[0009] Figure 1 depicts an example block diagram of an example system for low-latency filebased ordered message delivery at scale according to example embodiments of the present disclosure.

[0010] Figure 2 depicts a block diagram of an example parallel nonvolatile storage path and cache storage path according to example embodiments of the present disclosure.

[0011] Figure 3 depicts a block diagram of an example cache including writer and reader jobs according to example embodiments of the present disclosure.

[0012] Figure 4 depicts a flow chart diagram of an example system for low-latency file-based ordered message delivery at scale according to example embodiments of the present disclosure.

[0013] Figure 5 depicts a block diagram of an example reading of chunks according to example embodiments of the present disclosure.

[0014] Figure 6 depicts a block diagram of example read range of chunks according to example embodiments of the present disclosure.

[0015] Figure 7 depicts a block diagram of an example write according to example embodiments of the present disclosure.

[0016] Figure 8 depicts a block diagram of an example algorithm for delayed nonvolatile storage reads for a file according to example embodiments of the present disclosure.

[0017] Figure 9 depicts a flow chart diagram of an example method to perform low-latency file-based ordered message delivery at scale according to example embodiments of the present disclosure.

[0018] Figure 10 block diagram of an example computing system that performs low-latency file-based ordered message delivery at scale according to example embodiments of the present disclosure.DETAILED DESCRIPTION

[0019] Generally, the present disclosure is directed to improved methods for delivering low- latency filed-based ordered message delivery at scale. The present disclosure provides for a combination of data transmission along both a nonvolatile storage as well as cache to allow for low-latency transmission while providing for a back up of non-volatile memory data transmission. Additionally, the present disclosure provides for the ability for data transmission at scale utilizing an improved tree structure for message relay. The method can include distributing the responsibility for answering calls made by downstream devices across multiple machines in a server center to spread the processing load among the server machines and avoid points of failure. Existing methods cannot provide for the transmission of messages at scale without suffering from system degradation or crashes.

[0020] The present disclosure provides for improvements to achieving low latency copies of message data by isolation, design for low latency, stability for in-flight copies, and reducing dependency on non-volatile storage for meeting service level goals. Isolation can include isolating internal jobs within clusters for fast copies which can increase stability in copy performance and enable implementation of latency focused algorithm (e.g. in content item readers and writers). Design for low latency can include providing for detection of new data faster and schedule copy in near real-time, prioritizing latency over resource utilization in content copy algorithms, or setting tighter deadlines on dependent systems. Stability for inflight copies can include maintaining a stable copy to enable stable performance and reduction in the possibility of latency degradation due to reestablishing copy state. Reducing dependency on non-volatile storage for meeting aggressive service level goals can include introducing redundancy (e.g., parallel message transmission) in the storage layer for replication to not be effectively gated.

[0021] The proposed structure includes readers, writers, and a scheduler. The scheduler can include a scanner and a notification forwarder. The read / wri ter jobs can be configured to run with lower utilization levels to allow for bandwidth to handle traffic surges without disruption. Existing systems, function at or close to max bandwidth, as such surges cannot be handled and disruption in message transmission can be affected. Scheduler can provide for scheduling and maintaining copy progress.

[0022] The improvements associated with the systems and methods discussed herein can be further understood with reference to the figures.

[0023] Figure 1 depicts a block diagram of an example system for low-latency file-based ordered message delivery at scale. The example system can include a source cluster 105communicatively coupled to a destination cluster 115 over a network 110. In some implementations, the source cluster 105 and destination cluster 115 can be located in geographically distinct locations. As such, to accommodate geographically distributed client devices, message files can be copied from the source cluster’s storage to the destination cluster’s storage. Consumer client devices can tail-read the file from their cluster-local storage system as depicted herein. This can provide for cross-cluster file-based ordered message delivery for real-time (or nearly real-time) system.

[0024] The source cluster can include databases 120 or real-time log processors 125. Extractors and transformers 130 can access data from databases 120 or real-time log processors 125. Extractors and transformers 130 can take the data and format it in such a way that it can be transmitted from the source cluster 105 to the destination cluster 115. In some instances, the data can include content items to be served to end users. In some instances, the content items can be updated periodically or responsive to certain actions which serve as triggers. For instance, a trigger can include a content provider updating content items to be provided, updating inventory, or other time-sensitive updates.

[0025] Extractors and transformers 130 can transmit message stream files 135 via nonvolatile storage 150. Message stream files 135 can include updates or changes in the data since the last read.

[0026] Nonvolatile storage 150 can be slower to update. In some instances, nonvolatile storage can have a latency of around a few minutes. Additionally, the system can transmit a copy of message stream files 155 via cache 170. Copy of Message stream files 155 can be a copy of message stream files 135. Cache 170 can be faster for data transmission. As described herein, transmitting data via cache 170 can provide for updates in a matter of seconds (e.g., under 2 seconds). Cache can include reader / wri ter jobs which provide for fast transfer of data transfer as described herein.

[0027] Nonvolatile storage 150 can transmit message stream files via network 110 to nonvolatile storage 180 at destination cluster 115. For instance, destination cluster 115 can send requests for message stream files 175 which can then be written to nonvolatile storage 180. For instance, message stream files 175can include updates or changes in the data since the last read.

[0028] In addition to the data being transmitted via nonvolatile storage, copy of message stream files 155 can be transmitted via cache 170 (e.g., via random access memory) to provide for faster transmission of the message stream files. Copy of message stream files 185 can be copies of message stream files 175. Destination cluster 115 can include reader jobswhich can allow for reading and storing data and metadata associated with copy of message stream files 185 to be read and processed at destination cluster. The message stream files 185 that are read and processed at destination cluster 115 can be provided to serving system(s) 195 to be provided to a number of end user devices. In some instances, a source cluster 105 and destination cluster 115 can be a set of nodes in a larger tree structure as depicted in Figure 2.

[0029] Figure 2 depicts transfer of update data 205 via a nonvolatile storage path 225 alongside a parallel cache path 220. The cache path 220 can provide for faster transmission of data from an initial node 210A to subsequent nodes 210B-E. Each node can include a nonvolatile storage media 212A-212E. Traditional systems provide for data transfer through the nonvolatile storage media 212A-212E alone. These systems can result in delays in transmission of data. The replication via cache path 220 can be a redundant file copy path alongside the nonvolatile storage path. A scheduling component can schedule two copy operations for each file including a nonvolatile storage to nonvolatile storage transfer and cache to cache transfer.

[0030] Figure 2 depicts the parallel cache path 220 which can provide for transmission via cache 214B-214E of the respective nodes 210A-210E in the tree. In some instances, all nodes in the tree can have reads and writes performed to both nonvolatile storage 212A-E and cache 214A-E. In some instances, the first node in the tree can have reads and writes performed to a nonvolatile storage 212A and the remaining reads and writes can be via cache 214B-214E. Between each respective set of nodes, such as node 210A and node 210B can be a reader / writer job. A writer of node 210A can write data chunks to cache 214B which can be read by reader job at node 210D. Data in storage can be used at node 210B to serve data to end users as well as a writer job can write to cache 214B where the reader / writer process can occur for the subsequent nodes. An example of the reader / writer process at a respective set of nodes is depicted in Figure 3 and Figure 4.

[0031] Figure 3 depicts a cache 305. Cache 305 can include writer 310, memory 325, and reader 350. Writer 310 can access data 320 which can be stored in random access memory 315. Writer 310 can make two writes. Writer 310 can make a first write for data 320 in random access memory 335 as data file 340 and a second write for metadata 345. Metadata 345 can hold information about the data file 320. This can allow for replication of all message streams by both systems. The in-memory (e.g., cache) layer can absorb a majority of queries that traditionally would be handled by the nonvolatile storage 450 layer can behandled by the cache 305 layer. As such, a low-latency and high-throughput path for message delivery can be achieved

[0032] Reader 350 can access metadata 345 and data file 340. For instance, Reader 350 can poll the metadata 345 to determine file growth data. If the file has grown, the reader can read the new bytes from the data file 340 which can be stored in random access memory 355 and stored as data 360.

[0033] Figure 4 depicts an example system writer job 400 between two nodes of a tree structure. Writer job 400 can include producers / writers 405 writing data to nonvolatile storage 450 via remote procedure call (RPC) and data cache 445. The data can include file bytes written as chunks to the data cache 445 alongside writing the bytes to the nonvolatile storage 450. Producers / writers 405 can write metadata to metadata cache 410 and write chunks of data into data cache 445. By way of example data cache 445 can include an expired chunk 430, a non-expired chunk 435, and a partially filled chunk 440.

[0034] In some instances, each chunk can be represented as a key-value pair. For instance, the key for each chunk can include a hash of two components. The two components can include the nonvolatile storage file path which can be globally unique across the nonvolatile storage and the chunk’s sequence number. The value of the hash can include the corresponding chunk bytes. In some implementations, the chunk size can be 4 KB. The chunk size can be any reasonable size. The chunk size can be determined based on the observed message size and a maximum transmission unit in the cluster fabric. As such, the chunk can be read in a single packet.

[0035] As depicted in Figure 4, chunk 440 can be in the process of being written by producers / writers 405 to data cache 445. Producers / writers 405 can write chunk data to data cache 445 via remote procedure call (RPC). Alongside writing the chunks of data, ,the writers can update the length of the file in the metadata cache 410. Metadata cache 410 can include length 415 which describes the length of the message files.

[0036] Consumers / readers 425 can poll the metadata cache 410 for the latest length of the file in the metadata cache 410 and read the chunks from the data cache 445. Consumers / readers 425 can use (RMA) to reach chunks from data cache 445 and metadata cache 410. As described herein, the chunks can be message data that is split into even sized chunks which are written into cache to allow for fast reading and writing of the data. If the consumers / readers 425 were to attempt to read an expired chunk 430, an error would occur, and the system can fall back to read from nonvolatile storage 450.

[0037] Intra-cluster reads via RMA can provide for advantages given the read -heavy nature of intra-cluster traffic. Each key -value pair in the cache can be subject to garbage collection (GC) either the expiration of Time-To-Live (TTL) or when the cache approaches its storage capacity limit. The approach described herein can provide for resilience against single-point failures. By way of example, the replication provided by the cache can offer a range of read consistency modes. The modes can include a consistent mode and a relaxed mode. If for instance, the system provides for r=3.2 replication, the consistent mode can read values from at least 2 out of the 3 replicas which can ensure linearizability and delivering a most recent value. The relaxed mode can read from a random replica and may in some instances provide for a stale value.

[0038] As described in this example illustration, chunk 430 has expired and chunk 440 is partially-filled. The length 415 stored in the metadata cache 410 can point to the current position within the last, partially-filled chunk.

[0039] Figure 5 depicts an example of relaxed reads followed by consistent reads. For instance, in the illustration, bytes such as replica 505, replica 510, and replica 515 can be written across multiple chunks, such as chunk 520 and chunk 525. As the bytes are written across the chunks, the previously partially-filled chunk 520 can be filled and a new chunk 525 can be successfully written to all replicas and chunk 520 can be written to replica 510 and replica 515. When a reader such as reader 530 detects a new length from the metadata cache and performs a relaxed read for chunk 520 from replica 505, it will receive fewer bytes than expected. As such, the reader 530 can perform a consistent read to obtain the complete chunk 520.

[0040] Figure 6 depicts an example fallback reads to nonvolatile storage 610 responsive to determining there are missed chunks in the data cache. For instance, read range from data cache 605 where a portion of the chunks have been read and a portion of the chunks have been missed. In an instance where one or more chunks have been missed, the system can provide for accessing the nonvolatile storage to provide for fallback read range from nonvolatile storage 610. As such, the system can maintain the minimum requirements for transmitting data at least at the latency provided by reading from the nonvolatile storage while also providing for faster reading and writing of data when the system can successfully read from the cache.

[0041] By performing a read range to determine a number of missed chunks, the system can save resources by fetching all the missed chunks in a single read of nonvolatile storage to amortize the remote procedure call (RPC) cost.

[0042] Figure 7 depicts an example illustration of writes to data cache. Depicted is an example implementation where the 0thchunk is partially filled and there are two concurrent write requests. The first write request 705 can include a few bytes (e.g., not enough bytes to fill the 0thchunk) and the second write request 710 can include large bytes (e.g., spanning over part of the 0thchunk, fully over the 1stand 2ndchunk, and partially over the 3rdchunk). It is possible for second write request 710 to arrive before first write request 705. This would lead to the 1st, 2nd, and 3rdchunk being written before the 0thchunk. This can cause an issue because the second write request 710 for the 0thchunk must be serialized after the first write request 705 is received and the chunk is complete. The length which is shown by pointer 715 and pointer 720 can be updated only after all bytes up to the pointer are filled out.

[0043] Figure 8 depicts an algorithm 800 of an example delayed nonvolatile storage reads for a file. The algorithm can include determining a maximum tolerable read delay. The algorithm 800 can include a nonvolatile file size historian.

[0044] For a nonvolatile storage file, the operation can employ single-threaded appends as well as update the metadata functioning as an atomic step.

[0045] For the data cache, the operation can differentiate between complete chunks and last partially-filled chunks. A complete chunk can be full and a partially-filled chunk is at least one byte short of the full size. Complete chunks can be written in parallel as they are only written once. However, writes to partially-filled chunks can be serialized to ensure that the prefix of bytes in the chunk are always correct (e.g., as depicted in Figure 8). As such, there can be per-chunk dependencies between write requests and the operation can maintain a map of the write relationships for each chunk. Once the writes are complete, the length of the metadata cache can be updated to reflect the position up to which all completed chunks and the last partially-filled chunk have been written to the data cache. Unlike the nonvolatile storage file, the writes to the data and the metadata cache does not constitute an atomic step.

[0046] Figure 9 depicts a flowchart of a method 900 to perform low latency filed-based ordered message delivery at scale according to example implementations of aspects of the present disclosure. The method 900 can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, method 900 is performed by a server computing system (e.g., server computing system 1130) or client computing system (e.g., client computing device 1102). Although shown in a particular sequence or order, unless otherwise specified, the order of the processes can be modified.Thus, the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processors can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

[0047] At operation 902, processing logic can access message data indicative of an update. Message data can include a content item. Message data can include a current inventory update. Message data can include any message data accessible by processing logic.

[0048] At operation 904, processing logic can transmit, from a source cluster, a first replica of the message data via a nonvolatile storage of the source cluster to a nonvolatile storage of a destination cluster.

[0049] At operation 906, processing logic can transmit, from the source cluster, in parallel to transmitting the first replica, a second replica of the message data via a data cache of the source cluster to a data cache of the destination cluster by performing the operations of 906A- C. In some instances, transmitting the second replica of the message data via the data cache can be performed in a short period of time. A short period of time, for instance, can be under a few seconds or in some implementations in the order of sub-seconds. A few seconds can, in some instances, mean 3 or less seconds.

[0050] At operation 906A, processing logic can split the message data into a plurality of data chunks comprising at least a first data chunk. In some instances, each chunk of the plurality of chunks is the same size. For instance, the chunks can be a set number of bytes.

[0051] At operation 906B, processing logic can write, by a writer of the destination cluster, the first data chunk and first metadata to a memory of the data cache. In some instances, the data cache comprises a plurality of memory locations. In some implementations, processing logic can select a memory location for performing the writing and reading based on a capacity associated with a memory location. For instance, a number of storage devices and associated memory (e.g., random access memory (RAM)) can be utilized to store data. As described herein, a source cluster can be located in a first geographic region and the destination cluster can be located in a second geographic region. For instance, a geographic region can include the United States, Europe, Asia-Pacific, or any other geographic region or sub -region.

[0052] At operation 906C, processing logic can read, by a reader of the destination cluster, the first metadata and the first data chunk. For instance, the system can first reach the first metadata and responsive to reading the metadata, the processing logic can read the first data chunk.

[0053] At operation 908, processing logic can serve by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster. For instance, the plurality of devices can be client devices associated with a plurality of users. The client devices can be associated with content item campaigns. The message data sent to a respective device can be selected based on one or more characteristics associated with the respective device.

[0054] Processing logic can determine that a second data chunk is missing from the data cache. Responsive to determining that the second data chunk is missing from the data cache, processing logic can read the first replica of the message data from the nonvolatile storage of the destination cluster. As such, the nonvolatile storage can function as a back-up if or when a data chunk is missing from the cache.

[0055] FIG. 10 depicts a block diagram of an example computing system 1000 that performs low-latency file-based ordered message delivery at scale according to example embodiments of the present disclosure. The computing system 1000 includes a user computing system 1002, a server computing system 1030, and a training computing system 1050 that are communicatively coupled over a network 1080.

[0056] The user computing system 1002 can include a user device. A user device can be any type of computing device, such as, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0057] The user computing system 1002 includes one or more processors 1012 and a memory 1014. The one or more processors 1012 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, and the like) and can be one processor or a plurality of processors that are operatively connected. The memory 1014 can include one or more transitory or non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and the like, and combinations thereof. The memory 1014 can store data 1016 and instructions 1018 which are executed by the processor 1012 to cause the user computing system 1002 to perform operations.

[0058] In some implementations, the user computing system 1002 can store or include one or more machine-learned models 1020. For example, the machine-learned models 1020 can be or can otherwise include various machine-learned models such as Bayesian belief networks, kernel-based regularized least squares regression. Bayesian belief networks can includeprobabilistic graphical models comprising nodes and directed edges and learned from data. Kernel-based regularized least square regression can include non-linear regression estimation. In some implementations machine-learned models 1020 can be or can otherwise include neural networks (e.g., deep neural networks) or other types of machine-learned models, including non-linear models and / or linear models. Neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks or other forms of neural networks. Some example machine-learned models can leverage an attention mechanism such as self-attention. For example, some example machine-learned models can include multi-headed self-attention models (e.g., transformer models).

[0059] In some implementations, the one or more machine-learned models 1020 can be received from the server computing system 1030 over network 1080, stored in the user computing device memory 1014, and then used or otherwise implemented by the one or more processors 1012. In some implementations, the user computing system 1002 can implement multiple parallel instances of a single machine-learned model 1020 (e.g., to perform parallel learning across multiple instances of content item selection).

[0060] More particularly, the overall model can include a suite of machine-learned modeling capabilities that are robust to data assumptions and easy to scale. The machine-learned modeling capabilities can determine a structure of a network (e.g., Bayesian belief network) and utilize the network for resource allocation determination. The output of the modeling capabilities can be used to more efficiently allocate resources to improve desired target outcomes. The machined learned models can be used for expediting the modeling process of media channel performance and allow for near real-time learnings of the effect of resource allocation on target actions performed.

[0061] Additionally or alternatively, one or more machine-learned models 1040 can be included in or otherwise stored and implemented by the server computing system 1030 that communicates with the user computing system 1002 according to a client-server relationship. For example, the machine-learned models 1040 can be implemented by the server computing system 1030 as a portion of a web service (e.g., a marketing service). Thus, one or more models 1020 can be stored and implemented at the user computing system 1002 and / or one or more models 1040 can be stored and implemented at the server computing system 1030.

[0062] The user computing system 1002 can also include one or more user input components 1022 that receives user input. For example, the user input component 1022 can be a touch- sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive tothe touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input.

[0063] User computing system 1002 can include one or more user interface(s) 1024. For example user interface(s) 1024 can include graphical user interfaces, audio user interfaces, command line interfaces, menu-driven user interfaces, touch user interface, voice user interface, form-based user interface, or natural language user interfaces.

[0064] The server computing system 1030 includes one or more processors 1032 and a memory 1034. The one or more processors 1032 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, and the like) and can be one processor or a plurality of processors that are operatively connected. The memory 1034 can include one or more transitory or non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and the like, and combinations thereof. The memory 1034 can store data 1036 and instructions 1038 which are executed by the processor 1032 to cause the server computing system 1030 to perform operations.

[0065] In some implementations, the server computing system 1030 includes or is otherwise implemented by one or more server computing devices. In instances in which the server computing system 1030 includes plural server computing devices, such server computing devices can operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.

[0066] As described above, the server computing system 1030 can store or otherwise include one or more machine-learned models 1040. For example, the models 1040 can be or can otherwise include various machine-learned models. Example machine-learned models include can be or can otherwise include various machine-learned models such as Bayesian belief networks, kernel-based regularized least squares regression. Bayesian belief networks can include probabilistic graphical models comprising nodes and directed edges and learned from data. Kernel-based regularized least square regression can include non-linear regression estimation. In some implementations machine-learned models 1040 can be or can otherwise include neural networks or other multi-layer non-linear models. Example neural networks include feed forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine-learned models can leverage an attention mechanism such as self-attention. For example, some example machine-learned models can include multi-headed self-attention models (e.g., transformer models).

[0067] The user computing system 1002 and / or the server computing system 1030 can train the models 1020 and / or 1040 via interaction with the training computing system 1050 that is communicatively coupled over the network 1080. The training computing system 1050 can be separate from the server computing system 1030 or can be a portion of the server computing system 1030.

[0068] The training computing system 1050 includes one or more processors 1052 and a memory 1054. The one or more processors 1052 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, and the like) and can be one processor or a plurality of processors that are operatively connected. The memory 1054 can include one or more transitory or non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and the like, and combinations thereof. The memory 1054 can store data 1056 and instructions 1058 which are executed by the processor 1052 to cause the training computing system 1050 to perform operations. In some implementations, the training computing system 1050 includes or is otherwise implemented by one or more server computing devices.

[0069] The training computing system 1050 can include a model trainer 1060 that trains the machine-learned models 1020 and / or 1040 stored at the user computing system 1002 and / or the server computing system 1030 using various training or learning techniques, such as, for example, backwards propagation of errors. For example, a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the loss function). Various loss functions can be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations.

[0070] In some implementations, performing backwards propagation of errors can include performing truncated backpropagation through time. The model trainer 1060 can perform a number of generalization techniques (e.g., weight decays, dropouts, and the like) to improve the generalization capability of the models being trained.

[0071] In particular, the model trainer 1060 can train the machine-learned models 1020 and / or 1040 based on a set of training data 1062. The training data 1062 can include, for example, data associated with historical resource allocation (e.g., one or more months of media spend), daily data (e.g., resource allocation for various media channels, sales, conversions) at a geo level (e.g., zip code, city, or DMA level), weekly data (e.g., resource allocation for various media channels, sales, conversions) at a geo level (e.g., zip code, city,or DMA level), or different media weights across different time periods (e.g., when no resources are allocated to a media channel to correspond to an expected baseline). Media channels can include, for example, digital media, search, TV, and sales.

[0072] In some implementations, if the user has provided consent, the training examples can be provided by the user computing system 1002. Thus, in such implementations, the model(s) 1020 provided to the user computing system 1002 can be trained by the training computing system 1050 on user-specific data received from the user computing system 1002. In some instances, this process can be referred to as personalizing the model.

[0073] The model trainer 1060 includes computer logic utilized to provide desired functionality. The model trainer 1060 can be implemented in hardware, firmware, and / or software controlling a general purpose processor. For example, in some implementations, the model trainer 1060 includes program files stored on a storage device, loaded into a memory and executed by one or more processors. In other implementations, the model trainer 1060 includes one or more sets of computer-executable instructions that are stored in a tangible computer-readable storage medium such as RAM, hard disk, or optical or magnetic media.

[0074] The network 1080 can be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., Internet), or some combination thereof and can include any number of wired or wireless links. In general, communication over the network 1080 can be carried via any type of wired and / or wireless connection, using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0075] The machine-learned models described in this specification may be used in a variety of tasks, applications, and / or use cases.

[0076] In some implementations, the input to the machine-learned model(s) of the present disclosure can be statistical data. Statistical data can be, represent, or otherwise include data computed and / or calculated from some other data source. The machine-learned model(s) can process the statistical data to generate an output. As an example, the machine-learned model(s) can process the statistical data to generate a recognition output. As another example, the machine-learned model(s) can process the statistical data to generate a prediction output. As another example, the machine-learned model(s) can process the statistical data to generate a classification output. As another example, the machine-learned model(s) can process the statistical data to generate a segmentation output. As another example, the machine-learned model(s) can process the statistical data to generate a visualization output. As anotherexample, the machine-learned model(s) can process the statistical data to generate a diagnostic output.

[0077] In some implementations, the input to the machine-learned model(s) of the present disclosure can be text or natural language data. The machine-learned model(s) can process the text or natural language data to generate an output. As an example, the machine-learned model(s) can process the natural language data to generate a language encoding output. As another example, the machine-learned model(s) can process the text or natural language data to generate a latent text embedding output. As another example, the machine-learned model(s) can process the text or natural language data to generate a translation output. As another example, the machine-learned model(s) can process the text or natural language data to generate a classification output. As another example, the machine-learned model(s) can process the text or natural language data to generate a textual segmentation output. As another example, the machine-learned model(s) can process the text or natural language data to generate a semantic intent output. As another example, the machine-learned model(s) can process the text or natural language data to generate an upscaled text or natural language output (e.g., text or natural language data that is higher quality than the input text or natural language, and the like). As another example, the machine-learned model(s) can process the text or natural language data to generate a prediction output.

[0078] In some implementations, the input to the machine-learned model(s) of the present disclosure can be sensor data. The machine-learned model(s) can process the sensor data to generate an output. As an example, the machine-learned model(s) can process the sensor data to generate a recognition output. As another example, the machine-learned model(s) can process the sensor data to generate a prediction output. As another example, the machine- learned model(s) can process the sensor data to generate a classification output. As another example, the machine-learned model(s) can process the sensor data to generate a segmentation output. As another example, the machine-learned model(s) can process the sensor data to generate a visualization output. As another example, the machine-learned model(s) can process the sensor data to generate a diagnostic output. As another example, the machine-learned model(s) can process the sensor data to generate a detection output.

[0079] FIG. 10 illustrates one example computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some implementations, the user computing system 1002 can include the model trainer 1060 and the training data 1062. In such implementations, the models 1020 can be both trained and used locally at the user computing system 1002. In some of such implementations, the usercomputing system 1002 can implement the model trainer 1060 to personalize the models 1020 based on user-specific data.

[0080] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken, and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0081] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such alterations, variations, and equivalents.

[0082] The depicted and / or described steps are merely illustrative and can be omitted, combined, and / or performed in an order other than that depicted and / or described; the numbering of depicted steps is merely for ease of reference and does not imply any particular ordering is necessary or preferred.

[0083] The functions and / or steps described herein can be embodied in computer-usable data and / or computer-executable instructions, executed by one or more computers and / or other devices to perform one or more functions described herein. Generally, such data and / or instructions include routines, programs, objects, components, data structures, or the like that perform particular tasks and / or implement particular data types when executed by one or more processors in a computer and / or other data-processing device. The computer-executable instructions can be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, read-only memory (ROM), random-access memory (RAM), or the like. As will be appreciated, the functionality of such instructions canbe combined and / or distributed as desired. In addition, the functionality can be embodied in whole or in part in firmware and / or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or the like. Particular data structures can be used to implement one or more aspects of the disclosure more effectively, and such data structures are contemplated to be within the scope of computer-executable instructions and / or computer-usable data described herein.

[0084] Although not required, one of ordinary skill in the art will appreciate that various aspects described herein can be embodied as a method, system, apparatus, and / or one or more computer-readable media storing computer-executable instructions. Accordingly, aspects can take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, and / or an embodiment combining software, hardware, and / or firmware aspects in any combination.

[0085] As described herein, the various methods and acts can be operative across one or more computing devices and / or networks. The functionality can be distributed in any manner or can be located in a single computing device (e.g., server, client computer, user device, or the like).

[0086] Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications, and / or variations within the scope and spirit of the appended claims can occur to persons of ordinary skill in the art from a review of this disclosure. For example, one or ordinary skill in the art can appreciate that the steps depicted and / or described can be performed in other than the recited order and / or that one or more illustrated steps can be optional and / or combined. Any and all features in the following claims can be combined and / or rearranged in any way possible.

[0087] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and / or equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations, and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated and / or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such alterations, variations, and / or equivalents

[0088] Figure 10 illustrates one example computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some implementations, the user computing system 1002 can include the model trainer 1060 and the training data 1062. In such implementations, the models 1020 can be both trained and used locally at the user computing system 1002. In some of such implementations, the user computing system 1002 can implement the model trainer 1060 to personalize the models 1020 based on user-specific data.

[0089] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken, and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0090] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such alterations, variations, and equivalents.

[0091] The depicted and / or described steps are merely illustrative and can be omitted, combined, and / or performed in an order other than that depicted and / or described; the numbering of depicted steps is merely for ease of reference and does not imply any particular ordering is necessary or preferred.

[0092] The functions and / or steps described herein can be embodied in computer-usable data and / or computer-executable instructions, executed by one or more computers and / or other devices to perform one or more functions described herein. Generally, such data and / or instructions include routines, programs, objects, components, data structures, or the like thatperform particular tasks and / or implement particular data types when executed by one or more processors in a computer and / or other data-processing device. The computer-executable instructions can be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, read-only memory (ROM), random-access memory (RAM), or the like. As will be appreciated, the functionality of such instructions can be combined and / or distributed as desired. In addition, the functionality can be embodied in whole or in part in firmware and / or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or the like. Particular data structures can be used to implement one or more aspects of the disclosure more effectively, and such data structures are contemplated to be within the scope of computer-executable instructions and / or computer-usable data described herein.

[0093] Although not required, one of ordinary skill in the art will appreciate that various aspects described herein can be embodied as a method, system, apparatus, and / or one or more computer-readable media storing computer-executable instructions. Accordingly, aspects can take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, and / or an embodiment combining software, hardware, and / or firmware aspects in any combination.

[0094] As described herein, the various methods and acts can be operative across one or more computing devices and / or networks. The functionality can be distributed in any manner or can be located in a single computing device (e.g., server, client computer, user device, or the like).

[0095] Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications, and / or variations within the scope and spirit of the appended claims can occur to persons of ordinary skill in the art from a review of this disclosure. For example, one or ordinary skill in the art can appreciate that the steps depicted and / or described can be performed in other than the recited order and / or that one or more illustrated steps can be optional and / or combined. Any and all features in the following claims can be combined and / or rearranged in any way possible.

[0096] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and / or equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations, and / or additions to the present subject matter as would be readilyapparent to one of ordinary skill in the art. For instance, features illustrated and / or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such alterations, variations, and / or equivalents.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method comprising: accessing message data indicative of an update; transmitting, from a source cluster, a first replica of the message data via a nonvolatile storage of the source cluster to a nonvolatile storage of a destination cluster; transmitting, from the source cluster, in parallel to transmitting the first replica, a second replica of the message data via a data cache of the source cluster to a data cache of the destination cluster by: splitting the second replica of the message data into a plurality of data chunks comprising at least a first data chunk; writing, by a writer of the source cluster, the first data chunk and first metadata to a memory of the data cache; reading, by a reader of the destination cluster, the first metadata and the first data chunk; and serving, by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster.

2. The method of claim 1, comprising: determining that a second data chunk is missing from the data cache; and responsive to determining that the second data chunk is missing from the data cache, reading the first replica of the message data from the nonvolatile storage of the destination cluster.

3. The method of any preceding claim, wherein the message data comprises a content item.

4. The method of any preceding claim, wherein the message data comprises a current inventory update.

5. The method of any preceding claim, wherein transmitting the second replica of the message data via the data cache is performed in under a few seconds.

6. The method of any preceding claim, wherein each of the plurality of data chunks are the same size.

7. The method of any preceding claim, wherein the data cache comprises a plurality of memory locations, wherein the method comprises: selecting a memory location for performing the writing and reading based on a capacity associated with the memory location.

8. A computing system, comprising: one or more processors; and one or more computer-readable media storing instructions that are executable to cause the one or more processors to perform operations, the operations comprising: accessing message data indicative of an update; transmitting, from a source cluster, a first replica of the message data via a nonvolatile storage of the source cluster to a nonvolatile storage of a destination cluster; transmitting, from the source cluster, in parallel to transmitting the first replica, a second replica of the message data via a data cache of the source cluster to a data cache of the destination cluster by: splitting the second replica of the message data into a plurality of data chunks comprising at least a first data chunk; writing, by a writer of the source cluster, the first data chunk to a memory of the data cache; reading, by a reader of the destination cluster, the first data chunk; serving, by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster; determining that a second data chunk is missing from the data cache; and responsive to determining that the second data chunk is missing from the data cache, reading the first replica of the message data from the nonvolatile storage of the destination cluster.

9. The computing system of claim 8, comprising: writing, by the writer of the destination cluster, a first metadata associated with the first data chunk to the memory of the data cache; andreading, by the reader of the destination cluster, the first metadata associated with the first data chunk to the memory of the data cache.

10. The computing system of any of claim 8 to 9, wherein the message data comprises a content item.

11. The computing system of any of claim 8 to 10, wherein the message data comprises a current inventory update.

12. The computing system of any of claim 8 to 11, wherein transmitting the second replica of the message data via the data cache is performed in under a few seconds.

13. The computing system of any of claim 8 to 12, wherein each of the plurality of data chunks are the same size.

14. The computing system of any of claim 8 to 13, wherein the data cache comprises a plurality of memory locations, wherein the operations comprise: selecting a memory location for performing the writing and reading based on a capacity associated with the memory location.

15. One or more transitory or non-transitory computer readable media storing instructions that are executable by one or more processors to perform operations comprising: accessing message data indicative of an update; transmitting, from a source cluster, a first replica of the message data via a nonvolatile storage of the source cluster to a nonvolatile storage of a destination cluster; transmitting, from the source cluster, in parallel to transmitting the first replica, a second replica of the message data via a data cache of the source cluster to a data cache of the destination cluster by: splitting the second replica of the message data into a plurality of data chunks comprising at least a first data chunk; writing, by a writer of the source cluster, the first data chunk and first metadata to a memory of the data cache; reading, by a reader of the destination cluster, the first metadata and the first data chunk; andserving, by a serving device of the destination cluster, the first data chunk to a plurality of devices associated with the destination cluster.

16. The one or more transitory or non-transitory computer readable media of claim 15, the operations comprising: determining that a second data chunk is missing from the data cache; and responsive to determining that the second data chunk is missing from the data cache, reading the first replica of the message data from the nonvolatile storage of the destination cluster.

17. The one or more transitory or non-transitory computer readable media of any of claim 15 to claim 16, wherein the message data comprises a content item:

18. The one or more transitory or non-transitory computer readable media of any of claim 15 to claim 17, wherein the message data comprises a current inventory update.

19. The one or more transitory or non-transitory computer readable media of any of claim 15 to claim 18, wherein transmitting the second replica of the message data via the data cache is performed in under a few seconds.

20. The one or more transitory or non-transitory computer readable media of any of claim 15 to claim 19, wherein the data cache comprises a plurality of memory locations, wherein the operations comprise: selecting a memory location for performing the writing and reading based on a capacity associated with the memory location.

Images