Buffering elements for processing

Abstract

Techniques related to buffers are disclosed. The first buffer may be used to store a plurality of elements for processing. The disclosed techniques include providing a second buffer for storing elements for addition to the first buffer, and adding the obtained elements to the second buffer when the number of elements in the first buffer is greater than a first threshold. These techniques may prevent overflow of the first buffer while allowing the obtained element to be added to the buffer.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application No. 18 / 084,276, filed on December 19, 2022, the entire contents of which are incorporated herein by reference. Background Art

[0003] In computing, a buffer is generally understood to refer to a portion of memory used to hold data elements for processing. For example, incoming data elements to be processed by a processor can be added to a buffer. When the processor is ready to process one or more elements, the processor (or its thread) can remove one or more elements from the buffer and process them. The buffer is configured so that the processing rate of elements does not necessarily need to match the rate at which elements are received into the buffer. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Certain embodiments are disclosed with reference to the following drawings.

[0005] Figure 1 A block diagram of an example computing device that may be used to implement certain embodiments is shown.

[0006] Figure 2 A block diagram of an example system is shown in which certain embodiments may be implemented.

[0007] Figure 3 A block diagram is shown of another example system in which certain embodiments may be implemented.

[0008] Figure 4 A block diagram of queues and worker threads according to an example is shown.

[0009] Figure 5 A block diagram illustrating data flow between queues according to an example is shown.

[0010] Figure 6 A block diagram of a connector according to an example is shown.

[0011] Figure 7 A flow diagram of a process performed by a connector is shown according to an example.

[0012] Figure 8 A flow chart illustrating another process performed by a connector according to an example is shown.

[0013] Figure 9 A flow chart of a method performed by a connector according to an example is shown.

[0014] Figure 10 A block diagram of an example electronic transaction system is shown in which certain embodiments may be employed.

[0015] Figure 11 A block diagram of another example electronic transaction system in which certain embodiments may be employed is shown.

[0016] Certain embodiments will be better understood when read in conjunction with the accompanying drawings which illustrate examples. It should be understood, however, that the embodiments are not limited to the arrangements and instrumentality shown in the drawings. DETAILED DESCRIPTION

[0017] The disclosed embodiments relate generally to buffers, and more particularly to systems and methods in which elements are buffered for processing. A buffer, such as a ring buffer, may have a capacity for a specific number of elements. This presents a risk of buffer overrun, where the number of elements added to the buffer exceeds the buffer's capacity, potentially causing elements to overwrite adjacent memory locations. In some cases, these adjacent memory locations may contain unprocessed data in the buffer, and overwriting this unprocessed data may result in data loss.

[0018] To prevent buffer overflows, flow control techniques can be employed. Specifically, when the number of elements in the buffer approaches or reaches the buffer's upper capacity, flow control can be activated. When flow control is activated, a thread can be restricted from adding another element to the buffer. Instead, the thread can spin (i.e., repeatedly attempt to add elements to the buffer) until the number of elements in the buffer decreases.

[0019] However, this technique has drawbacks. One drawback is that it may affect the process of adding elements to the buffer. Specifically, such a process may be forced to wait until the number of elements in the buffer has decreased before continuing, thereby blocking the process.

[0020] In certain cases, this blocking can in turn create a risk of deadlock. Deadlock can refer to a situation where two (or more) interdependent processes cannot logically progress. As an example, a particular thread can work on an element in a particular buffer and also add elements to the particular buffer. However, if the buffer approaches or reaches an upper capacity limit such that the above flow control is activated, the thread will spin when attempting to add elements to the buffer. And since the thread is spinning, the thread cannot complete work on the element in the buffer, so the element will not be removed from the buffer. Thus, the number of elements in the buffer will not decrease, and the thread can spin indefinitely; neither the addition nor the removal of elements can logically progress, so a deadlock situation occurs. As another example, deadlock can occur when a loop is established between two or more buffers. For example, if a first thread that removes elements from a first buffer and adds elements to a second buffer is spinning (e.g., because the second buffer approaches or reaches an upper capacity limit, so flow control is activated), and a second thread that removes elements from the second buffer and adds elements to the first buffer is spinning (e.g., because the first buffer approaches or reaches an upper capacity limit, so flow control is activated), then both threads cannot logically progress; a deadlock situation occurs.

[0021] Deadlock is undesirable because it can abort processing and require restarting of processing, which is an inefficient use of resources, and can also bring undesirable consequences to the processing itself. As an example, deadlock in a real-time message processing system can result in message loss, which can be highly undesirable. Furthermore, the risk of deadlock itself is undesirable because it can impose restrictions on available code involving the buffers. For example, to reduce the risk of deadlock, code involving or at risk of establishing a loop between buffers can need to be avoided or rewritten. This can limit the ways in which buffers can be used. It is desirable to mitigate at least some of these drawbacks.

[0022] I. Brief Description of Specific Embodiments

[0023] Certain embodiments provide a system comprising: a memory configured to provide: a first buffer for storing a plurality of elements for processing; and a second buffer for storing elements to be added to the first buffer. The system further comprises at least one processor configured to: obtain an element to be processed; determine if the number of elements in the first buffer is greater than a first threshold; and add the obtained element to the second buffer. By providing a second buffer for storing elements to be added to the first buffer and, when the number of elements in the first buffer is greater than the first threshold, adding the obtained element to the second buffer, overflow of the first buffer can be prevented while still allowing the processor to add elements to the buffer. Thus, a process attempting to add an element to the first buffer does not need to wait until the number of elements in the first buffer decreases; instead, the at least one processor adds the element to the second buffer. Blocking of such a process can thus be avoided, and the overall speed of such a process can be improved. Furthermore, the risk of deadlock can be eliminated. Specifically, even when the number of elements in the first buffer is greater than the first threshold, the at least one processor (specifically, its worker thread) does not spin in an attempt to add an element to the first buffer, but instead adds the element to the second buffer. Thus, even when the number of elements in the first buffer is greater than the first threshold, the worker thread will be able to logically advance, thereby eliminating the risk of deadlock involving the first buffer. Eliminating the risk of deadlock can, in turn, avoid or reduce the risk of needing to restart processing, which can improve the overall efficiency of the system and / or help avoid losing elements in the buffer. Furthermore, eliminating the risk of deadlock in the first buffer can increase the flexibility with which the first buffer can be used. For example, because the risk of deadlock is eliminated, code can be written without having to avoid loops between buffers, which in turn can allow for simpler and / or more efficient code. Furthermore, eliminating the risk of deadlock in the first buffer can allow the size of the first buffer to be reduced, thereby reducing memory requirements. For example, without a second buffer, the first buffer might need to be relatively large to reduce the risk of deadlock, for example, based on the expected peak element addition rate or buffer usage. However, by eliminating the risk of deadlock in the second buffer, the first buffer (which can, for example, be provided by a ring buffer) does not need to be sized based on these parameters and can be made smaller without risk of deadlock.

[0024] In some embodiments, at least one processor is further configured to add the obtained element to the first buffer based at least in part on determining that the number of elements in the first buffer is less than a first threshold. This can be provided in "normal" operation, i.e., when the number of elements in the first buffer is less than the first threshold, the element is added to the first buffer. This provides for efficient processing of elements, i.e., only the first buffer is involved in normal operation, while providing for the second buffer to be used if the first buffer approaches or reaches capacity, thereby eliminating the risk of deadlock, for example.

[0025] In some embodiments, at least one processor is configured to set an overflow flag when the number of elements in the first buffer is greater than a first threshold; and wherein the at least one processor is configured to determine that the number of elements in the first buffer is greater than the first threshold by determining that the overflow flag is set. This provides an efficient way to implement the determination that the number of elements in the first buffer is greater than the threshold. If the at least one processor detects the overflow flag and it is set, the at least one processor can directly add the obtained elements to the second buffer. This can be more computationally efficient than, for example, obtaining a count of the number of elements in the first buffer and then comparing the count to the first threshold.

[0026] In some embodiments, at least one processor is further configured to: determine that the number of elements in the first buffer is less than a second threshold, where the second threshold is less than the first threshold; and remove elements from the second buffer and add the removed elements to the first buffer. This helps ensure that elements are removed from the second buffer and added to the first buffer when the number of elements in the first buffer has been sufficiently reduced, e.g., there is no risk of the first buffer overflowing. This helps ensure that the first buffer (which can be configured as a ring buffer for high performance, for example) is used to process elements. It should be noted that as long as the second threshold is equal to or greater than one element, using the second buffer does not add delay to the processing of elements from the first buffer.

[0027] In some embodiments, the first buffer is a first queue for queuing multiple elements for processing, and the second buffer is a second queue for queuing elements for addition to the first queue. This helps provide that elements can be processed in order (i.e., in the order in which they were obtained). This may be important for certain elements, such as protocol messages used to maintain a session between a client device and a host, as well as some other types of messages.

[0028] In some embodiments, the at least one processor is further configured to: determine that the number of elements in the second buffer is greater than zero; and add the obtained elements to the second buffer. This can help ensure that the elements are processed in the order in which they are obtained. For example, if there is an element already waiting in the second queue, the at least one processor does not deliver it to the first queue, but instead delivers it to the second queue. As described above, processing elements in the order in which they are obtained may be important for certain types of elements, and this may help achieve this sequential processing requirement.

[0029] In some embodiments, the at least one processor is further configured to: determine that the second buffer is empty and the number of elements in the first buffer is less than a first threshold; and add the obtained element to the first buffer. This can help ensure that the elements are processed in the order in which they were obtained. For example, the at least one processor may add the obtained message to the first buffer only when there are no elements waiting in the second buffer and the number of messages in the first buffer is less than a first threshold. This may help achieve sequential processing requirements.

[0030] In some embodiments, at least one processor is configured to: set an overflow flag when the number of elements in the first buffer is greater than a first threshold; and reset the overflow flag when the second buffer is determined to be empty, wherein the at least one processor is configured to determine that the second buffer is empty and the number of elements in the first buffer is less than the first threshold by determining that the overflow flag is reset. This can provide an efficient way to determine that the second buffer is empty and the number of elements in the first buffer is less than the first threshold; that is, by referring to a single flag. This can be more computationally efficient than, for example, making the corresponding determinations individually and / or explicitly each time.

[0031] In some embodiments, the first buffer is a ring buffer. This allows for high performance. Specifically, due to the contiguous nature of the memory in the ring buffer, elements can be read quickly from it. Ring buffers are inherently finite in size, which means that providing a second buffer can provide the performance of a ring buffer while also eliminating the risk of deadlock.

[0032] In some embodiments, the second buffer is a queue for adding elements to the first buffer. This may help achieve sequential processing requirements. In some embodiments, the second buffer is an unbounded buffer. This can ensure that the second buffer does not become full. For practical purposes, the second buffer can be unbounded. That is, although the second buffer may technically have a capacity, the capacity may be so large that for practical purposes the second buffer can be considered unbounded. That is, for practical purposes, there may be no risk of the second buffer becoming full. This can help reduce or eliminate the risk of buffer overflow in the second buffer.

[0033] In certain embodiments, at least one processor is configured to implement multiple worker threads that add to the first buffer, where the first threshold is at most the total capacity of elements that the first buffer has minus the number of worker threads. This can provide a maximum value for the first threshold while still ensuring that the first buffer does not exceed the limit. For example, each worker thread can add elements to the first buffer one element at a time. Even if all total worker threads add elements to the first buffer at the same time, the first buffer will not exceed the limit, and the worker threads will continue to add elements to the second buffer. This can help ensure that no elements are lost or overwritten, while also ensuring that the capacity of the first buffer can be utilized.

[0034] In some embodiments, the element is a message, and the at least one processor is configured to obtain the element by receiving the message from the external entity via the communication interface. This can help ensure that the process receiving the message at the communication interface is not blocked if the number of messages in the first buffer is greater than a first threshold. This in turn can help ensure that messages are not lost.

[0035] In some embodiments, at least one processor is configured to obtain elements by removing elements from a third buffer. In an example, processing elements from a first buffer may include adding elements to a third buffer. This situation may typically create a risk of deadlock. However, when the number of elements in the first buffer is greater than a threshold, having at least one processor add the obtained elements to a second buffer eliminates this risk of deadlock. Therefore, even if processing elements from a first buffer includes adding elements to a third buffer, elements can be safely obtained from any third buffer (i.e., without risk of deadlock). In an example, the first buffer may be an order queue, and the third buffer may be a worker queue.

[0036] In some embodiments, each element is a reference to one or more message queues. The first queue can be a worker queue to which references to one or more message queues, such as a session queue or an action queue, can be added. This can eliminate the risk of process blocking and / or deadlock even when the first queue is a buffer to which references to one or more message queues are added by a worker thread.

[0037] In certain embodiments, the element relates to a message, the system includes a server system, the server system includes at least one processor and a memory, and the server system is configured to promote the transmission of messages between the host system and one or more client systems. For example, the server system can implement a connector that promotes the transmission of messages between the host system and one or more client systems. It may be particularly important to avoid process blocking and / or deadlock in such a server system, because if a message such as a protocol message is not processed within a certain period of time, the connection between the host system and one or more client systems may be discarded. In an example, an element can relate to a message, because an element can be a message, a reference to a message, a reference to a message queue, or a reference to a queue of a reference to a message.

[0038] In certain embodiments, one or more client systems include trading devices, and the host system includes an electronic exchange. Avoiding process blocking and / or deadlock in such systems can be particularly important, as message timing between the trading device and the electronic exchange can be critical. Furthermore, eliminating the risk of deadlock reduces the risk of needing to restart the system, which can otherwise involve lost messages and / or resource-intensive message reconciliation processes. Consequently, a more efficient and / or consistent trading connector can be provided.

[0039] In certain embodiments, a server system is configured to establish multiple connections between a host system and corresponding multiple client systems, wherein a memory is configured to provide a first buffer and an associated second buffer for each of the multiple connections, and wherein at least one processor is configured to, for at least some of the connections, add an element (such as a message or a reference to a message) to the corresponding first buffer or second buffer for that connection. Having different first and second buffers for each different connection can help ensure that a burst of message transmissions on a first connection does not affect the processing rate on a second connection; instead, the burst of messages is added to the second buffer for the first connection. This can allow the system to operate consistently across the connections. It can also allow for parallel processing of messages from different connections, which can help increase processing speed.

[0040] In some embodiments, messaging between a host system and one or more client systems is bidirectional, a memory is configured to provide a first buffer and an associated second buffer for each of a first messaging direction and a second messaging direction, and at least one processor is configured to, for each messaging direction, add a message element (such as a message or a reference to a message) to the corresponding first buffer or second buffer for that direction. Having different first and second buffers for each messaging direction can help ensure that a burst of messages in one direction does not affect the processing rate in the other direction. This can allow, for example, messages from the host to be delivered to the client devices even when processing of messages from one or more client devices to the host is relatively slow. This can also allow for parallel processing of messages in different messaging directions, which in turn helps increase processing speed.

[0041] In certain embodiments, a server system includes an order connector for an electronic trading platform, and the order connector includes at least one processor and memory. An electronic trading platform can be a specific environment where it is important to process messages (e.g., trade orders and / or run reports) quickly, the connection between the trading device and the exchange is not down (as this could result in the cancellation of all trade orders for a session), certain sequencing requirements are met, and / or the processing rate on one connection is not affected by the processing rate on another connection. Thus, this is a context in which certain embodiments may find particular utility.

[0042] Certain embodiments provide a method comprising: providing, in a memory, a first buffer for storing a plurality of elements for processing; and a second buffer for storing elements to be added to the first buffer. The method further comprises: obtaining, by at least one processor, an element to be processed; determining that the number of elements in the first buffer is greater than a first threshold; and adding the obtained element to the second buffer.

[0043] Certain embodiments provide a tangible computer-readable medium comprising instructions that, when executed, cause a machine to at least: provide in a memory: a first buffer for storing elements for processing; and a second buffer for storing elements for adding to the first buffer; and by at least one processor: obtain an element to be processed; determine that the number of elements in the first buffer is greater than a first threshold; and add the obtained element to the second buffer.

[0044] II. Example Computing Device

[0045] Figure 1A block diagram of an example computing device 100 is shown. Computing device 100 can be used to implement certain embodiments described herein. In other examples, other computing devices may be used. Computing device 100 includes a communication bus 110, a processor 112, a memory 114, a network interface 116, an input device 118, and an output device 120. Processor 112, memory 114, network interface 116, input device 118, and output device 120 are coupled to communication bus 110. Computing device 100 is connected to an external network 140, such as a local area network (LAN) or a wide area network (WAN), such as the Internet. The computing device is connected to external network 140 via network interface 116. Computing device 100 may include additional, different, or fewer components. For example, multiple communication buses (or other types of component interconnections), multiple processors, multiple memory devices, multiple interfaces, multiple input devices, multiple output devices, or any combination thereof may be provided. As another example, computing device 100 may not include input device 118 or output device 120. As another example, one or more components of computing device 100 may be combined into a single physical element, such as a field programmable gate array (FPGA) or a system on a chip (SoC).

[0046] Communication bus 110 may include a channel, electrical or optical network, circuit, switch, fabric, or other mechanism for communicating data between components in computing device 100. Communication bus 110 may communicatively couple with and transfer data between any component of computing device 100.

[0047] The processor 112 may be any suitable processor, processing unit, or microprocessor. The processor 112 may include, for example, one or more general-purpose processors, digital signal processors, application-specific integrated circuits, FPGAs, analog circuits, digital circuits, programmed processors, and / or combinations thereof. The processor 112 may be a multi-core processor that may include multiple processing cores of the same or different types. The processor 112 may be a single device or a combination of devices, such as one or more devices associated with a network or distributed processing system. The processor 112 may support various processing strategies, such as multiprocessing, multitasking, parallel processing, and / or remote processing. Processing may be local or remote and may be moved from one processor to another. In some embodiments, the computing device 100 is a multi-processor system and, therefore, may include one or more additional processors communicatively coupled to the communication bus 110.

[0048] Processor 112 may be operable to execute logic and other computer-readable instructions encoded in one or more tangible media, such as memory 114. As used herein, logic encoded in one or more tangible media includes instructions executable by processor 112 or a different processor. For example, the logic may be stored as part of software, hardware, an integrated circuit, firmware, and / or microcode. The logic may be received from an external communication device via communication network 140. Processor 112 may execute the logic to perform the functions, actions, or tasks described herein.

[0049] The memory 114 can be one or more tangible media, such as, for example, computer-readable storage media. Computer-readable storage media can include various types of volatile and non-volatile storage media, including, for example, random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, any combination thereof, or any other tangible data storage device. As used herein, the term non-transitory or tangible computer-readable medium is expressly defined to include any type of computer-readable medium and to exclude propagating signals. The memory 114 can include any desired type of mass storage device, including a hard drive, optical media, magnetic tape or disk, etc.

[0050] The memory 114 may include one or more memory devices. For example, the memory 114 may include cache memory, local memory, mass storage devices, volatile memory, non-volatile memory, or a combination thereof. The memory 114 may be adjacent to the processor 112, part of the processor 112, programmed with the processor 112, networked with the processor 112, and / or remote from the processor 112, so that data stored in the memory 114 can be retrieved and processed, for example, by the processor 112. The memory 114 may store instructions executable by the processor 112. The instructions may be executed to perform one or more of the actions or functions described herein.

[0051] The memory 114 may store an application 130 that implements the disclosed technology. In some embodiments, the application 130 may be accessed from or stored in different locations. The processor 112 may access the application 130 stored in the memory 114 and execute computer-readable instructions included in the application 130.

[0052] The network interface 116 may include one or more network adapters. The network adapter may be a wired or wireless network adapter. The network interface 116 may allow the computing device 100 to communicate with the external network 140. The computing device 100 may communicate with other devices via the network interface 116 using one or more network protocols such as Ethernet, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), wireless network protocols such as Wi-Fi, Long Term Evolution (LTE) protocol, or other suitable protocols.

[0053] Input devices 118 may include pointing input devices, such as a mouse, touchpad, touch screen, etc.; keyboards, buttons, switches, etc.; and / or other human-computer interface devices. Output devices 120 may include a display, which may be a liquid crystal display (LCD), a cathode ray tube (CRT), a light emitting diode (LED) display (such as an OLED display), or other suitable display.

[0054] In some embodiments, during the installation process, the application may be transferred from the input device 118 and / or the network 140 to the memory 114. When the computing device 100 is running or preparing to run the application 130, the processor 112 may retrieve instructions from the memory 114 via the communication bus 110.

[0055] III. Example System

[0056] Figure 2 2 shows an example system 200 in which some embodiments disclosed herein may be employed. The system 200 includes client devices 202a, 202b, a connector 204, and a host 210. Each of the client devices 202a, 202b, the connector 204, and the host 210 may be operated by a corresponding computing device (such as a computer system described above). Figure 1 The computing device 100 of any example described herein is provided. The connector 204 may be provided by a server system. Figure 2 As shown, the connector 204 includes a processor 206 and a memory 208. The processor 206 and / or the memory 208 can be respectively connected to the processor 206 and / or the memory 208. Figure 1 The processor 112 and / or memory 114 of the computing device 100 are identical or similar. Each client device 202a, 202b communicates with the connector 204 via a first network 211. The first network 211 may include, for example, a local area network (LAN) and / or a wide area network (WAN) such as the Internet. The connector 204 communicates with the host 210 via a second network 212. The second network 212 may include a LAN, a WAN, a shared memory system, and / or a private network. In an example, the first network 211 and the second network 212 may be the same network.

[0057] The connector 204 is located between the client devices 202a, 202b and the host 210. In this example, the connector 204 acts as a mediator between the client devices 202a, 202b and the host 210. Specifically, the connector 204 performs protocol conversion on the data communicated between the client devices 202a, 202b and the host 210. In this example, the connector 204 processes messages received from the client devices 202a, 202b into a data format understood by the host 210. The connector 204 also processes messages received from the host 210 into a format understood by the client devices 202a, 202b. For example, the client devices 202a, 202b can send a request message to the host 210 via the connector 204, and the host 210 can execute the request in the request message and send a response message to the client devices 202a, 202b via the connector 204, indicating the extent to which the request has been executed. The message is transmitted by the connector 204 on the network interface ( Figure 1 Not shown, but see Figure 1 Messages received by connector 204 are processed by processor 206. Before being processed, received messages are temporarily stored in memory 208, specifically, in a buffer implemented by memory 208, as described in more detail below.

[0058] Figure 3 An example system 300 is shown in which some embodiments disclosed herein may be employed. The example system 300 may be used in conjunction with the system described above with reference to Figure 2 The example system 200 is the same as or similar to the example system 200 described above. The example system includes client devices 302a, 302b, a connector 304 having a processor 306 and a memory 308, and a host 310. The client devices 302a, 302b, the connector 304, and the host 310 can be respectively the same as those described above. Figure 1 and Figure 2 The client devices 202a, 202b, connector 204, and host 210 of any of the described examples are the same or similar.

[0059] from Figure 3As can be seen in FIG, each client device 302a, 302b establishes a corresponding connection or session 320, 330 with a host 310 via a connector 304. Connector 304 can establish a separate connection with host 310 for each session 320, 330. Each session 320, 330 can be established using a suitable communication protocol. As an example of messages in a first session 320, first client device 302a can send a first message 322 (such as a request) to connector 304. Connector 304 can process first message 322 to generate a transformed version of first message 324, and connector 304 can send this transformed message 324 to host 310. Host 310 can perform an action based on received message 324 and send a second message 326 (such as a response) to connector 304. Connector 304 can process second message 326 to generate a transformed version of second message 328, and connector 304 can send this transformed message 328 to client device 302a. Each of messages 322, 324, 326, and 328 is within the same first session 320. Thus, each of messages 322, 324, 326, and 328 has the same session identifier. Similarly, in a second session 330, second client device 302b may send a first message 332 (such as a request) to connector 304. Connector 304 may process first message 332 to generate a transformed version of first message 334, and then send this transformed message 334 to host 310. Host 310 may perform an action based on received message 334 and send a second message 336 (such as a response) to connector 304. Connector 304 may process second message 336 to generate a transformed version of second message 338, and then send this transformed message 328 to client device 302a. Each of messages 332, 334, 336, and 338 is within the same second session 330. As such, each of the messages 332 , 334 , 336 , 338 has the same session identifier (which is different from the session identifiers of the messages in the first session 320 ).

[0060] Messages 322 , 332 received by the connector 304 from the client devices 302 a , 302 b are referred to herein as outbound messages, and messages 326 , 336 received by the connector 304 from the host 310 are referred to herein as inbound messages.

[0061] IV. Queuing

[0062] In some examples, it may be important for connector 304 to process some messages in the order in which they are received. One example is messages from client devices 302a, 302b (or optionally from host 310) related to a specific action on host 310. For example, for a specific action on host 310, a particular client device 302a may send a request message to request that the specific action be performed, and subsequently send a cancel message or a change message to cancel or change the request in the request message, respectively. Importantly, connector 304 processes these messages in the order in which they are received at order connector 304. If the cancel message or change message were sent to host 310 before the request message, the request would not be canceled or changed, respectively. Another example is protocol messages, such as those between a particular client device 302a, 302b and host 310, to establish or maintain sessions 320, 330. An example of a protocol message is a "heartbeat" message, which may be periodically sent from the host 310 to a particular client device 302a, 302b (or from the client device 302a, 302b to the host 310) to check that the particular client device 302a, 302b (or host 310) is still there and, therefore, that the associated session 320, 330 should be maintained. It may be important that the connector 304 processes the protocol messages in the order in which they are received at the connector 304, otherwise the steps of the protocol may not be executed in the expected order, which may result in the session ending.

[0063] In order for connector 304 to process certain messages in the order in which they were received, connector 304 utilizes a queue. A queue can refer to a buffer that stores elements to be processed in a specific order. Specifically, a queue is a first-in-first-out (FIFO) buffer, whereby elements are processed from the queue in the same order in which they were added to the queue. When messages are received at connector 304, the messages (or references to those messages) can be placed in a queue in memory 308, where they can be held until processor 306 is available to process them.

[0064] Processing all messages in the order in which they are received by connector 304 may impose a limit on the speed at which messages can be processed. Specifically, speed can be increased by parallelizing the processing of some messages. That is, processing some messages simultaneously. In particular, messages from different sessions 320, 330 can be processed simultaneously, provided that the messages are not ordered relative to one another. Furthermore, messages related to different actions of host 310 can be processed simultaneously, provided that the messages are not ordered relative to one another. Furthermore, inbound and outbound messages received at connector 304 can be processed simultaneously, provided that the messages received from different directions (i.e., inbound and outbound messages) are not ordered relative to one another.

[0065] Figure 4 An architecture 400 of buffers and worker threads that can be employed by connectors 204, 304 according to an example is shown. In this example, each buffer is a queue. The architecture 400 allows some messages to be processed in the order they are received, and also allows some messages to be processed simultaneously. Figure 4 , for the first session (such as the one referenced above Figure 3 The architecture 400 includes an outbound session queue 402 and an inbound session queue 404, as well as an outbound action queue set 406 and an inbound action queue set 408. The architecture 400 also includes a queue for a second session (such as the one described above with reference to Figure 3 The architecture 400 also includes an outbound session queue 412 and an inbound session queue 414 for the second session 330 , as well as an outbound action queue set 416 and an inbound action queue set 418 . The architecture 400 also includes a worker queue 420 and a worker thread pool 422 .

[0066] Each session queue 402, 404, 412, 414 relates to a specific session. Figure 4 Only queues associated with two sessions are shown. However, it should be understood that architecture 400 may include more or fewer session queues as needed, depending on the number of sessions established between the client device and the host. Each set of action queues 406, 408, 416, 418 relates to a specific session, and within a specific set, each action queue relates to a different action of the host. Each different action can be associated with a corresponding different action ID. The action ID can be included in all messages related to a specific work action of the host to identify the work action to which the message relates. Outbound queues 402, 412, 406, 408 relate to messages received from client devices, and inbound queues 404, 408, 414, 418 relate to messages received from the host. In this example, each session queue 402, 404, 412, 414 and action queue 406, 408, 416, 418 is a queue for one or more messages. In an example, session queues 402, 404, 412, 414 and / or action queues 406, 408, 416, 418 may not store one or more messages, but rather store one or more references to the corresponding one or more messages. For example, for a particular message, the message itself may be stored elsewhere in memory 208, 308, and the reference in queue 402, 404, 412, 414, 406, 408, 416, 418 may indicate the location where the message is stored and from which it can be accessed. Worker queue 420 may include references to queues for one or more messages, particularly references to session and / or action queues, as described in more detail below. That is, worker queue 420 may be a queue of queue references.

[0067] Queues 402, 404, 406, 408, 412, 414, 416, 418, 420 are each a memory of connector 304 (such as the one referenced above). Figure 3 Specifically, each queue 402, 404, 406, 408, 412, 414, 416, 418, 420 can be implemented as a ring buffer. Each worker thread 422 can represent a physical processor (such as the one described above). Figure 3 Each worker thread 422 in the pool is assigned a task, performs the task, and when the task is completed, the worker thread 422 returns to the pool ready to be assigned another task. Each worker thread 422 works on one task at a time, but different worker threads 422 in the pool can work on different tasks at the same time. Figure 4 , three worker threads 422 are shown, but it should be understood that more or fewer worker threads can be used as needed. The worker threads 422 obtain queue references to one or more messages of a referenced queue (such as session queues 402, 404, 412, 414 or action queues 406, 408, 416, 418) from the worker queue 422 and process the one or more messages of the referenced queue.

[0068] As an illustrative example of the operation of the architecture 400, within a first session, a first message 410 is received from a client device. The message 410 is added to the outbound session queue 402 associated with the first session. At this point, the session queue 402 can have one or more messages in it, depending on how many messages have been added to the session queue 402. The session queue 402 is added to the worker queue 420. That is, a reference to the session queue 402 is added to the worker queue 420. The worker thread 422 obtains the reference to the session queue 402 from the worker queue 420 and processes one or more messages of the referenced session queue 402. Specifically, for each of the one or more messages of the referenced session queue 402, the worker thread 422 de-serializes the message. If the message does not relate to a particular action and thus does not include an action ID, e.g., is a protocol message, the worker thread 422 can process the message as needed and forward it to the host as needed. However, if the message is related to a particular action, the worker thread 422 determines the action ID included in the message and adds the de-serialized message to the action queue 406 associated with the determined action ID. The worker thread 422 then adds the action queue 406 to the worker queue 420. That is, a reference to the action queue 406 is added to the worker queue 420. The worker thread 422 obtains the reference to the action queue 406 from the worker queue 420 and runs the processing of one or more messages of the referenced action queue 406. This can involve, for example, translating the messages so as to be understood by the host and forwarding the translated messages to the host.

[0069] While receiving first message 410, a second message 411 may be received from the host within the second session. Second message 411 is added to an inbound session queue 414 associated with the second session. Session queue 414 is added to worker queue 420. That is, a reference to session queue 414 is added to worker queue 420. Worker thread 422 obtains a reference to session queue 414 from worker queue 420 and processes one or more messages in the referenced session queue 414. Specifically, worker thread 422 deserializes each of the one or more messages in session queue 414. If the message does not involve a specific action and therefore does not include an action ID, such as a protocol message, worker thread 422 may process the message as needed and forward it to the host as needed. However, if the message involves a specific action, worker thread 422 determines the action ID included in the message and adds the deserialized message to action queue 418 associated with the determined action ID. Worker thread 422 then adds action queue 418 to worker queue 420. That is, a reference to the action queue 418 is added to the worker queue 420. The worker thread 422 obtains the reference to the action queue 418 from the worker queue 420 and runs the processing of one or more messages of the referenced action queue 418. For example, this may involve converting the message so that it is understood by the client device and forwarding the converted message to the client device.

[0070] According to architecture 400, protocol messages and messages associated with a particular action ID for a particular session and a particular message direction are processed in the order in which they are received. However, messages on different sessions, with different message directions, and / or associated with different action IDs can be processed simultaneously. Thus, architecture 400 can provide for rapid processing of messages while allowing some messages to be processed in the order in which they are received.

[0071] V. Flow Control and Deadlock

[0072] As mentioned, each queue 402, 404, 406, 408, 412, 414, 416, 418, 420 can be implemented as a circular buffer with a fixed size. When implemented as a circular buffer, each queue has a fixed capacity for messages (or references to messages, or references to queues, as appropriate). Therefore, for each queue, there is a possibility that the queue may become full. Adding a message to an already full queue implemented as a circular buffer will cause the buffer to overflow, and in particular, will overwrite the first message in the queue. This will cause the first message in the queue to be lost. To avoid this, a flow control scheme can be used. According to the flow control scheme, when the number of messages in a particular queue 402, 404, 406, 408, 412, 414, 416, 418, 420 reaches a certain threshold, the worker thread 422 is restricted from adding another message to the particular queue. Instead, the worker thread 422 spins (i.e., repeatedly attempts to add messages to the particular queue) until the number of messages in the particular queue decreases. While this prevents overruns, the control scheme has disadvantages. One particular disadvantage is the risk of deadlock.

[0073] Figure 5 Figure 2 shows an example situation where the above flow control scheme may cause deadlock. Figure 5 In the block diagram of FIG, a session queue 502, an action queue 506, a worker queue 520 and two worker threads 522a and 522b are shown. The session queue 502 and the action queue 506 may be, for example, the one used in the above reference. Figure 4 The session queues 402, 404, 412, 414 and action queues 406, 408, 416, 418 for any of the specific sessions and message directions described. The worker queue 520 and worker threads 522a, 522b can be respectively the same as those described above. Figure 4 The worker queue 420 and worker thread 422 are depicted as being the same or similar.

[0074] With reference to the above Figure 4 In a similar manner to the process described in Figure 5 , a reference to the session queue 502 may be added to the worker queue 520. The first worker thread 522a may obtain a reference to the session queue 502 from the worker queue 520, obtain a message from the referenced session queue 502, deserialize the message, and attempt to add the deserialized message to the action queue 506. However, the action queue 506 may approach or reach capacity, causing a flow control scheme to be activated for the action queue 506. In this case, the first worker thread 522a is prohibited from adding the deserialized message to the action queue 506. Instead, the worker thread 522a spins, i.e., repeatedly attempts to add the deserialized message to the action queue 506 (as determined by the worker queue 520). Figure 5). Meanwhile, the worker queue 520 itself may approach or reach its upper capacity, causing a flow control scheme to be activated for the worker queue 520. The second worker thread 552b may attempt to add a reference to the action queue 506 to the worker queue 520. However, because the flow control scheme has already been activated for the worker queue 520, the second worker thread 552b is prohibited from adding the reference to the action queue 506 to the worker queue 520. Instead, the second worker thread 552b spins, i.e., repeatedly attempts to add the reference to the action queue 506 to the worker queue 520 (as indicated by the Figure 5 (Indicated by an "X" in the header). In this case, first worker thread 522a is spinning, waiting for the number of deserialized messages in action queue 506 to decrease. However, because second worker thread 552b is spinning, waiting for the number of queue references in worker queue 520 to decrease, a reference to action queue 506 cannot be added to worker queue 520, and the number of deserialized messages in action queue 506 cannot decrease. Conversely, the number of queue references in worker queue 520 cannot decrease because first worker thread 552a is spinning. Message processing has reached a deadlock and cannot continue.

[0075] Because message processing cannot continue during a deadlock, protocol messages such as "heartbeat" messages will not be processed. If more than a certain number of protocol messages between a client device and a host remain unprocessed, or if a certain amount of time passes between protocol messages exchanged between the client device and the host, one or both of the host and client device may conclude that the other is no longer participating in the session and may close the session. This may mean that messages sent by one or both of the client device and the host may be lost.

[0076] Deadlocks can be broken by restarting connectors 204, 304, but this is time-consuming and resource-intensive, and also carries the risk of losing messages being queued or processed. Deadlocks are therefore undesirable, and it would be useful if the risk of deadlock could be eliminated. Furthermore, the risk of deadlock itself is undesirable because it may impose restrictions on the code that can be used involving queues. For example, to reduce the risk of deadlock between worker queue 520 and session queue 502, code that involves worker threads 522a, 522b adding messages to session queue 502, or code that carries the risk of worker threads 522a, 522b adding messages to session queue 502, needs to be avoided or rewritten. This may limit the ways in which queues can be used. It would be useful to allow the use of queues, or more generally, buffers, while eliminating the risk of deadlock.

[0077] It should be understood that Figure 5These are merely examples, and deadlock or the risk of deadlock may occur or exist in other example situations or systems. As an example, even when using a queue, there may be a risk of deadlock. For example, a particular thread may process messages from a particular queue and also add messages to the queue. However, if the queue becomes full, according to the control scheme described above, the thread will spin while attempting to add messages to the queue. Because the thread is spinning, it cannot process messages from the queue. Consequently, the number of messages in the queue will not decrease, and the thread will spin indefinitely; neither the addition nor the processing of messages can logically progress, and a deadlock situation will occur.

[0078] More generally, even in situations where there may not be a risk of deadlock, the control scheme described above has disadvantages. For example, the control scheme affects processes that add messages to a queue or, more generally, add elements to a buffer. Specifically, such a process may be forced to wait until the number of messages in the queue has decreased in order to proceed, which may block the process from progressing. This blocking may reduce the speed at which the process is executed, which in turn may reduce the speed at which the overall processing, such as that performed by connectors 204, 304, can be performed. As an example, a process may include the one described above with reference to Figures 2 to 4 For example, a control scheme may result in blocking the reception of outbound messages 322, 332, 410 and / or inbound messages 326, 336, 411 by the connector 304, which may result in unstable functionality of the entire system 300. It would be useful to mitigate these drawbacks.

[0079] VI. Overflow Queue

[0080] Figure 6 6 shows a system 600 in which embodiments may be employed. The system 600 includes a memory 604 and a processor 602. The memory 604 may be configured as described above with reference to FIG. Figures 1 to 4 The memory 114, 208, 308 of any of the examples described above may be the same or similar. The processor 602 may be the same as the one described above. Figures 1 to 4 The processors 112, 206, 306 of any of the examples described may be the same or similar. The processor 602 and the memory 604 may be part of the server system 600. In an example, the server system 600 may be configured to facilitate a host system (such as the one described above) Figures 1 to 4 Any of the example hosts 210, 310 described herein) and one or more client systems (such as those described above with reference to Figures 1 to 5 For example, the server system 600 may be a server system according to the above reference Figures 1 to 4A portion of or implementation of the connector 204 , 304 of any of the examples described.

[0081] In system 600, memory 604 is configured to provide a standard queue 612 (an example of a first buffer) and an overflow queue 610 (an example of a second buffer). Standard queue 612 is used to store multiple elements for processing. For example, for a particular queue 612, an element can be any one of a message, a reference to a message, and a reference to a queue. Overflow queue 610 is used to store elements to be added to standard queue 612. Overflow queue 610 is associated with standard queue 612 because it is used to store elements to be added to standard queue 612. Standard queue 612 has a fixed capacity, that is, it has a capacity for only a certain number of elements. In this example, standard queue 612 is a circular buffer. In this example, overflow queue 610 is unbounded. That is, overflow queue 610 can have an unbounded capacity for elements. This ensures that overflow queue 610 does not become full. It should be understood that, while all memories can theoretically be bounded, overflow queue 610 can be referred to as unbounded because the capacity is large enough or can be made large enough so that for practical purposes overflow queue 612 can be considered unbounded. That is, in some examples, for practical purposes, there may be no risk of overflow queue 612 becoming full, and therefore, for practical purposes, there is no risk of the buffer in overflow queue 612 being overrun.

[0082] Figure 6 The standard queue 612 in the above reference can represent Figures 1 to 5 That is, in the example, the standard queue 612 can represent any one of the session queues 402, 404, 412, 414, the action queues 406, 408, 416, 418, and the worker queue 420, such as those described above with reference to Figures 1 to 5 For example, where the standard queue 612 represents a session queue 402, 404, 412, 414, an element may be a message or a reference to a message, such as a serialized message. Where the standard queue 612 represents an action queue 406, 408, 416, 418, an element may be a message or a reference to a message, such as a deserialized message. Where the standard queue 612 represents a worker queue 420, an element may be a queue reference, i.e., a reference to a session queue 402, 404, 412, 414 and / or a reference to an action queue 406, 408, 416, 418. Although for the sake of brevity, in Figure 6only one standard queue 612 and one overflow queue 610 are shown, it will be appreciated that in some examples the memory 604 can be configured to provide multiple standard queues 612, e.g. one standard queue 612 for each of the session queues 402, 404, 412, 414, action queues 406, 408, 416, 418 and worker queue 420 described above with reference to Figures 1 to 5 each of one or more of the session queues 402, 404, 412, 414, action queues 406, 408, 416, 418 and worker queue 420 described above. In this case, the memory 604 can be configured to provide a corresponding plurality of overflow queues 610. For example, the memory 604 can be configured to provide one overflow queue 610 for each of the standard queues 612. In this case, each overflow queue 610 can be associated with a respective standard queue 612, as each overflow queue 610 is used to store elements for addition to the respective standard queue 612.

[0083] As described above with reference to Figure 3 and Figure 4 the connector 304 can establish multiple connections or sessions between the host system 310 and a respective plurality of client systems 302a, 302b. In these cases, where the system 600 implements the connector 304, the memory 604 can be configured to provide a standard queue 612 and associated overflow queue 610 for each of the multiple sessions. For example, there can be a first standard queue (e.g. session queue 402, 404 or action queue 406, 408) and associated first overflow queue for a first session, and a second standard queue (e.g. session queue 412, 414 or action queue 416, 418) and associated second overflow queue for a second session. Figure 4 Figure 4 As described above with reference to Figure 3 and Figure 4 the messaging between the host system 310 and the client systems 302a, 203b is bidirectional. In these cases, where the system 600 implements the connector 304, the memory 604 can be configured to provide a standard queue 612 and associated overflow queue 610 for each of a first messaging direction (e.g. outbound) and a second messaging direction (e.g. inbound). For example, there can be a first standard queue (e.g. session queue 402 or action queue 406) and associated first overflow queue for a first messaging direction (e.g. outbound), and a second standard queue (e.g. session queue 404 or action queue 408) and associated second overflow queue for a second messaging direction (e.g. inbound). Figure 4 Figure 4 As described above with reference to

[0084] Referring to Figure 6 ​​, the processor 602 is configured to obtain the element to be processed. In some examples, the processor 602 is configured to obtain the element to be processed via a communication interface (such as Figure 1 The network interface 116) receives a message from an external entity to obtain an element. For example, for an outbound message, the external entity may be the one referenced above. Figures 2 to 3 As another example, for inbound messages, the external entity may be the client device 202a, 202b, 302a, 302b described above. Figures 2 to 3 In an example, the processor 602 may receive a message and determine, based on the session on which the message is received and / or the message's messaging direction, which session queue 402, 404, 412, 414, 502 the message (or a reference to the message) is to be added to, e.g., as described above with reference to Figures 3 to 5 In such an example, the standard queue 612 may represent a particular session queue 402, 404, 412, 414, 502, and the overflow queue 610 may represent an overflow queue 610 for and associated with the particular session queue 402, 404, 412, 414.

[0085] In some examples, the processor 602 receives the data from the third queue ( Figure 6 For example, the processor 602 may implement a worker thread (e.g., the one referenced above). Figure 4 and Figure 5 408, 416, 418, 506, such as described above with reference to the worker thread 422, 522a, 522b), which removes the message from the referenced session queue 402, 404, 412, 414, 502 to deserialize the message and add the deserialized message (or a reference thereto) to the appropriate action queue 406, 408, 416, 418, 506. Figure 4 and Figure 5 In these examples, standard queue 612 may represent action queues 406, 408, 416, 418, 506, and a third queue (not shown) may represent session queues 402, 404, 412, 414, 502. In these examples, overflow queue 610 may represent an overflow queue 610 for and associated with action queues 406, 408, 416, 418, 506.

[0086] In some examples, the processor 602 obtains an element by obtaining a reference to a queue, such as a reference to a session queue 402, 404, 412, 414 or a reference to an action queue 406, 408, 416, 418, 506. For example, a worker thread may add a message (or a reference to a message) to a session queue 402, 404, 412, 414 or an action queue 406, 408, 416, 418, 506, and then add a reference to the session queue 402, 404, 412, 414 or the action queue 406, 408, 416, 418, 506 to a worker queue 420, 520, for example, as described above with reference to FIG. Figure 4 and Figure 5 In these examples, the standard queue 612 may represent a worker queue 420 , 520 , and the overflow queue 610 may represent an overflow queue 610 for and associated with the worker queue 420 , 520 .

[0087] Regardless of how the processor 602 obtains an element, for a particular standard queue 612 and associated overflow queue 610, the processor 602 is configured to add the obtained element to the standard queue 612 or the overflow queue 610 associated with the standard queue 612. Specifically, the processor 602 is configured to add the obtained element to the overflow queue 610 based on a determination that the number of elements in the standard queue 612 is greater than a first threshold value T1. The processor 602 is configured to add the obtained element to the standard queue 612 based, at least in part, on a determination that the number of elements in the standard queue 612 is less than the first threshold value T1. Thus, even if the number of elements in the standard queue 612 is greater than the first threshold value, the processor 602 (or its worker thread) is able to add the element to the queue, thereby avoiding process blocking and eliminating the risk of deadlock associated with the standard queue 612.

[0088] The first threshold T1 is less than the element capacity of the standard queue 612. In particular, the first threshold T1 can be set for the standard queue 612 to prevent overflow. That is, the first threshold can be set to the number of elements at which the standard queue 612 reaches or approaches its upper element capacity. For example, the standard queue 612 can be implemented as a ring buffer with a fixed number of element slots. Each element slot can have a capacity for only one element. Therefore, the ring buffer can have a capacity for a fixed number of elements. For example, if the elements are messages and the ring buffer holds messages, the ring buffer can have a capacity for a fixed number of messages. If the elements are references to messages and the ring buffer holds references to messages, the ring buffer can have a capacity for a fixed number of references to messages. If the elements are references to queues and the ring buffer holds references to queues, the ring buffer can have a capacity for a fixed number of references to queues. The first threshold can be set to be less than the total element capacity of the standard queue 612. This can help ensure that when the standard queue 612 reaches or approaches its upper element capacity, new elements are added to the overflow queue 610 instead, thereby reducing the risk of overflow. In an example, processor 602 may implement a specific number of worker threads (such as worker thread 422) that can add to standard queue 612. In an example, first threshold T1 may be at most the total capacity of elements held by standard queue 612 minus the number of worker threads. This can provide a maximum value for the first threshold while still ensuring that the first buffer does not overflow. For example, each worker thread may add elements to standard queue 612 one at a time. Even if all worker threads add an element to standard queue 612 simultaneously, the first buffer will not overflow, but the first threshold will be exceeded for another element, so the processor will add the other element to overflow queue 610.

[0089] For elements added to the standard queue 612, the processor 602 (e.g., its worker thread) can remove the elements from the standard queue 612 for processing. For example, if the standard queue 612 is a session queue 402, 404, 412, 414, the processor 602 can remove the message from the standard queue 612, deserialize the message, and add the deserialized message (or a reference thereto) to the appropriate action queue 406, 408, 416, 418, 506 (an associated overflow queue can also be implemented in a similar manner for the action queue). As another example, if the standard queue 612 is a worker queue 420, 520, the processor 602 can remove the reference to the session queue 402, 404, 412, 414 or the action queue 406, 408, 416, 418, 506 from the worker queue 420, 520 and process one or more messages of the referenced queue. As another example, where the standard queue 612 is an action queue 406, 408, 416, 418, 506, the processor 602 can remove the deserialized message from the action queue 406, 408, 416, 418, 506 and perform processing on the deserialized message, for example to convert the message into a format understood by the client device 202a, 202b, 302a, 302b or the host 210, 310, as the case may be.

[0090] For elements added to the overflow queue 612, the processor 602 may add these messages to the standard queue 612. In an example, the functionality of the processor 602 may be provided by at least one processor. In an example, at an appropriate time, a worker thread (such as the one referenced above) may be called upon to process the message. Figure 4 The worker thread 422 of the pool described above can use the processor 602 to remove messages from the overflow queue 610 and add the messages to the associated standard queue 612. In this way, the processor 602 can remove messages from multiple overflow queues 610 and add these messages to the appropriate associated standard queue 612. For each overflow queue 612, the processor 602 is configured to determine whether the number of elements in the standard queue 612 is less than a second threshold value T2, and based on determining that the number of elements in the standard queue 612 is less than the second threshold value T2, remove the element from the overflow queue 610 and add the removed element to the standard queue 612. The second threshold value T2 can be lower than the first threshold value T1.

[0091] A second threshold T2 can be set for standard queue 612 at a point where the standard queue faces low or no risk of overflow. When the number of elements in standard queue 612 has been sufficiently reduced (e.g., by removing elements from standard queue 612 for processing, as described above), elements are removed from overflow queue 610 (e.g., one by one) and added to standard queue 612. This helps ensure that elements are processed from standard queue 612. In an example, second threshold T2 is set to be equal to or greater than one element. This helps ensure that if there are elements in overflow queue 610, standard queue 612 has at least one element in it. This helps ensure that the use of overflow queue 610 adds minimal delay to the processing of elements from standard queue 612. In an example, first threshold T1 can be between 80% and 98% of the capacity of standard queue 612. In an example, second threshold T2 can be between 60% and 90% of the capacity of standard queue 612. In a specific example, the capacity of the session queues 402, 404, 412, 414 may be 1024, the first threshold T1 may be 1000 (i.e., 98% of capacity), and the second threshold T2 may be 900 (i.e., 88% of capacity). In another specific example, the capacity of the action queues 406, 408, 416, 418, 506 may be 64, the first threshold T1 may be 54 (i.e., 84% of capacity), and the second threshold T2 may be 44 (i.e., 69% of capacity). Other examples may be used.

[0092] Once an element is removed from the overflow queue 610 and added to the associated standard queue 612, the element may be processed from the standard queue 612, for example, in the manner described above.

[0093] In an example, processor 602 is configured to add the element obtained to overflow queue 610 based on the number of elements in overflow queue 610 being greater than zero. Processor 602 can be configured to add the element obtained to standard queue 612 based on the number of elements in overflow queue 610 being zero. This can help ensure that elements are processed in the order in which they are obtained. For example, if there is an element waiting in overflow queue 610, processor 602 may not add the newly obtained element to standard queue 612 because this will make the newly obtained element out of order with the element already in overflow queue 610. On the contrary, processor 602 can add the newly obtained element to overflow queue 610.

[0094] refer to Figure 7, shows a block diagram illustrating a determination that the processor 602 may make when a message is received for a particular standard queue 612, according to an example. At block 702, the processor 602 determines whether the number of elements in the standard queue 612 is greater than a first threshold value T1. If the number of elements in the standard queue 612 is greater than the first threshold value T1, then, according to block 708, the processor 602 adds the element to the overflow queue 610 associated with the standard queue 612. If the number of elements in the standard queue 612 is not greater than the first threshold value T1, then, according to block 704, the processor 602 determines whether the number of elements in the overflow queue 610 is greater than zero. If the number of elements in the overflow queue 610 is greater than zero, then the processor 602 adds the element to the overflow queue according to block 708. If the number of elements in the overflow queue 610 is zero, then the processor adds the element to the standard queue 706 according to block 706. As described above, this means that overflow of the standard queue 612 is avoided, eliminating the risk of blocking or deadlock associated with standard queues, and processing elements in the order in which they were obtained.

[0095] refer to Figure 8 , shows a block diagram of determinations that the processor 602 may make regarding elements in an overflow queue 610, according to an example. For a particular overflow queue 610 that includes an element, at block 802, the processor 602 (and specifically its worker thread) determines whether the number of elements in the associated standard queue 612 is less than a second threshold value T2. If the number of elements in the associated standard queue 612 is less than the second threshold value T2, the processor 602 removes the element from the overflow queue 610 and adds the removed element to the associated standard queue 610, according to block 806. If the number of elements in the associated standard queue 612 is not less than the second threshold value T2, the processor 602 does not add the element from the overflow queue 610 to the standard queue 612, but instead returns to block 802, where it again determines whether the number of elements in the associated standard queue 612 is less than the second threshold value T2.

[0096] Reference again Figure 6 In an example, the determination by the processor 602 may be made with reference to an overflow flag 614. The overflow flag 614 is stored in the memory 604. The overflow flag 614 may be set (e.g., set to one of true and false) or reset (e.g., set to the other of true and false). The overflow flag 614 is associated with the standard queue 612 and the overflow queue 610. In the case where there are multiple standard queues 612 and associated overflow queues 610 ( Figure 6In an example (not shown), each standard queue 612 and associated overflow queue 610 may have a corresponding associated overflow flag 614. The overflow flag 614 may be used by the processor 602 to effectively determine whether to add an obtained message to the standard queue 612 or the overflow queue 610.

[0097] Specifically, in this example, when the number of elements in the standard queue 612 is greater than a first threshold value T1, the processor 602 (particularly its worker thread) may set an overflow flag 614. By determining that the overflow flag 614 is set, the processor 602 can determine that the number of elements in the standard queue 612 is greater than the first threshold value T1. This provides an efficient way to determine that the number of elements in the standard queue 612 is greater than the first threshold value T1. If the processor 602 detects the overflow flag 614 and it is set, the processor 602 can directly add the received message to the overflow queue 612. This can be computationally more efficient than, for example, obtaining a count of the number of messages in the standard queue 610 for each message and comparing it to the first threshold value T1. On the other hand, when the overflow queue 610 is determined to be empty (i.e., when the number of elements in the overflow queue 610 is zero), the processor 602 may reset the overflow flag. By determining that the overflow flag 614 is reset, the processor 602 can determine that the overflow queue 610 is empty (i.e., the number of elements in the overflow queue 610 is zero) and that the number of elements in the standard queue 612 is less than the first threshold value T1. This can provide an efficient way for the processor 602 to determine that the number of messages in the standard queue 612 is less than the first threshold T1 and that the overflow queue 612 is empty (i.e., the number of elements in the overflow queue 610 is zero). If the processor 602 detects the overflow flag 614 and it is reset, the processor 602 can add the obtained element to the standard queue 612. This can be more computationally efficient than, for example, making a corresponding determination individually and / or explicitly for each element.

[0098] As an example, a specific overflow flag 614 is associated with a specific standard queue 612 and overflow queue 610. The processor 602 (specifically, its worker thread) can obtain an element and check the overflow flag 614 to determine whether it is set. If the overflow flag 614 is set (e.g., set to true), the processor 602 can add the element to the overflow queue 610. If the overflow flag 614 is reset (e.g., set to false), the processor 602 can check a counter indicating the number of elements currently in the standard queue 612 and compare this number with a first threshold value T1. If the processor 602 determines that the number of elements is not greater than the first threshold value T1, the processor 602 can add the element to the standard queue 612 and increment the counter by 1. If the processor 602 determines that the number of elements is equal to the first threshold value T1, the processor 602 can add the element to the standard queue 612, increment the counter by 1, and set the overflow flag 614. In this case, if the overflow flag 614 is set, the processor 602 can further add the obtained element directly to the overflow queue 612. This process can continue until the overflow flag 614 is reset (e.g., set to false) again. Specifically, as described above, the processor 602 (particularly its worker thread) waits until the number of elements in the standard queue 612 falls below the second threshold T2, at which point the processor 602 removes elements from the overflow queue 610 and adds the removed elements to the standard queue 612. This process continues until the processor 602 removes the last element from the overflow queue 610. For example, the processor 602 may increment a counter for the overflow queue 610 by one each time an element is added to the overflow queue 610, and may decrement the counter by one each time a message is removed from the overflow queue 610. When the processor 602 determines that the number of elements in the overflow queue 610 is zero, the processor 602 resets the overflow flag 614 (e.g., sets the overflow flag 614 to false). The processor 602 (particularly its worker thread) may then further add the obtained elements to the standard queue 612 until the number of elements in the standard queue 612 again exceeds the first threshold T1, at which point the overflow flag 614 is reset again.

[0099] In some cases, there's a risk that between a worker thread checking overflow flag 614 and adding an element to the queue, another worker thread might set or reset the flag. For example, between a worker thread of processor 602 checking overflow flag 614, determining it's set, and adding an element to overflow queue 610, another worker thread might reset the overflow flag. This can create a risk of sending elements to the incorrect queue. In some examples, a lock can be used. For example, processor 602 (particularly its worker thread) can check overflow flag 614, and if overflow flag 614 is set (e.g., set to true), processor 602 can lock overflow flag 614 and check the locked overflow flag 614. If the locked overflow flag 614 is set (e.g., set to true), processor 602 can add the obtained element to overflow queue 610. If the locked overflow flag 614 is reset (e.g., set to false), the processor can add the obtained element to standard queue 612. If the overflow flag 614 is reset (e.g., set to false), the processor 602 may not lock the overflow flag 614, but instead directly add the obtained element to the standard queue. Thus, in "normal" operation (i.e., where elements are directly added to the standard queue 610), no locking is used, which can allow checking the overflow flag 614 in "normal" operation to be computationally cheap. Since "normal" operation can account for a large portion of the overall operation of the processor 602, this can in turn allow the overall operation of the processor to be relatively computationally cheap while still ensuring that the correct overflow flag 614 value is used.

[0100] VII. Variants

[0101] Reference above Figures 1 to 6 The described system 600, its components, and the operation of the components relate to illustrative examples. In other examples, variations can be made to the above examples.

[0102] For example, although in the above example, Figure 6 The system 600 is described as implementing Figure 2 and Figure 3 6. The connectors 204, 304 of FIG. 60 are shown in FIG. 60, but it should be understood that this is not necessarily the case, and in other examples, the system 600 can be used in other contexts. For example, the system 600 (such as may be used by a computing device) can be used with any standard queue 612 or more generally a buffer.

[0103] As another example, although reference is made to the standard queue 612 and / or the overflow queue 610 in the above examples, it should be understood that these queues are not necessarily queues and, more generally, may be a first buffer and a second buffer, respectively. A buffer may be a portion of a memory used to hold elements for processing.

[0104] As another example, it should be understood that the elements obtained by the processor 602 and stored in the queues 610, 612 are not necessarily messages, and more generally, can be elements such as data elements. For example, for a specific first buffer, an element can be one of a message, a reference to a message, a reference to a queue of one or more messages, and a reference to a queue of references to one or more messages. In addition, it should be understood that an element does not need to involve a message at all, and in some examples can be any element, such as a data element. For example, the first buffer can store multiple elements and can have a capacity for a specific number of elements. For example, the buffer can have a certain amount of space, and an element can fill a space in the buffer. For example, each element can have a fixed size. Other variations are possible, and it should be understood that in some examples, the system 600 can include more or fewer components than those described in the above examples.

[0105] Therefore, it should be understood that in an example, a system 600 is provided, comprising: a memory 604 configured to provide: a first buffer 612 for storing a plurality of elements for processing; and a second buffer 610 for storing elements for adding to the first buffer 612; and at least one processor 602 configured to: obtain elements to be processed; determine that the number of elements in the first buffer 612 is greater than a first threshold T1; and add the obtained elements to the second buffer 610.

[0106] VIII. Example Method

[0107] refer to Figure 9 , shows a flow chart of a method according to an example. The method can be performed by a server system, for example, according to the above reference Figures 1 to 8 For example, the method can be performed by the connector 204, 304 according to the above reference Figures 1 to 8 In an example, the server system includes at least one processor (e.g., according to the above reference Figures 1 to 8 Any of the example processors 112, 206, 306, 602 described herein) and memory (e.g., as described above with reference to Figures 1 to 814, 208, 308, 604 of any example described). Aspects of the method may be performed by at least one processor. The at least one processor may be configured to perform aspects of the method. The memory may store instructions that, when executed, cause the at least one processor to perform aspects of the method or functions defined by the method. For example, the memory may store an application program including instructions (e.g., Figure 1 application 130).

[0108] The method includes, in step 902, providing a first buffer in a memory for storing a plurality of elements for processing. Figures 1 to 8 In any of the examples described, the memory may be the memory 604 , the first buffer may be the standard queue 612 , and the elements may be messages.

[0109] The method includes, in step 904, providing a second buffer in the memory for storing elements to be added to the first buffer. For example, the second buffer may be based on the above reference Figures 1 to 8 The overflow queue 610 of any example described.

[0110] The method includes: in step 906, at least one processor obtains an element to be processed. For example, at least one processor may include the element to be processed according to the above reference Figures 1 to 8 The processor 602 of any example described.

[0111] The method includes, in step 908, determining by at least one processor that the number of elements in the first buffer is greater than a first threshold. Figures 1 to 8 In any of the described examples, the first threshold may be a first threshold T1.

[0112] The method includes, in step 910, adding, by at least one processor, the obtained elements to the second buffer. That is, in response to determining that the number of elements in the first buffer is greater than a first threshold, adding the obtained elements to the second buffer.

[0113] IX. Sample Electronic Trading System

[0114] Figure 10 A block diagram is shown representing an example electronic trading system 1000 in which certain embodiments may be employed. System 1000 includes a trading device 1010, a gateway 1020, and an exchange 1030. Trading device 1010 communicates with gateway 1020. Gateway 1020 communicates with exchange 1030. As used herein, the phrase "in communication with" encompasses direct communication and / or indirect communication through one or more intermediary components. Trading device 1010, gateway 1020, and / or exchange 1030 may include Figure 1One or more computing devices 100. Figure 10 The exemplary electronic transaction system 1000 depicted in FIG. 1 may communicate with additional components, subsystems, and elements to provide additional functionality and capabilities without departing from the teachings and disclosure provided herein.

[0115] In operation, trading device 1010 may receive market data from exchange 1030 via gateway 1020. Trading device 1010 may send messages to exchange 1030 via gateway 1020. A user may utilize trading device 1010 to monitor market data and / or, based on the market data, decide to send an order message to exchange 1030 to buy or sell one or more tradable objects. Trading device 1010 may use the market data to take trading actions, such as sending an order message to exchange 1030. For example, the trading device may execute an algorithm that uses market data as input and outputs a trading action, such as sending an order message to exchange 1030. The algorithm may or may not require input from the user in order to take a trading action.

[0116] Market data may include data about the market for a tradable asset. For example, market data may include the internal market, market depth, last traded price ("LTP"), last traded quantity ("LTQ"), or a combination thereof. The internal market refers to the highest available bid (best bid) and lowest available ask (best ask or best offer) for a tradable asset in the market at a specific point in time (as the internal market may vary over time). Market depth refers to the quantity available at price levels that include and are away from the internal market. Market depth may have "gaps" because prices are not based on the quantity of orders in the market.

[0117] Price levels associated with internal markets and market depth can be provided as value levels, which can include a price and a derived and / or calculated value representation. For example, a value level can be displayed as the net change relative to the opening price. As another example, a value level can be provided as a value calculated based on prices in two other markets. In another example, a value level can include a consolidated price level.

[0118] A tradable object is anything that can be traded. For example, a certain quantity of a tradable object can be bought or sold at a specific price. Tradable objects may include, for example, financial instruments, stocks, options, bonds, futures contracts, currencies, warrants, fund derivatives, securities, commodities, swaps, interest rate products, index-based products, trading events, goods, or any combination thereof. Tradable objects may include products listed and / or managed by an exchange, user-defined products, a combination of real or synthetic products, or any combination thereof. Synthetic tradable objects may correspond to and / or be similar to real tradable objects.

[0119] An order message is a message that includes a trade order. A trade order can be, for example, a command to place an order to buy or sell a tradable object; a command to initiate a management order based on a defined trading strategy; a command to change, modify, or cancel an order; an instruction to an electronic exchange related to an order; or a combination thereof.

[0120] Trading device 1010 may include one or more electronic computing platforms. For example, trading device 1010 may include a desktop computer, a handheld device, a laptop computer, a server, a portable computing device, a trading terminal, an embedded trading system, a workstation, an algorithmic trading system such as a "black box" or "gray box" system, a computer cluster, or a combination thereof. As another example, trading device 1010 may include a single-core or multi-core processor in communication with a memory or other storage medium configured to accessibly store one or more computer programs, applications, libraries, computer-readable instructions, etc. for execution by the processor.

[0121] For example, trading device 1010 may comprise a computing device, such as a personal computer or mobile device, in communication with one or more servers, where the computing device and the one or more servers are collectively trading device 1010. For example, trading device 1010 may comprise a computing device and one or more servers that together run the TT® platform, an electronic trading platform provided by Trading Technologies International, Inc. ("Trading Technologies") of Chicago, Illinois. For example, one or more servers may run a portion of the TT platform, such as a portion that provides a web server, and the computing device may run another portion of the TT platform, such as a portion that provides user interface functionality within a web browser. The computing device and servers may communicate with each other using, for example, browser session requests and responses or network sockets to implement the TT platform. As another example, trading device 1010 may comprise a computing device, such as a personal computer or mobile device, running an application such as TT® Desktop or TT® Mobile (both electronic trading applications also provided by Trading Technologies). As another example, trading device 1010 may comprise one or more servers running trading tools such as ADL®, AUTOSPREADER®, and / or MD TRADER® (also provided by Trading Technologies).

[0122] The trading device 1010 can be controlled or used by a user. As used herein, the phrase "user" can include, but is not limited to, a human (e.g., a trader), a group of users (e.g., a group of traders), or an electronic trading device (e.g., an algorithmic trading system). One or more users can be involved in the control or other use of the trading device.

[0123] The trading device 1010 can include one or more trading applications. As used herein, a trading application is an application that facilitates or improves electronic trading. A trading application provides one or more electronic trading tools. For example, a trading application stored by the trading device can be executed to arrange and display market data in one or more trading windows. In another example, a trading application can include an automated spread trading application that provides spread trading tools. In yet another example, a trading application can include an algorithmic trading application that automatically processes algorithms and performs certain actions, such as placing orders, modifying existing orders, deleting orders. In yet another example, a trading application can provide one or more trading screens. A trading screen can provide one or more trading tools that allow interaction with one or more markets. For example, a trading tool can allow a user to obtain and view market data, set order entry parameters, submit order messages to an exchange, deploy trading algorithms, and / or monitor positions when implementing various trading strategies. The electronic trading tools provided by a trading application can be available at all times, or can be available only in certain configurations or operating modes of the trading application.

[0124] A trading application can be implemented with computer-readable instructions stored in a computer-readable medium and executable by a processor. The computer-readable medium can include various types of volatile and non-volatile storage media, including, for example, a random access memory, a read-only memory, a programmable read-only memory, an electrically programmable read-only memory, an electrically erasable read-only memory, a flash memory, any combination thereof, or any other tangible data storage device. As used herein, the term non-transitory or tangible computer-readable medium is expressly defined to include any type of computer-readable storage medium and exclude propagating signals.

[0125] One or more components or modules of the transaction application can be loaded into the computer-readable medium of transaction device 1010 from another computer-readable medium. For example, the transaction application (or updates to the transaction application) can be stored by the manufacturer, developer, or publisher on one or more CDs, DVDs, or USB drives, and then loaded onto transaction device 1010 or onto a server from which transaction device 1010 retrieves the transaction application. As another example, transaction device 1010 can receive the transaction application (or updates to the transaction application) from a server, for example, via the Internet or an intranet. Transaction device 1010 can receive the transaction application or updates upon request by transaction device 1010 (e.g., "pull distribution") and / or without request by transaction device 1010 (e.g., "push distribution").

[0126] The trading device 1010 may be adapted to send an order message. For example, the order message may be sent to the exchange 1030 via the gateway 1020. As another example, the trading device 1010 may be adapted to send an order message to a simulated exchange in a simulated environment that does not implement real-world trading.

[0127] An order message can be sent at the user's request. For example, a trader can use trading device 1010 to send an order message or manually enter one or more parameters for a trade order (e.g., order price and / or amount). As another example, an automated trading tool provided by a trading application can calculate one or more parameters for a trade order and automatically send an order message. In some cases, the automated trading tool may prepare an order message for sending but may not actually send the order message without confirmation from the user.

[0128] The order message may be sent in one or more data packets or via a shared memory system. For example, the order message may be sent from the trading device 1010 to the exchange 1030 via the gateway 1020. The trading device 1010 may communicate with the gateway 1020 using a local area network, a wide area network, a multicast network, a wireless network, a virtual private network, an intranet, a cellular network, a peer-to-peer network, a point of presence, a dedicated line, the Internet, a shared memory system, and / or a proprietary network.

[0129] The gateway 1020 may include one or more electronic computing platforms. For example, the gateway 1020 may be implemented as one or more desktop computers, handheld devices, laptop computers, servers, portable computing devices, trading terminals, embedded trading systems, workstations with single-core or multi-core processors, algorithmic trading systems such as "black box" or "grey box" systems, computer clusters, or any combination thereof.

[0130] The gateway 1020 facilitates communication. For example, the gateway 1020 can perform protocol conversion on data communicated between the trading device 1010 and the exchange 1030. The gateway 1020 can process order messages received from the trading device 1010 into a data format understood by the exchange 1030, for example. Similarly, for example, the gateway 1020 can convert market data received from the exchange 1030 in an exchange-specific format into a format understood by the trading device 1010. Figure 11 As described in more detail, in some examples, gateway 1020 may communicate with a cloud service that may support the functionality of gateway 1020 and / or transaction device 1010 .

[0131] Gateway 1020 may include a trading application that facilitates or improves electronic trading, similar to the trading applications discussed above. For example, gateway 1020 may include a trading application that tracks orders from trading device 1010 and updates the status of the orders based on trade confirmations received from exchange 1030. As another example, gateway 1020 may include a trading application that consolidates market data from exchange 1030 and provides it to trading device 1010. In yet another example, gateway 1020 may include a trading application that provides risk processing, calculates implied terms, handles order processing, handles market data processing, or a combination thereof.

[0132] In certain embodiments, gateway 1020 communicates with switch 1030 using a local area network, a wide area network, a multicast network, a wireless network, a virtual private network, an intranet, a cellular network, a peer-to-peer network, a point of presence, a dedicated line, the Internet, a shared memory system, and / or a proprietary network.

[0133] Exchange 1030 may be owned, operated, controlled, or used by an exchange entity. Example exchange entities include CME Group, CBOE, Intercontinental Exchange, and Singapore Exchange. Exchange 1030 may be an electronic exchange that includes an electronic matching system, such as a computer, server, or other computing device, adapted to allow tradable items (e.g., offered for trading by the exchange) to be bought and sold. Exchange 1030 may include separate entities, for example, some of which list and / or manage tradable items while others receive and match orders. Exchange 1030 may include, for example, an electronic communications network ("ECN").

[0134] Exchange 1030 is adapted to receive order messages and match contra-side trade orders to buy and sell tradable items. Unmatched trade orders may be listed for trading on exchange 1030. Once an order to buy or sell a tradable item is received and confirmed by the exchange, it is considered a valid order until it is fulfilled or canceled. If only a portion of the order is matched, the partially fulfilled order remains a valid order. For example, trade orders may include trade orders received from trading device 1010 or other devices in communication with exchange 1030. For example, exchange 1030 will typically communicate with various other trading devices (which may be similar to trading device 1010) that also provide trade orders to be matched.

[0135] Exchange 1030 is adapted to provide market data. Market data may be provided in one or more messages or data packets, or via a shared memory system. For example, exchange 1030 may publish data feeds to subscribing devices such as trading device 1010 or gateway 1020. The data feeds may include market data.

[0136] System 1000 may include additional, different, or fewer components. For example, system 1000 may include multiple transaction devices, gateways, and / or exchanges. In another example, system 1000 may include other communication devices, such as middleware, firewalls, hubs, switches, routers, servers, switch-specific communication equipment, modems, security managers, and / or encryption / decryption devices.

[0137] In an example, the transaction device 1010 may be configured by the client devices 202a, 202b, 302a, 302b according to the above referenced Figures 1 to 9 For example, the transaction device 1010 may be configured to provide a Figures 1 to 9 The client devices 202a, 202b, 302a, 302b of any of the examples described herein may function the same or similarly. In an example, the exchange 1030 may be provided by the host 210, 310 according to the above reference Figures 1 to 9 For example, the exchange 1030 may be configured to provide Figures 1 to 9 The gateway 1020 may be configured to function as the host 210, 310 of any of the examples described above. Figures 1 to 9 The system 600 and / or connector 204, 304 of any example described herein provides or includes the system 600 and / or connector 204, 304. For example, the gateway 1010 may be configured to provide a communication interface with the system 600 and / or connector 204, 304 according to the above reference. Figures 1 to 9 The system 600 and / or connectors 204 , 304 of any of the examples described may function the same or similarly.

[0138] In the example above, reference Figures 1 to 9 The message mentioned may include a trade order sent from the trading device 1010 via the gateway 1020 to the exchange 1030. For example, the gateway 1020 may receive (outbound) trade orders from the trading device 1010, and the gateway 1020 may add the trade orders to a standard queue or an overflow queue, such as described above with reference to Figures 1 to 9 The gateway 1020 can process the trade order by converting the trade order into a format understood by the exchange 1030.

[0139] In the example above, reference Figures 1 to 9 The mentioned messages may include an execution report sent from the exchange 1030 to the trading device 1010 via the gateway 1020 when a trading order is executed at the exchange 1030. For example, the gateway 1020 may receive an (inbound) execution report from the exchange 1030, and the gateway 1020 may add the execution report to a standard queue or an overflow queue, such as described above with reference to Figures 1 to 9 The gateway 1020 may process the operational report by converting the operational report into a format understood by the transaction device 1010 .

[0140] In some examples, the above reference Figures 1 to 9 The host-performed actions mentioned may include the exchange 1030 performing actions regarding a specific effective order. For example, the trading device 1010 may send a trading order to the exchange 1030 via the gateway 1020. The trading order may include an order ID indicating the specific effective order associated with the trading order. All trading orders from the trading device 1010 involving the same effective order may include the same order ID. The exchange 1030 may execute the received trading order (such as placing an order, changing an order, and / or canceling an order). The exchange 1030 may then send an operation report back to the trading device 1010 via the gateway 1020. For example, the operation report may detail the results of the trading order. Again, each operation report may include an order ID indicating the specific effective order associated with the operation report. In these examples, the above reference Figures 1 to 9 Each action queue 406, 408, 416, 418 mentioned can be a corresponding order queue. That is, each order queue 406, 408, 416, 418 can be associated with a specific active order, and the messages in a particular order queue 406, 408, 416, 418 can all have the same order ID. It is important that the trade orders associated with a particular active order are processed in the order in which they are received by gateway 1020, because if they are received out of order at exchange 1030, exchange 1030 may not be able to execute the trade orders correctly.

[0141] In the example above, refer to Figures 1 to 9 The messages mentioned may include protocol messages between trading device 1010 and exchange 1030. For example, trading device 1010 (or an application running thereon) and exchange 1030 may communicate with each other using Financial Information Exchange (FIX) messages. Some of these messages may relate to establishing or maintaining a FIX connection or session. For example, the messages may include heartbeat messages, which are sent from exchange 1030 to trading device 1010 (and / or from trading device 1010 to exchange 1030) to confirm that trading device 1010 (and / or exchange 1030) is still present on the connection and should therefore maintain the connection.

[0142] X. Specific Example Electronic Trading System

[0143] Figure 11 A block diagram of an example electronic trading system 1100 in which certain embodiments may be employed is shown. The electronic trading system 1100 includes a trading device 1110, a hybrid cloud system 1120, and an exchange 1130. The trading device 1110 may be similar to the one described above. Figure 10 The exchange 1130 may be the same as or similar to the trading device 1010 described above. Figure 10 The hybrid cloud system 1120 or one or more components thereof may be the same or similar to the exchange 1030 described above. Figure 10 One or more of the functions of the gateway 1020 described above. Figure 10 The described functionality of gateway 1020 , or one or more portions of that functionality, may be included within hybrid cloud system 1120 .

[0144] The hybrid cloud system 1120 includes a cloud service 1140 and a data center 1160. Figure 11 In the example shown, cloud service 1140 and its components are separate from data center 1160. However, in other examples (not shown), one or more or all of the components and / or functions of cloud service 1140 may instead be implemented in data center 1160. In such an example, or in other aspects, electronic transaction system 1100 may not include cloud service 1140. In such an example, the above referenced Figure 10 One or more of the functions described for gateway 1020 may be provided solely by data center 1160 or one or more components thereof.

[0145] To provide lower latency for time-sensitive processes, data center 1160 can be co-located with or located near exchange 1130. Thus, time-sensitive functions of hybrid cloud system 1120, or functions that otherwise benefit from the lower latency of exchange 1130, can be performed by data center 1160. Typically, functions of hybrid cloud system 1120 that are not time-sensitive or do not benefit from the lower latency of exchange 1130 can be performed by cloud services 1140. Hybrid cloud system 1120 allows electronic trading system 1100 to be scalable relative to non-time-critical functions while still providing the relatively low latency of exchange 1130.

[0146] exist Figure 11 In the example, trading device 1110 communicates with cloud service 1140 via a first network 1171. For example, first network 1171 may be a wide area network, such as the Internet using a Hypertext Transfer Protocol (HTTP) connection. Trading device 1110 communicates with data center 1160 via a second network 1172. For example, trading device 1110 may communicate with data center 1160 via a virtual private network (VPN) or using a secure web socket or TCP connection. First network 1171 and second network 1172 may be the same network. Data center 1160 communicates with cloud service 1140 via a third network 1173. For example, data center 1160 may communicate with cloud service 1140 via a private network or a virtual private network (VPN) tunnel. Third network 1173 may be the same as first network 1171 and / or second network 1172. Data center 1160 communicates with exchange 1130 via a fourth network 1174. For example, data center 1160 can communicate with exchange 1130 using a local area network, a wide area network, a multicast network, a wireless network, a virtual private network, an intranet, a cellular network, a peer-to-peer network, a point of presence, a dedicated line, the Internet, a shared storage system, and / or a proprietary network. Fourth network 1174 can be the same as first network 1171, second network 1172, and / or third network 1173.

[0147] Cloud service 1140 may be implemented as a virtual private cloud that may be provided by a logically isolated portion of the overall web service cloud. In this example, cloud service 1140 includes a web database 1141 and associated web server 1142, a product database 1143 and associated product data server (PDS) 1144, a user settings database 1145 and associated user settings server 1146, and a merchandising database 1147 and associated merchandising server 1148.

[0148] The transaction device 1110 can communicate with a web server 1142. As one example, the transaction device 1110 can run a web browser, referred to in this disclosure as a browser, which establishes a browsing session with the web server 1142. This can occur after appropriate domain name resolution to an IP address of the cloud service 1140 and / or after appropriate authentication of the transaction device 1110 (or its user) with the cloud service 1140. The browser sends requests to the web server 1142 and the web server 1142 provides responses to the browser, for example using HyperText Transfer Protocol (HTTP) or HyperText Transfer Protocol Secure (HTTPS) protocols. The web server 1142 can provide a user interface to the browser via which a user can interact with the electronic trading platform. The user interface can allow display of market data and / or allow placing of trading orders. As another example, the transaction device 1110 can run an application that communicates with the web server 1142, such as through an application programming interface (API), to allow a user to interact with the electronic trading platform. The application can provide a user interface through which a user can interact with the electronic trading platform.

[0149] The transaction device 1110 can communicate with a PDS 1144. The PDS 1144 interfaces with a product DB 1143. The product DB 1143 stores definitions of instruments and permissions of users with respect to the instruments. In particular, the product DB 1143 stores definitions of tradable objects and permissions of users to place trading orders for the tradable objects. This information can be provided to the transaction device 1110. The user interface of the transaction device 1110 can use this information to determine for a given user of the transaction device 1110 which tradable objects are permitted to have trading orders placed for them.

[0150] The transaction device 1110 can communicate with a user settings server 1146. The user settings server 1146 interfaces with a user settings database 1145 that stores settings, preferences, and other information associated with a user account of a user. This information can be provided to the user settings server 1146 by the transaction device 1110 at the time of user registration or at some time after registration, and the user settings server 1146 can store this information in the user settings database 1145. This information can be provided to the transaction device 1110. The user interface of the transaction device 1110 can use this information to determine which market data to display and in which format to display.

[0151] The trade database 1147 stores information regarding trades executed using the electronic trading system 1100. The trade database 1147 may store all trade orders submitted by users and all corresponding order execution reports provided by the exchange 1130 when executing trade orders. The trade server 1148 may query the trade database 1147 to generate an audit trail 1149 for a given user, for example. This audit trail 1149 may be provided to the trading device 1110 (or another device) to allow for review and / or analysis of the given user's trading activity.

[0152] Data center 1160 includes a multicast bus 1161, a price server 1162, an edge server 1163, a risk server 1164, a ledger uploader server 1165, an order connector 1166, and a strategy engine server 1167. Various components within data center 1160 communicate with each other using multicast bus 1161. This allows for efficient and scalable communication between components within data center 1160. For example, information provided by one component can be received by multiple other components. Transmitting this information over multicast bus 1161, which other components subscribe to, allows the information to be transmitted in a single message, regardless of how many components may receive it.

[0153] Price server 1162 receives market data from exchange 1130. Price server 1162 converts this information into a format and / or syntax associated with (e.g., used by) electronic trading system 1100. Price server 1162 transmits the converted information as one or more multicast messages over multicast bus 1161. Specifically, price server 1162 multicasts this information over first multicast bus A for receipt by price clients. Edge server 1163 and policy engine server 1167 subscribe to first multicast bus A and receive market data from price server 1162. Price server 1162 can communicate with cloud service 1140. For example, price server 1162 can provide information about products or tradable objects to PDS server 1144 for use in defining tradable objects.

[0154] Edge server 1163 communicates with trading device 1110. For example, trading device 1110 can communicate with edge server 1163 via a secure web socket or TCP connection. In some examples, edge server 1163 can be implemented as a server cluster. The number of servers in the cluster can be determined and scaled as needed based on utilization. Edge server 1163 receives market data via a first multicast bus A and routes the market data to trading device 1110. Users of trading device 1110 can decide to place trade orders based on the market data. Edge server 1163 routes the trade orders from trading device 1110 to exchange 1130. Specifically, when edge server 1163 receives an order message from trading device 1110, it multicasts the order message (or at least a portion thereof) onto a second multicast bus B for receipt by order clients. Risk server 1164 subscribes to second multicast bus B and receives command messages from edge server 1163.

[0155] Risk server 1164 is responsible for determining the pre-trade risk of a given trade order contained in a given order message. For example, for a given trade order, risk server 1164 may determine whether the user placing the trade order is permitted to do so. Risk server 1164 may determine whether the user is permitted to trade the quantity of the tradable object specified in the trade order. Risk server 1164 may prevent unauthorized trade orders from being placed. Risk server 1164 receives order messages from edge server 1163 via second multicast bus B and processes the order messages to determine the risk of the trade order in the message. If risk server 1164 determines that the trade order should not be placed (e.g., because the risk associated with the trade order exceeds a threshold), risk server 1164 prevents the trade order from being placed. For example, in this case, risk server 1164 may not transmit the order message to order connector 1166, but instead may transmit a message indicating to the user that the order has not been placed. If risk server 1164 determines that the trade order should be placed (e.g., because the risk associated with the trade order is below a threshold), risk server 1164 forwards the order message to order connector 1166. Specifically, the risk server 1164 multicasts the order message on the second multicast bus B. The order connector 1166 and the ledger uploader 1165 subscribe to the second multicast bus B and receive the order message from the risk server 1164 .

[0156] The ledger uploader server 1165 communicates with the trade database 1147 of the cloud service 1140. The ledger uploader server 1165 receives the order message from the risk server 1164 and transmits the order message to the trade database 1147. The trade database 1147 then stores the order message (or at least a portion of its content) in a ledger stored in the trade database 1147.

[0157] Order connector 1166 communicates with exchange 1130. Order connector 1166 receives order messages from risk server 1164, processes the order messages for transmission to exchange 1130, and sends the processed order messages to exchange 1130. Specifically, processing includes converting the order messages into a data format understood by exchange 1130. If the trade order within the order message is executed by exchange 1130, exchange 1130 sends a corresponding execution report message to order connector 1166. The execution report message includes an execution report detailing the execution of the trade order. Order connector 1166 applies processing to the execution report message. Specifically, this processing includes converting the execution report message into a data format understood by the electronic trading system and trading device 1110. Order connector 1166 multicasts the processed execution report message on the third multicast bus C for reception by the execution report client. Edge server 1163 and ledger uploader 1165 subscribe to third multicast bus C and receive the processed execution report message. Ledger uploader 1165 communicates with trade database 1147 to update the ledger with the execution report message (or at least a portion of its contents). Edge server 1163 forwards the execution report message to trading device 1110. Trading device 1110 may display information based on the execution report message to indicate that the trade order has been executed.

[0158] In some examples, order messages may be submitted by a strategy engine server 1167. For example, strategy engine server 1167 may implement one or more strategy engines using an algorithmic strategy engine and / or an automated spreader strategy engine. Strategy engine 1167 may receive market data from price server 1162 (via a first multicast bus A) and automatically generate order messages based on the market data and appropriately configured algorithms. Strategy engine server 1167 may transmit the order message to order connector 1166 (via risk server 1164 and a second multicast bus B), which processes the order message in the same manner as described above. Similarly, when exchange 1130 executes an order, strategy engine 1167 may receive a corresponding order execution report message from order connector 1166 (via a third multicast bus C). The order message and execution report message may be transmitted to ledger uploader 1165 in a similar manner to that described above, so that ledger uploader 1165 can update the ledger stored in trade database 1147.

[0159] In some examples, the trade orders sent by trading device 1110 may not be submitted by a human. For example, trading device 1110 may be a computing device implementing an algorithmic trading application. In these examples, trading device 1110 may not communicate with web server 1142, PDS 1144, and / or user settings server 1146, and may not utilize a browser or user interface to submit trades. An application running on trading device 1110 may communicate with an adapter associated with edge server 1163. For example, the application and adapter may communicate using Financial Information Exchange (FIX) messages. In these examples, the adapter may be a fixed adapter. The application running on trading device 1110 may receive market data in FIX format (provided by price server 1162 and converted to FIX format by a FIX adapter associated with edge server 1163). The application running on trading device 1110 may generate a trade order based on the received market data and transmit the FIX-formatted order message to the FIX adapter associated with edge server 1163. A FIX adapter associated with edge server 1163 may process order messages received in FIX format into a format understood by components of data center 1160 .

[0160] It should be understood that the electronic trading system 1100 is merely an example and other electronic trading systems may be used. As an example, the electronic trading system 1100 does not necessarily need to include the cloud service 1140. As another example, the data center 1160 may include more than the above referenced Figure 11 More or fewer components may be described. As another example, forms of messaging between components of data center 1160 other than multicast messaging may be used.

[0161] In the example, the order connector 1166 may be referenced above Figures 1 to 9 The system 600 and / or connector 204, 304 of any example described herein provides or includes the system 600 and / or connector 204, 304. For example, the order connector 204, 304 can be configured to provide a connection to the order according to the above reference. Figures 1 to 9 The system 600 and / or connectors 204, 304 of any example described herein may function the same or similarly. Thus, in the example, the system 600 and / or connectors 204, 304 of any example described herein may function the same or similarly. Figures 1 to 9Any of the described example server systems 600 and / or connectors 204, 304 may implement the order connector 1166 of the example electronic trading system 1100 (or the electronic trading platform provided thereby). The trading device 1110 and / or the strategy engine 1167 may be provided by the client devices 202a, 202b, 302a, 302b and / or provide the same or similar functionality to the client devices 202a, 202b, 302a, 302b as described above. Figures 1 to 9 Any example described. Exchange 1130 may be performed according to the above reference Figures 1 to 9 Any of the example hosts 210 , 310 described provide, and / or provide to the hosts 210 , 310 , the same or similar functionality.

[0162] In this example, the order connector 1166 may receive messages from the trading device 1100 (or the strategy engine 1167), such as trade orders or protocol messages. The order connector 1166 may then add the messages to a standard queue or an overflow queue, as described above. In the case of trade order messages, the order connector 1166 may process the messages by converting them into a format understood by the exchange 1130 and then sending the converted messages to the exchange 1130. In the case of protocol messages (such as FIX heartbeat messages), the order connector 1166 may forward the messages to the exchange 1130 as needed. The exchange 1130 may process the messages. For example, in the case of trade order messages, the exchange 1130 may execute the trade orders. The exchange may then send a performance report back to the trading device 1100. In the case of protocol messages, the exchange 1130 may process the protocol messages and send a response protocol message back to the trading device 1100 as needed. The order connector 1166 may receive messages from the exchange 1130 and add them to a standard queue or an overflow queue, as described above. In the case of an operational report message, the order connector 1166 may process the message by converting it into a format understood by the trading device 1100 (or the policy engine 1167) and then sending the converted message to the trading device 1100 (or the policy engine 1167). In the case of a protocol message, the order connector 1166 may forward the message to the trading device 1100 (or the policy engine 1167) as needed.

[0163] As a specific example, trading device 1100 (or policy engine 1167) may send a trade order message related to a specific live order to order connector 1166. Order connector 1166 may add the trade order message for the specific live order to an outbound session queue associated with the session between trading device 1100 and exchange 1130 (or an overflow queue associated with the outbound session queue, as appropriate). Order connector 1116 may then add a reference to the session queue to a worker queue (or an overflow queue associated with the worker queue, as appropriate). Order connector 1166 may then remove the trade order message from the referenced session queue, determine the specific live order to which the trade order message relates (e.g., by referencing an order ID included in the message), and add the trade order message to an outbound order queue associated with the specific live order (or an overflow queue associated with the outbound order queue, as appropriate). Order connector 1166 may then add a reference to the order queue to a worker queue (or an overflow queue associated with the worker queue, as appropriate). The order connector may then remove the trade order message from the referenced order queue and process the trade order message by converting the trade order message into a format understood by the exchange 1130 and sending the converted trade order message to the exchange 1130 .

[0164] Exchange 1130 may process the trade order and, in response, send an operational report related to a specific active order. Order connector 1166 may add the operational report message to the inbound session queue associated with the session between trading device 1100 and exchange 1130 (or, as appropriate, the overflow queue associated with the inbound session queue). Order connector 1116 may then add a reference to the inbound session queue to a worker queue (or, as appropriate, the overflow queue associated with the worker queue). Order connector 1166 may then remove the operational report message from the referenced inbound session queue, determine the specific active order to which the operational report message relates (e.g., by referencing the order ID included in the message), and add the operational report message to the inbound order queue associated with the specific active order (or, as appropriate, the overflow queue associated with the outbound order queue). Order connector 1166 may then add a reference to the inbound order queue to the worker queue (or, as appropriate, the overflow queue associated with the worker queue). The order connector 1166 can then process the run report message from the referenced inbound order queue by converting the run report message into a format understood by the trading device 1100 (or the strategy engine 1167), and send the converted run report to the trading device 1100 (or the strategy engine 1167). The trading device 1100 (or the strategy engine 1167) can then process the run report appropriately.

[0165] Therefore, messages on different sessions (i.e., associated with different trading devices 1100) can be processed simultaneously by order connector 1166. Furthermore, messages related to different active orders (whether within the same session or between sessions) can be processed simultaneously by order connector 1166. This allows order connector 1166 to process messages quickly. Furthermore, because each queue has an associated overflow queue, even if a message burst causes any standard queue to reach or approach capacity, messages (or, more generally, elements) can still be added to the associated overflow queue. This prevents process blocking and / or deadlocks that could affect the processing rate of order connector 1166. This helps order connector 1166 operate at a higher overall speed and / or more consistently. Furthermore, a message burst on a particular session (i.e., for a particular trading device 1100) does not affect the processing rate of messages on different sessions by order connector 1166. Instead, the message burst is added to the specific session's standard queue or overflow queue, as appropriate. This allows order connector 1166 to operate consistently across sessions. Thus, the order connector 1166 can protect the processing of messages on one session (i.e., for a particular trading device 1100 or policy engine 1167) from being disrupted by message bursts on another session (i.e., for another trading device 1100 or policy engine 1167). Improved order connector 1166 operation can thus be provided.

[0166] As used herein, the phrases "configured to" and "adapted to" encompass that an element, structure or apparatus has been modified, arranged, changed or adapted to perform a particular function or for a particular purpose.

[0167] Some of the described figures depict example block diagrams, systems, and / or flow diagrams representing methods that can be used to implement all or part of certain embodiments. For example, one or more of the components, elements, blocks, and / or functions of the example block diagrams, systems, and / or flow diagrams may be implemented, alone or in combination, in hardware, firmware, discrete logic, as a set of computer-readable instructions stored on a tangible computer-readable medium, and / or any combination thereof. For example, the example block diagrams, systems, and / or flow diagrams may be implemented using any combination of an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable logic device (FPLD), discrete logic, hardware, and / or firmware.

[0168] For example, one or more processors, controllers, and / or other processing devices may be used to execute the example block diagrams, systems, and / or flow diagrams. For example, the examples may be implemented using coded instructions (e.g., computer-readable instructions) stored on a tangible computer-readable medium. Tangible computer-readable media may include various types of volatile and non-volatile storage media, including, for example, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), electrically programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), flash memory, hard drives, optical media, magnetic tape, file servers, any other tangible data storage devices, or any combination thereof. Tangible computer-readable media are non-transitory.

[0169] Furthermore, although example block diagrams, systems, and / or flow charts have been described above with reference to the accompanying drawings, other implementations may be employed. For example, the order in which components, elements, blocks, and / or functions are executed may be changed and / or some of the components, elements, blocks, and / or functions described may be changed, eliminated, subdivided, or combined. Furthermore, any or all of the components, elements, blocks, and / or functions may be executed sequentially and / or in parallel, for example, by separate processing threads, processors, devices, discrete logic, and / or circuits.

[0170] Although embodiments have been disclosed, various changes may be made and equivalents may be substituted. In addition, many modifications may be made to adapt to particular circumstances or materials. Therefore, it is intended that the disclosed technology is not limited to the particular embodiments disclosed, but is intended to include all embodiments falling within the scope of the appended claims.

[0171] Terms

[0172] 1. A system comprising: a memory configured to provide: a first buffer for storing a plurality of elements for processing; and a second buffer for storing elements for adding to the first buffer; and at least one processor configured to: obtain an element to be processed; determine that the number of elements in the first buffer is greater than a first threshold; and add the obtained element to the second buffer.

[0173] 2. The system of clause 1, wherein the at least one processor is further configured to: add the obtained element to the first buffer based at least in part on determining that the number of elements in the first buffer is less than the first threshold.

[0174] 3. A system as described in clause 1 or clause 2, wherein the at least one processor is configured to: set an overflow flag when the number of elements in the first buffer is greater than the first threshold; and wherein the at least one processor is configured to determine that the number of elements in the first buffer is greater than the first threshold by determining that the overflow flag is set.

[0175] 4. A system as described in any one of clauses 1 to 3, wherein the at least one processor is further configured to: determine that the number of elements in the first buffer is less than a second threshold, wherein the second threshold is lower than the first threshold; and remove elements from the second buffer and add the removed elements to the first buffer.

[0176] 5. The system of any one of clauses 1 to 4, wherein the first buffer is a first queue for queuing a plurality of elements for processing, and the second buffer is a second queue for queuing elements for addition to the first queue.

[0177] 6. The system of any one of clauses 1 to 5, wherein the at least one processor is further configured to: determine that the number of elements in the second buffer is greater than zero; and add the obtained element to the second buffer.

[0178] 7. Clause 6: The system of any one of clauses 1 to 6, wherein the at least one processor is further configured to: determine that the second buffer is empty and the number of elements in the first buffer is less than a first threshold; and add the obtained element to the first buffer.

[0179] 8. A system as described in clause 7, wherein the at least one processor is configured to: set an overflow flag when the number of elements in the first buffer is greater than the first threshold; and reset the overflow flag when it is determined that the second buffer is empty, wherein the at least one processor is configured to determine that the second buffer is empty and the number of elements in the first buffer is less than the first threshold by determining that the overflow flag is reset.

[0180] 9. The system of any one of clauses 1 to 8, wherein the first buffer is a ring buffer.

[0181] 10. The system of any one of clauses 1 to 9, wherein the second buffer is a queue for queuing elements for addition to the first buffer.

[0182] 11. A system as described in any of clauses 1 to 10, wherein the at least one processor is configured to implement multiple worker threads added to the first buffer, wherein the first threshold is at most the total capacity of the first buffer with elements minus the number of worker threads.

[0183] 12. The system of any one of clauses 1 to 11, wherein the element is a message and the at least one processor is configured to obtain the element by receiving the message from the external entity via the communication interface.

[0184] 13. The system of any one of clauses 1 to 11, wherein the at least one processor is configured to obtain the element by removing the element from a third buffer.

[0185] 14. The system of any one of clauses 1 to 13, wherein the elements are each a reference to a queue of one or more messages.

[0186] 15. A system as described in any of clauses 1 to 14, wherein the element relates to a message, wherein the system includes a server system, the server system includes the at least one processor and the memory, and the server system is configured to facilitate message transmission between a host system and one or more client systems.

[0187] 16. The system of clause 15, wherein the one or more client systems comprise trading devices and the host system comprises an electronic exchange.

[0188] 17. A system as described in clause 15 or clause 16, wherein the server system is configured to establish multiple connections between the host system and a corresponding plurality of client systems, wherein the memory is configured to provide a first buffer and an associated second buffer for each of the multiple connections, and wherein the at least one processor is configured to add elements to the corresponding first buffer or second buffer for at least some of the connections.

[0189] 18. A system as described in any of clauses 15 to 17, wherein the message transmission between the host system and the one or more client systems is bidirectional, wherein the memory is configured to provide a first buffer and an associated second buffer for each of a first message transmission direction and a second message transmission direction, and wherein the at least one processor is configured to add elements to the corresponding first buffer or second buffer for each message transmission direction.

[0190] 19. The system of any one of clauses 15 to 18, wherein the server system comprises an order connector of an electronic trading platform, and the order connector comprises at least one processor and memory.

[0191] 20. A method comprising: providing in a memory: a first buffer for storing a plurality of elements for processing; and a second buffer for storing elements for adding to the first buffer; and by at least one processor: obtaining elements to be processed; determining that the number of elements in the first buffer is greater than a first threshold; and adding the obtained elements to the second buffer.

[0192] 21. The method of clause 20, wherein the method further comprises, by the at least one processor: adding the obtained element to the first buffer based at least in part on determining that the number of elements in the first buffer is less than the first threshold.

[0193] 22. A method as described in clause 20 or clause 21, wherein the method includes, by the at least one processor: setting an overflow flag when the number of elements in the first buffer is greater than the first threshold; and wherein determining that the number of elements in the first buffer is greater than the first threshold is by determining that the overflow flag is set.

[0194] 23. A method as described in any one of clauses 20 to 22, wherein the method further includes, by the at least one processor: determining that the number of elements in the first buffer is less than a second threshold, wherein the second threshold is lower than the first threshold; and removing elements from the second buffer and adding the removed elements to the first buffer.

[0195] 24. The method of any one of clauses 20 to 23, wherein the first buffer is a first queue for queuing a plurality of elements for processing, and the second buffer is a second queue for queuing elements for addition to the first queue.

[0196] 25. The method of any one of clauses 20 to 24, wherein the method further comprises, by the at least one processor: determining that the number of elements in the second buffer is greater than zero; and adding the obtained element to the second buffer.

[0197] 26. The method of any one of clauses 20 to 25, further comprising, by the at least one processor: determining that the second buffer is empty and the number of elements in the first buffer is less than the first threshold; and adding the obtained element to the first buffer.

[0198] 27. A method as described in clause 26, wherein the method further includes, by the at least one processor: setting an overflow flag when the number of elements in the first buffer is greater than the first threshold; and resetting the overflow flag when it is determined that the second buffer is empty, wherein determining that the second buffer is empty and the number of elements in the first buffer is less than the first threshold is performed by determining that the overflow flag is reset.

[0199] 28. The method of any one of clauses 20 to 27, wherein the first buffer is a ring buffer.

[0200] 29. The method of any one of clauses 20 to 28, wherein the second buffer is a queue for queuing elements for addition to the first buffer.

[0201] 30. A method as described in any of clauses 20 to 29, wherein the method includes, by the at least one processor: providing multiple worker threads to add to the first buffer, wherein the first threshold is at most the total capacity of elements that the first buffer has minus the number of worker threads.

[0202] 31. The method of any one of clauses 20 to 30, wherein the element is a message, and wherein obtaining the element is by receiving the message from the external entity via the communication interface.

[0203] 32. The method of any one of clauses 20 to 30, wherein obtaining the element is by removing the element from the third buffer.

[0204] 33. The method of any one of clauses 20 to 32, wherein the elements are each a reference to a queue of one or more messages.

[0205] 34. A method as described in any of clauses 20 to 33, wherein the element relates to a message, wherein the method is performed by a server system including the at least one processor and the memory, and wherein the method includes, by the at least one processor: facilitating message transmission between a host system and one or more client systems.

[0206] 35. The method of clause 34, wherein one or more of the client systems comprises a trading device and the host system comprises an electronic exchange.

[0207] 36. A method as described in clause 34 or clause 35, wherein the method further includes establishing multiple connections between the host system and a corresponding plurality of client systems; providing a first buffer and an associated second buffer in the memory for each of the multiple connections; and, for at least some of the connections, by at least one processor: adding an element to the corresponding first buffer or second buffer for the connection.

[0208] 37. A method as described in any one of clauses 34 to 36, wherein the message transmission between the host system and the one or more client systems is bidirectional, wherein the method further comprises: providing a first buffer and an associated second buffer for each of the first message transmission direction and the second message transmission direction; and adding, by the at least one processor, for each message transmission direction, an element to the corresponding first buffer or second buffer for that direction.

[0209] 38. The method of any one of clauses 34 to 37, wherein the memory and the at least one processor are part of an order connector of an electronic trading platform.

[0210] 39. A tangible computer-readable medium comprising instructions that, when executed, cause a machine to at least: provide in a memory: a first buffer for storing elements for processing; and a second buffer for storing elements for adding to the first buffer; and by at least one processor: obtain an element to be processed; determine that the number of elements in the first buffer is greater than a first threshold; and add the obtained element to the second buffer.

[0211] 40. The tangible computer-readable medium of clause 39, wherein the instructions, when executed, cause the machine: via the at least one processor: to add the obtained element to the first buffer based at least in part on determining that the number of elements in the first buffer is less than the first threshold.

[0212] 41. A tangible computer-readable medium as described in clause 39 or clause 40, wherein the instructions, when executed, cause the machine: by the at least one processor: setting an overflow flag when the number of elements in the first buffer is greater than the first threshold; and wherein determining that the number of elements in the first buffer is greater than the first threshold is by determining that the overflow flag is set.

[0213] 42. A tangible computer-readable medium as described in any one of clauses 39 to 41, wherein the instructions, when executed, cause the machine to: by the at least one processor: determine that the number of elements in the first buffer is less than a second threshold, wherein the second threshold is lower than the first threshold; and remove elements from the second buffer and add the removed elements to the first buffer.

[0214] 43. The tangible computer-readable medium of any one of clauses 39 to 42, wherein the first buffer is a first queue for queuing a plurality of elements for processing, and the second buffer is a second queue for queuing elements for addition to the first queue.

[0215] 44. The tangible computer-readable medium of any one of clauses 39 to 43, wherein the instructions, when executed, cause the machine to: via the at least one processor: determine that the number of elements in the second buffer is greater than zero; and add the obtained element to the second buffer.

[0216] 45. A tangible computer-readable medium as described in any one of clauses 39 to 44, wherein the instructions, when executed, cause the machine to: by the at least one processor: determine that the second buffer is empty and the number of elements in the first buffer is less than the first threshold; and add the obtained elements to the first buffer.

[0217] 46. ​​A tangible computer-readable medium as described in claim 45, wherein the instructions, when executed, cause the machine: by the at least one processor: setting an overflow flag when the number of elements in the first buffer is greater than the first threshold; and resetting the overflow flag when the second buffer is determined to be empty, wherein determining that the second buffer is empty and the number of elements in the first buffer is less than the first threshold is achieved by determining that the overflow flag is reset.

[0218] 47. The tangible computer-readable medium of any one of clauses 39 to 46, wherein the first buffer is a ring buffer.

[0219] 48. The tangible computer-readable medium of any one of clauses 39 to 47, wherein the second buffer is a queue for queuing elements for addition to the first buffer.

[0220] 49. A tangible computer-readable medium as described in any one of clauses 39 to 48, wherein the instructions, when executed, cause the machine to: via the at least one processor: provide a plurality of worker threads added to the first buffer, wherein the first threshold is at most the total capacity of the elements that the first buffer has minus the number of worker threads.

[0221] 50. The tangible computer-readable medium of any one of clauses 39 to 49, wherein the element is a message, and wherein the element is obtained by receiving a message from an external entity via a communication interface.

[0222] 51. The tangible computer-readable medium of any one of clauses 39 to 49, wherein the element is obtained by removing the element from a third buffer.

[0223] 52. The tangible computer-readable medium of any one of clauses 39 to 51, wherein the elements are each a reference to a queue of one or more messages.

[0224] 53. A tangible computer-readable medium as described in any one of clauses 39 to 52, wherein the element relates to a message, wherein the instructions, when executed, cause the machine to: implement a server system including the memory and the at least one processor; and, through the at least one processor, facilitate message transmission between a host system and one or more client systems.

[0225] 54. The tangible computer-readable medium of clause 53, wherein one or more of the client systems comprises a trading device and the host system comprises an electronic exchange.

[0226] 55. A tangible computer-readable medium as described in clause 53 or clause 54, wherein the instructions, when executed, cause the machine to: establish a plurality of connections between the host system and a corresponding plurality of client systems; provide a first buffer and an associated second buffer in the memory for each of the plurality of connections; and, for at least some of the connections, by at least one processor: add an element to the corresponding first buffer or second buffer for the connection.

[0227] 56. A tangible computer-readable medium as described in any of clauses 53 to 55, wherein the message transfer between the host system and the one or more client systems is bidirectional, wherein the instructions, when executed, cause the machine to: provide a first buffer and an associated second buffer for each of a first message transfer direction and a second message transfer direction; and, by the at least one processor, add elements to the corresponding first buffer or second buffer for each message transfer direction.

[0228] 57. The tangible computer-readable medium of any one of clauses 53 to 56, wherein the instructions, when executed, cause the machine to: implement an order connector for an electronic trading platform, the order connector comprising the at least one processor and the memory.

Claims

1. A system comprising: Memory, configured to provide: a first buffer for storing a plurality of elements for processing; and a second buffer for storing elements to be added to the first buffer; and At least one processor configured to: Get the element to be processed; determining that the number of elements in the first buffer is greater than a first threshold; and The obtained element is added to the second buffer.

2. The system according to claim 1, wherein: The at least one processor is further configured to: Based at least in part on determining that the number of elements in the first buffer is less than the first threshold, the obtained element is added to the first buffer.

3. The system according to claim 1, wherein: The at least one processor is configured to: When the number of elements in the first buffer is greater than the first threshold, setting an overflow flag; and The at least one processor is configured to determine that the number of elements in the first buffer is greater than the first threshold by determining that the overflow flag is set.

4. The system according to claim 1, wherein: The at least one processor is further configured to: determining that the number of elements in the first buffer is less than a second threshold, wherein the second threshold is lower than the first threshold; and The elements are removed from the second buffer and added to the first buffer.

5. The system according to claim 1, wherein: The first buffer is a first queue for queuing a plurality of elements for processing, and the second buffer is a second queue for queuing elements for addition to the first queue.

6. The system according to claim 1, wherein: The at least one processor is further configured to: determining that the number of elements in the second buffer is greater than zero; and The obtained element is added to the second buffer.

7. The system according to claim 1, wherein: The at least one processor is further configured to: determining that the second buffer is empty and the number of elements in the first buffer is less than the first threshold; and The obtained element is added to the first buffer.

8. The system according to claim 7, wherein: The at least one processor is configured to: When the number of elements in the first buffer is greater than the first threshold, setting an overflow flag; and When it is determined that the second buffer is empty, resetting the overflow flag, The at least one processor is configured to determine that the second buffer is empty by determining that the overflow flag is reset and the number of elements in the first buffer is less than the first threshold.

9. The system of claim 1, wherein the first buffer is a ring buffer.

10. The system according to claim 1, wherein: The second buffer is a queue for queuing elements for addition to the first buffer.

11. The system according to claim 1, wherein: The at least one processor is configured to implement a plurality of worker threads added to the first buffer, wherein the first threshold is at most a total of a capacity of elements held by the first buffer minus a number of worker threads.

12. The system according to claim 1, wherein: The element is a message, and the at least one processor is configured to obtain the element by receiving a message from an external entity via a communication interface.

13. The system of claim 1, wherein the at least one processor is configured to obtain the element by removing the element from a third buffer.

14. The system according to claim 1, wherein: The elements are each a reference to a queue of one or more messages.

15. The system of claim 1, wherein the element relates to a message, wherein the system comprises a server system comprising the at least one processor and the memory, and wherein the server system is configured to facilitate message transfer between a host system and one or more client systems.

16. The system of claim 15, wherein one or more of the client systems comprises a trading device and the host system comprises an electronic exchange.

17. The system of claim 15 , wherein the server system is configured to establish a plurality of connections between the host system and a corresponding plurality of client systems, wherein the memory is configured to provide, for each of the plurality of connections, a first buffer and an associated second buffer, and wherein the at least one processor is configured to, for at least some of the connections, add elements to the corresponding first buffer or second buffer for the connection.

18. A system according to claim 15, wherein the message transmission between the host system and the one or more client systems is bidirectional, wherein the memory is configured to provide a first buffer and an associated second buffer for each of a first message transmission direction and a second message transmission direction, and wherein the at least one processor is configured to, for each message transmission direction, add an element to the corresponding first buffer or second buffer for that direction.

19. The system of claim 15, wherein the server system comprises an order connector of an electronic trading platform, and the order connector comprises the at least one processor and the memory.

20. A method comprising: Provided in memory: a first buffer for storing a plurality of elements for processing; and a second buffer for storing elements to be added to the first buffer; and By at least one processor: Get the element to be processed; determining that the number of elements in the first buffer is greater than a first threshold; and The obtained element is added to the second buffer.

21. A tangible computer-readable medium comprising instructions that, when executed, cause a machine to at least: Provided in memory: a first buffer for storing elements for processing; and a second buffer for storing elements to be added to the first buffer; and By at least one processor: Get the element to be processed; determining that the number of elements in the first buffer is greater than a first threshold; and The obtained element is added to the second buffer.

Images