Data transmission method, system and device and storage medium

Abstract

The embodiment of the invention provides a data transmission method, system and device and a storage medium. The method includes reading, at a first service device of the plurality of service devices, first data from a message queue in response to a data acquisition request from a data requester, the message queue being associated to a database and coupled to the plurality of service devices, the message queue storing target data read from the database, the first data read by the first service device is a part of the stored target data; in response to the read first data, writing the first data into an annular buffer area of the first service equipment through a writing pointer of the first service equipment; in response to a sending request of sending data to a data requester initiated by the first service equipment, reading first data from an annular buffer area of the first service equipment through a read pointer of the first service equipment; and sending the first data to the data requester. Therefore, the data transmission efficiency can be improved.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein relate generally to the field of computers, and in particular to methods, systems, apparatus and computer-readable storage media for data transmission. Background Technology

[0002] With the continuous popularization of information technology, data transmission between different devices is becoming increasingly frequent. For example, in distributed internet systems, data transmission is required to achieve data sharing and communication within the system. Traditional data transmission technologies still have drawbacks such as high resource consumption and difficulty in debugging, which urgently need to be addressed. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for data transmission is provided. The method includes: at a first service device among a plurality of service devices, in response to a data retrieval request from a data requester, reading first data from a message queue, the message queue being associated with a database and coupled to the plurality of service devices, the message queue storing target data read from the database, and the first data read by the first service device being a portion of the stored target data; in response to reading the first data, writing the first data to a circular buffer of the first service device via a write pointer of the first service device; in response to the first service device initiating a send request to send data to the data requester, reading the first data from the circular buffer of the first service device via a read pointer of the first service device; and sending the first data to the data requester.

[0004] In a second aspect of this disclosure, a data transmission system is provided. The system includes: a message queue coupled to a database, adapted to acquire and store target data in the database related to a data acquisition request from a data requester; and a plurality of service devices coupled to the message queue, adapted to acquire at least partially the data from the message queue and send the data to the data requester.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic flowchart illustrating the use of thread pool technology for data transmission in traditional techniques is shown.

[0010] Figure 2 A schematic diagram of a data transmission system according to some embodiments of the present disclosure is shown;

[0011] Figure 3A and Figure 3B Schematic diagrams of writing and reading data in the circular buffer of a service device according to some embodiments of the present disclosure are shown respectively;

[0012] Figure 3C and Figure 3D Schematic diagrams showing a full or empty circular buffer according to some embodiments of the present disclosure are shown respectively;

[0013] Figure 4 A flowchart of a data transmission method according to some embodiments of the present disclosure is shown; and

[0014] Figure 5 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0018] As briefly mentioned earlier, traditional technologies often use thread pools to transfer data when handling concurrent tasks. Figure 1 This diagram illustrates a schematic flowchart of the traditional technique of using thread pools for data transfer. Figure 1 As shown in box 110, after receiving a data read request from the data requester, the server device reads data from the database. In box 120, the server device performs data validation, and if the validation passes, sends the data to the thread pool.

[0019] In box 130, after receiving a data transfer task, the thread pool system checks if the core thread pool is full. In box 131, if the core thread pool is not full, a thread is created. In box 140, if the core thread pool is full, the thread pool system places the current data transfer task into a waiting queue. In box 141, if the waiting queue is not full, the thread pool system places the task into the waiting queue so that the core thread pool can process it when it becomes idle. In box 150, if the waiting queue is full, the thread pool system checks if other thread pools besides the core thread pool are full. In box 151, if other thread pools are not full, a thread is created in one of those thread pools. In box 160, if other thread pools are full, the task requiring data transfer is processed according to a predetermined strategy. In box 170, after a thread is created for a task, the corresponding thread pool system can transfer data by calling a Remote Procedure Call (RPC) interface. This is how data transfer is performed between different systems.

[0020] However, using thread pools for data transfer also has some drawbacks. For example, a thread pool needs to maintain a certain number of threads, which consume system memory and CPU resources. If the thread pool is too large, it will lead to resource waste. If the thread pool is too small, it may affect the timely execution of tasks. In addition, in a multithreaded environment, if concurrent access to shared resources is not handled properly when using thread pools for data transfer, it may lead to thread safety issues such as deadlocks and resource contention. Furthermore, since the threads in the thread pool are reused, when an error or exception occurs, multiple tasks may share the same thread, making exception stack tracing complex and difficult to locate the specific task.

[0021] This disclosure provides methods, systems, devices, and storage media for data transmission to solve, or at least partially solve, the aforementioned problems and other potential problems existing in conventional solutions. According to various embodiments of this disclosure, upon receiving a data retrieval request from a data requester, a message queue retrieves and stores target data related to the data request from a database. Subsequently, multiple service devices coupled to the message queue retrieve data from the message queue respectively. For example, a first service device of the multiple service devices retrieves first data from the message queue, and a second service device of the multiple service devices retrieves second data from the message queue, where the first and second data are respectively parts of the target data. The multiple service devices each send the data retrieved from the message queue to the data requester through their respective circular buffers. In this way, data transmission from the database to the data requester is realized. Thus, by having a single machine (i.e., the message queue) produce the data to be transmitted, and multiple machines (i.e., multiple service devices) consume the data to be transmitted, a good decoupling and peak-shaving effect is achieved, thereby improving data transmission efficiency and performance, and reducing congestion and losses during data transmission. Furthermore, by using multiple machines for data transmission, the problem of slow data transmission by a single machine is solved. The circular buffer setup also reduces the performance overhead of locking and unlocking.

[0022] Figure 2 A schematic diagram of a data transmission system 200 according to some embodiments of the present disclosure is shown. Figure 2 As shown, the data transmission system 200 generally includes a message queue 210 and multiple service devices 220-1, 220-2, ..., 220-N (collectively or individually referred to as service devices 220) coupled to the message queue 210. Multiple service devices 220 can be associated with a database 230 and can serve data requesters 240 of the database 230. A data requester 240 refers to an entity capable of performing data operations and / or data write operations on the database 230. Although a single data requester 240 is shown in the figure, data in the database 230 can be read and written by multiple different data requesters 240.

[0023] Message queue 210 (also called a message queue system) retrieves target data from database 230 based on data retrieval requests from data requester 240, and caches this target data in message queue 210 for retrieval by multiple service devices 220. Multiple service devices 220-1, 220-2, ..., 220-N are configured as corresponding circular buffers 300-1, 300-2, ..., 300-N (collectively or individually referred to as circular buffer 300). Each service device 220 retrieves corresponding data from message queue 210 and stores it in its corresponding circular buffer 300. Service device 220 then sends the data in the circular buffer 300 to data requester 240 based on their data retrieval requests. In this way, through message queue 210 and multiple service devices 220, a single machine (i.e., message queue 210) produces the data that needs to be transmitted, and multiple machines (i.e., multiple service devices 220) consume the data that needs to be transmitted, which improves the data transmission efficiency, reduces the processing burden of a single service device 220, and also plays a role in decoupling and peak shaving for data transmission.

[0024] Figure 3A and Figure 3B Schematic diagrams illustrating the writing and reading of data in the circular buffer 300 of the service device 220 according to some embodiments of the present disclosure are shown. Figure 3A and Figure 3B As shown, each of the multiple service devices 220 is provided with a circular buffer 300, which is defined by multiple buffer blocks 310-1, 310-2, ..., 310-M (collectively referred to as buffer blocks 310 or individually). Each service device 220 is also configured with a write pointer 320 for writing data to the circular buffer 300 and a read pointer 330 for reading data from the circular buffer 300. The write pointer 320 and the read pointer 330 can move along a predetermined direction between the multiple buffer blocks 310 and write or read data from the buffer blocks 310.

[0025] In some embodiments, the circular buffer 300 can be determined by a segment of memory on the service device 220. For example, a segment of memory can be defined by a pair of pointers arranged on the memory so that the segment of memory logically forms a ring, thus forming the circular buffer 300. Specifically, the actual start position in the memory can be determined by a head pointer, and the actual end position in the memory can be determined by a tail pointer. This allows the write pointer 320 and the read pointer 330 to move in a predetermined direction between the head pointer and the tail pointer, and to jump back after the write pointer 320 and the read pointer 330 reach the tail pointer, so that the write pointer 320 and the read pointer 330 can continuously cyclically move in the predetermined direction within the circular buffer 300. In some embodiments, the predetermined direction refers to the direction from the head pointer to the tail pointer in the memory. Figures 3A to 3D In the illustration, the predetermined direction can be either clockwise or counterclockwise.

[0026] In some embodiments, the service device 220 can also perform initialization operations on the circular buffer 300. For example, the initialization operation can set the actual positions that the head pointer and tail pointer point to in memory, thereby changing the size of the circular buffer. Through the initialization operation, the service device 220 can define all buffer blocks within the circular buffer 300 as empty buffer blocks and define the initial positions of the write pointer 320 and the read pointer 330. For example, the service device 220 can set the initial positions of the write pointer 320 and the read pointer 330 to buffer block 310-1. Although after the initialization operation, the write pointer 320 and the read pointer 330 can point to the same buffer block 310 (e.g., buffer block 310-1), since the write speed of the write pointer 320 and the read speed of the read pointer 330 are different, after the circular buffer has been running for a period of time, the write pointer 320 and the read pointer 330 can point to different buffer blocks 310 respectively. Figure 3A In this context, the write pointer can point to buffer block 310-1 or buffer block 310-2, and the read pointer 330 can point to buffer block 310-M. For example, in... Figure 3B In the buffer, the write pointer 320 can point to buffer block 310-2, and the read pointer 330 can point to 310-M or buffer block 310-1.

[0027] Therefore, when the service device 220 receives a data retrieval request from the data requester 240, the write pointer 320 can move along a predetermined direction between buffer blocks 310-1, 310-2, ... 310-M, and write the data retrieved from the message queue 210 into the empty buffer blocks in these buffer blocks. The read pointer can also move between buffer blocks 310-1, 310-2, ... 310-M according to the data retrieval request and read data from the buffer blocks 310-1, 310-2, ... 310-M where data has been written, and send it to the data requester 240.

[0028] In some embodiments, based on a data retrieval request from a data requester 240, a given service device (e.g., a first service device) among a plurality of service devices retrieves data (e.g., first data) from a message queue 210. The first service device then continuously writes the first data into at least one empty buffer block of the circular buffer 300 via a write pointer 320. Figure 3A As shown, write pointer 320 writes at least a portion of the first data (e.g., data a) into one of the empty buffer blocks (e.g., buffer block 310-1) of the circular buffer 300, so that the empty buffer block is marked as a full buffer block storing data. Subsequently, write pointer 320 moves in a predetermined direction (e.g., counterclockwise) to the next empty buffer block (e.g., buffer block 310-2) and continues to write at least a portion of the remaining data in the first data (e.g., data b), and then the write pointer continues to move in the predetermined direction to the next empty buffer block.

[0029] In this manner, as the write pointer 320 moves continuously between multiple empty buffer blocks within the circular buffer 300, the write pointer 320 writes the first data into multiple buffer blocks 310 of the circular buffer 300. In some embodiments, the first service device may also write some first data with a small overall data volume into a buffer block 310 via the write pointer 320, so that it can be read by the read pointer 330 when needed.

[0030] In some embodiments, the first service device may read first data from the circular buffer 300 via the read pointer 330 based on a data acquisition request from the data requester 240, and send the first data to the data requester 240. For example... Figure 3B As shown, read pointer 330 reads data a from the first full buffer block (e.g., buffer block 310-1) and sends the read data a to the data requester 240. Simultaneously, the full buffer block from which read pointer 330 reads data is marked as an empty buffer block. Read pointer 330 then moves in a predetermined direction to the next full buffer block (e.g., buffer block 310-2) and reads data b from that full buffer block. In this way, read pointer 330 can continuously read data from buffer block 310.

[0031] The circular buffer 300 allows writes and reads to be processed asynchronously, enabling the service device 220 to independently retrieve data from the message queue 210 (i.e., produce data) and independently send data to the data requester 240 (i.e., consume data), thus improving concurrency and system responsiveness. In some embodiments, the write pointer 320 of the service device 220 can continuously write data to the circular buffer 300, while the read pointer 330 can continuously read data from the circular buffer 300. This locality of access pattern is beneficial to the cache hit rate and improves the efficiency of data access.

[0032] In some embodiments, the size of the circular buffer 300 can be adjusted according to actual usage. For example, the length of memory defined by the head and tail pointers can be changed by adjusting their actual positions in memory, thereby changing the size of the circular buffer 300. Adjusting the size of the circular buffer 300 allows for optimization based on actual needs. A larger circular buffer 300 can reduce frequent communication between the sender and receiver, increasing throughput; a smaller circular buffer 300 can reduce latency and ensure real-time requirements.

[0033] During the writing of data to the circular buffer 300 via the write pointer 320 and the reading of data from the circular buffer 300 via the read pointer 330, the service device 220 needs to determine whether the circular buffer 300 is full or empty (i.e., whether all buffer blocks 310 within the circular buffer 300 are full or empty) to avoid data loss or overwriting. For example, the position of the read pointer 330 and the write pointer 320 within the circular buffer 300 can be used to determine whether the circular buffer 300 is full or empty. Specifically, if the write pointer 320 and the read pointer 330 point to the same position within the circular buffer 300, then the circular buffer 300 is empty. If the position pointed to by the read pointer 330 is the next position adjacent to the position pointed to by the write pointer 320 (i.e., the next position along a predetermined direction), then the circular buffer 300 is full.

[0034] It should be understood that the above-described strategy for determining whether the circular buffer 300 is full or empty is merely illustrative. In fact, any other appropriate strategy can be used to determine whether the circular buffer 300 is full or empty, which will not be elaborated upon here.

[0035] Figure 3C and Figure 3D Schematic diagrams are shown for a full or empty annular buffer 300 according to embodiments of the present disclosure. Figure 3C and Figure 3DAs shown, if the service device 220 detects that the circular buffer 300 is full, the service device 220 stops retrieving data (e.g., data i) from the message queue 210, thereby stopping the writing of data to the circular buffer 300. Data retrieval from the message queue 210 resumes only after the service device 220 detects that more than a predetermined threshold of empty buffer blocks have been released from the circular buffer 300. If the service device 220 detects that the circular buffer 300 is empty, the read pointer 330 of the service device 220 loops and waits until new data is received in the circular buffer 300.

[0036] In some embodiments, data read by read pointer 330 can be sent to data requester 240 via a remote procedure call (RPC) interface. In some other embodiments, service device 220 may also send data to data requester 240 via other interfaces or protocols.

[0037] In some embodiments, the data request issued by the data requester 240 further includes an ordered read request. Multiple service devices 220 can retrieve data from the message queue 210 in a predetermined order based on the ordered read request. For example, according to the ordered read request, the first service device may retrieve first data from the message queue 210, and the second service device may retrieve second data from the message queue 210 in a predetermined order. Furthermore, according to the ordered read request, the multiple service devices 220 can also send data to the data requester 240 in a predetermined order. For example, the second service device may send the second data to the data requester 240 only after the first service device has finished sending the first data.

[0038] In some embodiments, multiple service devices 220 may read data from message queue 210 in an unordered manner. For example, if the data retrieval request issued by data requester 240 does not specify the order of data retrieval, multiple service devices 220 may retrieve data from message queue 210 in any order, and multiple service devices 220 may also send data to data requester 240 in any order.

[0039] Figure 4 A flowchart of a data transmission method 400 according to some embodiments of the present disclosure is shown. Figure 4 The method shown can be derived from, for example Figure 2 The service device shown performs the operation. It should be understood that method 400 may also include additional boxes not shown and / or one (or some) of the boxes shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0040] In box 410, at the first service device among a plurality of service devices, in response to a data retrieval request from a data requester, first data is read from a message queue, the message queue being associated with a database and coupled to the plurality of service devices, the message queue storing the target data read from the database, and the first data read by the first service device being a part of the stored target data.

[0041] In box 420, in response to reading the first data, the first service device writes the first data to the circular buffer of the first service device via the write pointer of the first service device.

[0042] In some embodiments, in response to reading first data, the first serving device moves a write pointer in a predetermined direction among a plurality of empty buffer blocks in a circular buffer. The first serving device writes the first data to at least one empty buffer block in the circular buffer via the write pointer, so that at least one empty buffer block becomes at least one full buffer block, and the first data is distributed within at least one full buffer block of the circular buffer. In some embodiments, if the write pointer of the first serving device fills a buffer block in the circular buffer with data, the write pointer of the first serving device moves in the predetermined direction on the circular buffer to the next adjacent buffer block (since the read operation of the read pointer is continuous in the circular buffer, the next adjacent buffer block is an empty buffer block), and continues to write data to that buffer block. In this way, the first data obtained by the first serving device from the message queue can be continuously written into the circular buffer.

[0043] In box 430, in response to a data transmission request initiated by the first service device to send data to the data requester, the first service device reads the first data from the circular buffer of the first service device via the read pointer of the first service device.

[0044] In box 440, the first service device sends the first data to the data requester.

[0045] In some embodiments, in response to a data transmission request initiated by the first service device to send data to a data requester, the first service device moves a read pointer in a predetermined direction among a plurality of full buffer blocks in a circular buffer. The first service device reads first data from at least one full buffer block of the circular buffer via the read pointer.

[0046] In some embodiments, the read pointer of the first service device moves in a predetermined direction among multiple buffer blocks of the circular buffer. If the read pointer moves to a full buffer block, it reads the data within the full buffer block and sends this data to the data requester. Subsequently, the read pointer moves in the predetermined direction to the next adjacent full buffer block and reads data.

[0047] In some embodiments, the data acquisition request includes a sequential read request. The data transmission method provided according to embodiments of this disclosure further includes, at a first service device among a plurality of service devices, in response to receiving a sequential read request from a data requester, reading first data from a message queue. At a second service device among the plurality of service devices, in response to receiving a sequential read request from a data requester, after the first service device has read the first data from the message queue, reading second data from the message queue, wherein the second data read by the second service device is a portion of the target data stored in the message queue. In this manner, after receiving a sequential read request from a data requester, the plurality of service devices can sequentially acquire data from the message queue in a predetermined order.

[0048] In some embodiments, a first service device among a plurality of service devices sends first data to a data requester. A second service device among the plurality of service devices sends target data to the data requester in response to the first service device having finished sending the first data.

[0049] In some embodiments, after receiving an ordered read request from a data requester, the server device sends data to the data requester in a predetermined order. For example, based on the ordered read request, a first server device first sends first data to the data requester, and after the first data is sent, a second server device then sends second data to the data requester. This method achieves ordered data transmission.

[0050] It should be understood that the above-mentioned first service device and second service device are merely exemplary distinctions made for the convenience of illustrating the sequential reading or sending of data by multiple service devices.

[0051] In some embodiments, a data retrieval request includes an out-of-order read request. After receiving an out-of-order read request, the service device can retrieve data from the message queue in any appropriate order. Alternatively, the service device can also send data to the data requester in any appropriate order.

[0052] In some embodiments, target data in the message queue is read from the database into the message queue based on a data retrieval request from a data requester.

[0053] Figure 5 A block diagram is shown of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. The electronic device 500 may, for example, be used to implement... Figure 2 The device 200 shown. It should be understood that, Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0054] like Figure 5As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors 510 or processing units, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0055] Electronic device 500 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 500.

[0056] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include a computer program product having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0057] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0058] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0059] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions, wherein one or more computer instructions are executed by a processor to implement the methods described above.

[0060] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0061] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0062] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0063] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0064] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method of data transmission, comprising: reading, at a first service device of a plurality of service devices, first data from a message queue in response to a data fetch request from a data requestor, the message queue being associated to a database and coupled to the plurality of service devices, the message queue storing target data read from the database, and the first data read by the first service device being a portion of the target data stored; writing, in response to reading the first data, the first data to a ring buffer of the first service device via a write pointer of the first service device; reading, in response to the first service device initiating a send request to send data to the data requestor, the first data from the ring buffer of the first service device via a read pointer of the first service device; and sending the first data to the data requestor.

2. The method of claim 1, wherein writing the first data comprises: moving the write pointer in a predetermined direction among a plurality of empty buffer blocks of the ring buffer in response to reading the first data; and writing the first data to the at least one empty buffer block of the ring buffer via the write pointer to cause the at least one empty buffer block to become at least one full buffer block, the first data being distributed within the at least one full buffer block of the ring buffer.

3. The method of claim 1 or 2, wherein reading the first data comprises: moving the read pointer in the predetermined direction among a plurality of full buffer blocks of the ring buffer in response to the first service device initiating the send request to send data to the data requestor; and reading the first data from at least one full buffer block of the ring buffer via the read pointer.

4. The method of claim 1, wherein the data fetch request comprises an in-order read request, the method further comprising: reading, at a first service device of a plurality of service devices, the first data from the message queue in response to receiving the in-order read request from the data requestor; and reading, at a second service device of a plurality of service devices, second data from the message queue after the first service device has read the first data from the message queue in response to receiving the in-order read request from the data requestor, the second data read by the second service device being a portion of the target data stored by the message queue.

5. The method of claim 4, further comprising: sending, at the first service device of a plurality of service devices, the first data to the data requestor; and sending, at the second service device of a plurality of service devices, the target data to the data requestor in response to the first service device having sent the first data to the data requestor.

6. The method of claim 1, wherein sending the first data to the data requestor comprises: sending the first data to the data requestor via a remote procedure call protocol (RPC). ​ ​ ​ ​ ​ 7. The method of claim 1, wherein the target data in the message queue is read from the database into the message queue based on the data acquisition request of the data requestor.

8. A data transmission system comprising a message queue coupled to a database, adapted to acquire and store target data in the database related to a data acquisition request of a data requestor; and a plurality of service devices coupled to the message queue, adapted to acquire at least in part data from the message queue and send the data to the data requestor.

9. An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit cause the electronic device to perform the method according to any one of claims 1 to 7.

10. A computer readable storage medium having stored thereon a computer program, the computer program being executable by a processor to implement the method according to any one of claims 1 to 7. ​

Images