Asynchronous consumption method and device based on ring buffer

Abstract

The present invention discloses an asynchronous consumption method and device based on a ring buffer, and relates to the field of computer technology. A specific implementation of the method includes: calculating the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer, and judging whether the real-time distance is greater than the safe zone distance; if so, writing the user records in the write thread into the ring buffer in sequence; reading the user records from the ring buffer through the read thread, and writing them into the remote queue to realize asynchronous consumption. This implementation can avoid the problem of concurrent reading and writing, reduce time consumption, and significantly improve the efficiency of asynchronous consumption.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an asynchronous consumption method and device based on a ring buffer. Background Art

[0002] When users access web pages through external channels, the online marketing system will persist the link access records with sampling parameters.

[0003] Common persistence processing methods include:

[0004] 1. Synchronously write to the database / remote queue, write each user access record to the database / remote queue and then return;

[0005] 2. Asynchronously write to the database / remote queue, cache the user access records in the local queue, and then write them to the database in batches through the background thread, or pull them individually through the background thread, serialize them, and send them to the remote queue.

[0006] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0007] 1. When writing synchronously, a large number of requests will lead to a decrease in writing efficiency and a longer time consumption;

[0008] 2. When writing to the database asynchronously, it is easy to cause memory overflow or block the user request thread, resulting in increased user request latency;

[0009] 3. When writing to a remote queue asynchronously, each single pull by the background thread will cause CAS (compare and swap) contention, resulting in a waste of CPU (Central Processing Unit). Summary of the invention

[0010] In view of this, an embodiment of the present invention provides an asynchronous consumption method and device based on a ring buffer, which can avoid the reading and writing concurrency problem, reduce time consumption, and significantly improve the efficiency of asynchronous consumption.

[0011] To achieve the above object, according to a first aspect of an embodiment of the present invention, a ring buffer-based asynchronous consumption method is provided, comprising:

[0012] Calculate the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer, and determine whether the real-time distance is greater than the safe zone distance;

[0013] If so, write the user records in the writing thread into the ring buffer in sequence;

[0014] The user records are read from the ring buffer by the reading thread and written into the remote queue to realize asynchronous consumption.

[0015] Furthermore, before the step of reading the user record from the circular buffer area through the reading thread, the asynchronous consumption method based on the circular buffer area further includes: determining that the address corresponding to the write pointer is consistent with the address corresponding to the wake-up pointer.

[0016] Furthermore, the step of calculating the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer includes: converting the address corresponding to the write pointer and the address corresponding to the read pointer into array subscripts, and calculating the real-time distance between the array subscripts.

[0017] Furthermore, the step of reading user records from the ring buffer through the reading thread includes:

[0018] Loop through the following steps: determine whether there is a user record in the address corresponding to the read pointer; if so, add the user record to the consumption table and move the address corresponding to the read pointer down by one position;

[0019] Until there is no user record in the address corresponding to the read pointer or the number of user records in the consumption table is equal to the quantity threshold.

[0020] Furthermore, if there is no user record in the address corresponding to the read pointer, the address corresponding to the read pointer is set to the address corresponding to the wake-up pointer and waiting is performed. When the waiting time exceeds the threshold time, the loop execution step is performed.

[0021] Furthermore, before the step of writing into the remote queue, the asynchronous consumption method based on the ring buffer further includes: serializing the user records read from the ring buffer.

[0022] According to a second aspect of an embodiment of the present invention, there is provided an asynchronous consumption device based on a ring buffer, comprising:

[0023] A real-time distance calculation module, used to calculate the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer;

[0024] The safety zone distance module is used to determine whether the real-time distance is greater than the safety zone distance;

[0025] A writing module, when the real-time distance is greater than the safe zone distance, the writing module is used to write the user records in the writing thread into the ring buffer in sequence;

[0026] The reading module is used to read user records from the ring buffer through the reading thread and write them into the remote queue to achieve asynchronous consumption.

[0027] Furthermore, the asynchronous consumption device based on the ring buffer further comprises an address determination module, and before the reading module reads the user record, the address determination module is used to determine whether the address corresponding to the write pointer is consistent with the address corresponding to the wake-up pointer.

[0028] According to a third aspect of an embodiment of the present invention, a terminal is provided, including:

[0029] one or more processors;

[0030] a storage device for storing one or more programs,

[0031] When one or more programs are executed by one or more processors, the one or more processors implement any of the above-mentioned asynchronous consumption methods based on the ring buffer.

[0032] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the program implements any of the above-mentioned asynchronous consumption methods based on a ring buffer.

[0033] An embodiment of the above invention has the following advantages or beneficial effects: because the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer is calculated, it is determined whether the real-time distance is greater than the safety zone distance; if so, the user records in the write thread are written into the ring buffer in sequence; the user records are read from the ring buffer by the read thread and written into the remote queue to realize the technical means of asynchronous consumption, thereby overcoming the technical problems of low efficiency, long delay, and easy occurrence of read-write concurrency in the prior art, thereby achieving the technical effect of avoiding read-write concurrency problems, reducing time consumption, and significantly improving the efficiency of asynchronous consumption.

[0034] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0036] Figure 1 is a schematic diagram of the main process of the asynchronous consumption method based on the ring buffer provided according to the first embodiment of the present invention;

[0037] Figure 2a is a schematic diagram of the main process of the asynchronous consumption method based on the ring buffer provided according to the second embodiment of the present invention;

[0038] Figure 2b yes Figure 2a Schematic diagram of the ringbuffer structure in the method;

[0039] Figure 3 is a schematic diagram of main modules of an asynchronous consumption device based on a ring buffer provided according to an embodiment of the present invention;

[0040] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0041] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0043] Figure 1 is a schematic diagram of the main process of the asynchronous consumption method based on the ring buffer according to the first embodiment of the present invention, such as Figure 1 As shown, the asynchronous consumption method based on the ring buffer provided by the embodiment of the present invention includes:

[0044] Step S101, calculate the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer, and determine whether the real-time distance is greater than the safe zone distance. If not, that is, the real-time distance is less than the safe zone distance, then end; if yes, that is, the real-time distance is greater than the safe zone distance, then execute step S102.

[0045] The safe zone is set to prevent concurrent problems between write threads and read threads. The safe zone distance is not less than the number of write threads. Through the above settings, less space can be used to avoid concurrent problems between read and write threads.

[0046] According to an embodiment of the present invention, before the step of reading user records from the circular buffer area by a read thread, the asynchronous consumption method based on the circular buffer area further includes: determining that the address corresponding to the write pointer is consistent with the address corresponding to the wake-up pointer. By judging whether the address corresponding to the write pointer is equal to the address corresponding to the wake-up pointer, if they are equal, it indicates that there is a read thread waiting there, and at this time, all read threads waiting at the address are woken up to perform a read operation, which can effectively improve the reading efficiency.

[0047] Specifically, according to one embodiment of the present invention, the step of calculating the real-time distance between the address corresponding to the write pointer (pushIndex) and the address corresponding to the read pointer (pollIndex) in the ring buffer includes: converting the address corresponding to the write pointer and the address corresponding to the read pointer into array indices, and calculating the real-time distance between the array indices.

[0048] Before performing the above steps, the ring buffer array is initialized to allocate memory space in the ring buffer array. The elements in the array are all NULL by default, that is, null pointers. During asynchronous consumption, the array index conversion formula is used. When Push (write), the array index = pushIndex & (capacity-1), when Poll (read), the array index = pollIndex & (capacity-1), where capacity refers to the length of the ringbuffer array, and then the real-time distance between the read pointer and the write pointer is calculated.

[0049] Step S102, writing the user records in the writing thread into the ring buffer in sequence.

[0050] Specifically, the write pointer address to be written is obtained according to the CAS (compare and swap) instruction, and the user records are written in sequence. There is no lock operation during the writing process of the push thread (user request), only one CAS operation, and the time consumption is extremely small.

[0051] Step S103, reading user records from the ring buffer through a reading thread and writing them into a remote queue to realize asynchronous consumption.

[0052] Specifically, when multiple read threads perform a read operation, they first compete for the poll lock (read lock), thereby reducing the number of CAS operations performed by the multiple read threads and further reducing time consumption.

[0053] Specifically, according to an embodiment of the present invention, the step of reading user records from the ring buffer through a reading thread includes:

[0054] Loop through the following steps: determine whether there is a user record in the address corresponding to the read pointer; if so, add the user record to the consumption table and move the address corresponding to the read pointer down by one position;

[0055] Until there is no user record in the address corresponding to the read pointer or the number of user records in the consumption table is equal to the quantity threshold.

[0056] That is, by cyclically determining whether there is a user record in the subscript element (in the corresponding address) corresponding to pollIndex (read pointer) to determine whether consumption (reading operation) is required, the background processing thread (reading thread) can read data in batches at one time, further reducing CAS competition and improving the efficiency of asynchronous consumption.

[0057] Furthermore, if there is no user record in the address corresponding to the read pointer, the address corresponding to the read pointer is set to the address corresponding to the wake-up pointer and waits. When the waiting time exceeds the threshold time, the loop execution step is performed. Specifically, waiting (Thread.sleep) is to release the CPU and block other poll threads (at this time, the element corresponding to pollIndex is NULL, and there is no element in the ringbuffer). The waiting (sleeping) time should not be too long, generally within 3 seconds. Too long may cause the poll thread to consume untimely. The wake-up pointer speeds up the startup speed of the poll thread to consume again when there is no element (user record) to consume, further shortening the time consumption of asynchronous consumption.

[0058] Specifically, according to an embodiment of the present invention, before the step of writing into the remote queue, the asynchronous consumption method based on the ring buffer further includes: serializing the user records read from the ring buffer.

[0059] According to the technical solution of the embodiment of the present invention, the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer is calculated to determine whether the real-time distance is greater than the safety zone distance; if so, the user records in the write thread are written into the ring buffer in sequence; the user records are read from the ring buffer by the read thread and written into the remote queue to realize the technical means of asynchronous consumption, thereby overcoming the technical problems of low efficiency, long delay and easy occurrence of read-write concurrency in the prior art, thereby achieving the technical effect of avoiding read-write concurrency problems, reducing time consumption and significantly improving the efficiency of asynchronous consumption.

[0060] Figure 2a is a schematic diagram of the main process of the asynchronous consumption method based on the ring buffer according to the second embodiment of the present invention; Figure 2a As shown, the asynchronous consumption method based on the ring buffer provided by the embodiment of the present invention includes:

[0061] Step S201, initializing the ringbuffer array.

[0062] Specifically, the structural diagram of the ring buffer is as follows: Figure 2bAs shown, the ringbuffer is an array, and the memory addresses of its elements are stored continuously. The ringbuffer array is initialized to allocate array memory space, so that the elements in the array are NULL (null pointer) by default. In order to facilitate the subsequent calculation of real-time distance conversion, the length of the ringbuffer is 2 to the power of N.

[0063] Step S202, calculating the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer.

[0064] Specifically, according to an embodiment of the present invention, during the asynchronous consumption process, the array subscript conversion calculation formula is used: when Push (write), the array subscript = pushIndex & (capacity-1); when Poll (read), the array subscript = pollIndex & (capacity-1), where capacity refers to the length of the ringbuffer array, and then the real-time distance between the read pointer and the write pointer is calculated.

[0065] Step S203, determine whether the real-time distance is greater than the safety zone distance. If yes, execute step S204, if no, end the process.

[0066] The safe zone is set to prevent concurrent problems between write threads and read threads. The safe zone distance is not less than the number of write threads. Through the above settings, less space can be used to avoid concurrent problems between read and write threads.

[0067] Step S204: Write the user requests in the write thread into the ring buffer in sequence through the CAS instruction.

[0068] There is no locking operation in the writing process of the writing thread, and only one CAS operation is required, which greatly shortens the time consumption.

[0069] Step S205, determine whether the address corresponding to the write pointer is consistent with the address corresponding to the wake-up pointer. If they are consistent, execute step S206, if not, go to step S202.

[0070] By judging whether the address corresponding to the write pointer is equal to the address corresponding to the wake-up pointer, if they are equal, it indicates that there is a read thread waiting here. At this time, all read threads waiting at the address are woken up to perform the read operation, which can effectively improve the reading efficiency.

[0071] Step S206: Multiple read threads compete for the read lock and read the user request from the ring buffer.

[0072] When multiple read threads perform read operations, they compete for the poll lock (read lock) first and then perform the read operation, thereby reducing the number of CAS operations performed by multiple read threads and further reducing time consumption.

[0073] Step S207, serialize the pulled user requests and write them into the remote queue to achieve asynchronous consumption.

[0074] Specifically, according to an embodiment of the present invention, the step of reading user records from the ring buffer through a reading thread includes:

[0075] Loop through the following steps: Determine whether there is a generation consumption element (user record) in the address corresponding to the read pointer. If so, remove the user record from the ringbuffer and add it to the consumption table. Update the read pointer address to pollIndex++ (move the address corresponding to the read pointer one position in the reading direction).

[0076] Until there is no user record in the address corresponding to the read pointer or the number of user records in the consumption table is equal to the quantity threshold.

[0077] That is, by cyclically determining whether there is a user record in the subscript element (in the corresponding address) corresponding to pollIndex (read pointer) to determine whether consumption (reading operation) is required, the background processing thread (reading thread) can read data in batches at one time, further reducing CAS competition and improving the efficiency of asynchronous consumption.

[0078] Furthermore, if there is no user record in the address corresponding to the read pointer, the address corresponding to the read pointer is set to the address corresponding to the wake-up pointer and waits. When the waiting time exceeds the threshold time, the loop execution step is performed. Specifically, waiting (Thread.sleep) is to release the CPU and block other poll threads (at this time, the element corresponding to pollIndex is NULL, and there is no element in the ringbuffer). The waiting (sleeping) time should not be too long, generally within 3 seconds. Too long may cause the poll thread to consume untimely. The wake-up pointer speeds up the startup speed of the poll thread to consume again when there is no element (user record) to consume, further shortening the time consumption of asynchronous consumption.

[0079] According to the technical solution of the embodiment of the present invention, the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer is calculated to determine whether the real-time distance is greater than the safety zone distance; if so, the user records in the write thread are written into the ring buffer in sequence; the user records are read from the ring buffer by the read thread and written into the remote queue to realize the technical means of asynchronous consumption, thereby overcoming the technical problems of low efficiency, long delay and easy occurrence of read-write concurrency in the prior art, thereby achieving the technical effect of avoiding read-write concurrency problems, reducing time consumption and significantly improving the efficiency of asynchronous consumption.

[0080] Figure 3 is a schematic diagram of main modules of an asynchronous consumption device based on a ring buffer according to an embodiment of the present invention; Figure 3 As shown, the asynchronous consumption device 300 based on the ring buffer provided by the embodiment of the present invention includes:

[0081] The real-time distance calculation module 301 is used to calculate the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer.

[0082] The safe zone is set to prevent concurrent problems between write threads and read threads. The safe zone distance is not less than the number of write threads. Through the above settings, less space can be used to avoid concurrent problems between read and write threads.

[0083] According to an embodiment of the present invention, the above-mentioned asynchronous consumption device 300 based on the ring buffer also includes an address determination module. Before the reading module reads the user record from the ring buffer area through the read thread, the address determination module is used to determine whether the address corresponding to the write pointer is consistent with the address corresponding to the wake-up pointer.

[0084] By judging whether the address corresponding to the write pointer is equal to the address corresponding to the wake-up pointer, if they are equal, it indicates that there is a read thread waiting here. At this time, all read threads waiting at the address are woken up to perform the read operation, which can effectively improve the reading efficiency.

[0085] Specifically, according to an embodiment of the present invention, the real-time distance calculation module 301 is further used to: convert the address corresponding to the write pointer and the address corresponding to the read pointer into array subscripts, and calculate the real-time distance between the array subscripts.

[0086] Initialize the ringbuffer array first to allocate memory space in the ringbuffer array. The elements in the array are all NULL by default, that is, null pointers. During asynchronous consumption, the array index conversion formula is used. When Push (write), the array index = pushIndex & (capacity-1), when Poll (read), the array index = pollIndex & (capacity-1), where capacity refers to the length of the ringbuffer array, and then calculate the real-time distance between the read pointer and the write pointer.

[0087] The safety zone distance module 302 is used to determine whether the real-time distance is greater than the safety zone distance. If not, that is, the real-time distance is less than the safety zone distance, then the process ends; if yes, that is, the real-time distance is greater than the safety zone distance, then the write operation is performed.

[0088] The writing module 303 is used to write the user records in the writing thread into the ring buffer in sequence when the real-time distance is greater than the safety zone distance.

[0089] Specifically, the write pointer address to be written is obtained according to the CAS (compare and swap) instruction, and the user records are written in sequence. There is no lock operation during the writing process of the push thread (user request), only one CAS operation, and the time consumption is extremely small.

[0090] The reading module 304 is used to read user records from the ring buffer through a reading thread and write them into a remote queue to realize asynchronous consumption.

[0091] Specifically, when multiple read threads perform a read operation, they first compete for the poll lock (read lock), thereby reducing the number of CAS operations performed by the multiple read threads and further reducing time consumption.

[0092] Specifically, according to an embodiment of the present invention, the reading module 304 is further used for:

[0093] Loop through the following steps: determine whether there is a user record in the address corresponding to the read pointer; if so, add the user record to the consumption table and move the address corresponding to the read pointer down by one position;

[0094] Until there is no user record in the address corresponding to the read pointer or the number of user records in the consumption table is equal to the quantity threshold.

[0095] That is, by cyclically determining whether there is a user record in the subscript element (in the corresponding address) corresponding to pollIndex (read pointer) to determine whether consumption (reading operation) is required, the background processing thread (reading thread) can read data in batches at one time, further reducing CAS competition and improving the efficiency of asynchronous consumption.

[0096] Further, if there is no user record in the address corresponding to the read pointer, the reading module 304 is used to set the address corresponding to the read pointer to the address corresponding to the wake-up pointer and wait, and when the waiting time exceeds the threshold time, the loop execution step is performed. Specifically, waiting (Thread.sleep) is to release the CPU and block other poll threads (at this time, the element corresponding to pollInde4 is NULL, and there is no element in the ringbuffer). The waiting (sleeping) time should not be too long, generally within 3 seconds. Too long may cause the poll thread to consume untimely. The wake-up pointer speeds up the startup speed of the poll thread to consume again when there is no element (user record) to consume, further shortening the time consumption of asynchronous consumption.

[0097] Specifically, according to an embodiment of the present invention, before the step of writing into the remote queue, the reading module 304 is further used for: the asynchronous consumption method based on the ring buffer further includes: serializing the user records read from the ring buffer.

[0098] According to the technical solution of the embodiment of the present invention, the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer is calculated to determine whether the real-time distance is greater than the safety zone distance; if so, the user records in the write thread are written into the ring buffer in sequence; the user records are read from the ring buffer by the read thread and written into the remote queue to realize the technical means of asynchronous consumption, thereby overcoming the technical problems of low efficiency, long delay and easy occurrence of read-write concurrency in the prior art, thereby achieving the technical effect of avoiding read-write concurrency problems, reducing time consumption and significantly improving the efficiency of asynchronous consumption.

[0099] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part and will not be repeated here.

[0100] Figure 4 An exemplary system architecture 400 is shown to which an asynchronous consumption method based on a ring buffer or an asynchronous consumption device based on a ring buffer according to an embodiment of the present invention can be applied.

[0101] like Figure 4As shown, the system architecture 400 may include terminal devices 401, 402, 403, a network 404 and a server 405 (this architecture is only an example, and the components included in the specific architecture may be adjusted according to the specific application). The network 404 is used to provide a medium for a communication link between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0102] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0103] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0104] The server 405 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 401, 402, and 403. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0105] It should be noted that the asynchronous consumption method based on the ring buffer provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the asynchronous consumption device based on the ring buffer is generally disposed in the server 405 .

[0106] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0107] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device suitable for implementing an embodiment of the present invention. Figure 5 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0108] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0109] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0110] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0111] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor, for example, may be described as: a processor includes a real-time distance calculation module, a safety zone distance module, a writing module and a reading module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the real-time distance calculation module may also be described as "a module for calculating the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer".

[0114] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: calculating the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer, and determining whether the real-time distance is greater than the safe zone distance; if so, writing the user records in the write thread into the ring buffer in sequence; reading the user records from the ring buffer through the read thread and writing them into the remote queue to realize asynchronous consumption.

[0115] According to the technical solution of the embodiment of the present invention, the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer is calculated to determine whether the real-time distance is greater than the safety zone distance; if so, the user records in the write thread are written into the ring buffer in sequence; the user records are read from the ring buffer by the read thread and written into the remote queue to realize the technical means of asynchronous consumption, thereby overcoming the technical problems of low efficiency, long delay and easy occurrence of read-write concurrency in the prior art, thereby achieving the technical effect of avoiding read-write concurrency problems, reducing time consumption and significantly improving the efficiency of asynchronous consumption.

[0116] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An asynchronous consumption method based on a ring buffer, characterized in that: include: Calculate the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer, and determine whether the real-time distance is greater than the safe zone distance, and the safe zone distance is not less than the number of threads of the write thread; If so, write the user records in the writing thread into the ring buffer in sequence; The user records are read from the ring buffer by a reading thread and written into a remote queue to realize asynchronous consumption.

2. The asynchronous consumption method based on the ring buffer according to claim 1, characterized in that: Before the step of reading user records from the circular buffer area through a read thread, the asynchronous consumption method based on the circular buffer area further includes: determining that the address corresponding to the write pointer is consistent with the address corresponding to the wake-up pointer.

3. The asynchronous consumption method based on the ring buffer according to claim 1, characterized in that: The step of calculating the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer includes: converting the address corresponding to the write pointer and the address corresponding to the read pointer into array subscripts, and calculating the real-time distance between the array subscripts.

4. The asynchronous consumption method based on the ring buffer according to claim 1, characterized in that: The step of reading user records from the ring buffer through the reading thread comprises: Loop through the following steps: determine whether there is a user record in the address corresponding to the read pointer, if so, add the user record to the consumption table, and move the address corresponding to the read pointer down by one position; Until there is no user record in the address corresponding to the read pointer or the number of user records in the consumption table is equal to the quantity threshold.

5. The asynchronous consumption method based on the ring buffer according to claim 4, characterized in that: If there is no user record in the address corresponding to the read pointer, the address corresponding to the read pointer is set to the address corresponding to the wake-up pointer and waiting is performed. When the waiting time exceeds the threshold time, the loop execution step is performed.

6. The asynchronous consumption method based on the ring buffer according to claim 1, characterized in that: Before the step of writing into the remote queue, the asynchronous consumption method based on the ring buffer further includes: serializing the user records read from the ring buffer.

7. An asynchronous consumption device based on a ring buffer, characterized in that: include: A real-time distance calculation module, used to calculate the real-time distance between the address corresponding to the write pointer and the address corresponding to the read pointer in the ring buffer; A safety zone distance module, used to determine whether the real-time distance is greater than the safety zone distance, and the safety zone distance is not less than the number of threads of the writing thread; A writing module, when the real-time distance is greater than the safety zone distance, the writing module is used to write the user records in the writing thread into the ring buffer in sequence; The reading module is used to read user records from the ring buffer through a reading thread and write them into a remote queue to realize asynchronous consumption.

8. The asynchronous consumption device based on the ring buffer according to claim 7, characterized in that: The asynchronous consumption device based on the ring buffer further comprises an address determination module, and before the reading module reads the user record, the address determination module is used to determine whether the address corresponding to the write pointer is consistent with the address corresponding to the wake-up pointer.

9. A terminal, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images