A data cache processing method and device

Abstract

The present invention discloses a data cache processing method and device. A specific embodiment of the method is applied to a local cache, wherein the local cache includes two blocking queues and a pointer for indicating a switch between the two blocking queues. The method comprises the following steps: obtaining data to be processed; determining the first blocking queue of the two blocking queues to which the pointer currently points, and writing the data to be processed into the first blocking queue; when the data to be processed in the first blocking queue meets a preset condition, copying the data to be processed in the first blocking queue to a target thread so that the target thread can process the data to be processed; and pointing the pointer to the second blocking queue of the two blocking queues so that the data to be processed can continue to be written into the second blocking queue. The method relates to the field of computer technology. This embodiment improves transmission speed and throughput.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data cache processing method and device. Background Art

[0002] In high-concurrency scenarios, when the concurrency is high enough, local cache is often used to reduce the pressure on third-party caches (such as distributed caches).

[0003] When using the local cache to read and write data, the data is first stored in the local cache, which requires locking. When the amount of data in the local cache or the time interval reaches a threshold, the data is read from the local cache and written to the third-party cache, which also requires locking. Once the data is successfully written to the third-party cache, the corresponding data can be deleted and the lock released.

[0004] In the process of implementing the present invention, the inventors discovered that the prior art has at least the following problems:

[0005] Writing data from the local cache to a third-party cache involves network IO (Input / Output), which takes a long time. In high-concurrency scenarios, when the data volume is large enough, the local cache will quickly reach the threshold, triggering a lock. The write operation will have to wait until the data is written to the third-party cache and the local cache is cleared before it can continue, which will significantly reduce transmission speed and throughput. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a data cache processing method and device, which can improve transmission speed and throughput.

[0007] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for processing data cache is provided.

[0008] A data cache processing method according to an embodiment of the present invention is applied to a local cache, wherein the local cache includes two blocking queues and a pointer for indicating a switch between the two blocking queues; the method includes:

[0009] Get the data to be processed;

[0010] Determine a first blocking queue of the two blocking queues currently pointed to by the pointer, and write the to-be-processed data into the first blocking queue;

[0011] When the data to be processed in the first blocking queue meets a preset condition, copying the data to be processed in the first blocking queue to a target thread, so as to process the data to be processed by the target thread;

[0012] The pointer is pointed to the second blocking queue of the two blocking queues, so as to continue writing the data to be processed through the second blocking queue.

[0013] Optionally, the method further includes:

[0014] The pointer is locked while pointing from the first blocking queue to the second blocking queue, and the lock is released after the pointer points to the second blocking queue.

[0015] Optionally, the processing the data to be processed by using the target thread includes:

[0016] The thread enclosing technology is adopted to enclose the data to be processed inside the target thread, so that the target thread is used to write the data to be processed into the distributed cache.

[0017] Optionally, after using the target thread to write the to-be-processed data into a distributed cache, the method further includes:

[0018] Release the data to be processed in the target thread.

[0019] Optionally, the method further includes: writing a duration threshold or a storage space threshold in a callback manner;

[0020] The preset conditions include: the storage amount of the data to be processed is not less than the storage space threshold, or the time length for writing the data to be processed into the first blocking queue is not less than the time length threshold.

[0021] Optionally, the storage space of the first blocking queue and the second blocking queue are equal.

[0022] Optionally, the first blocking queue and / or the second blocking queue is a ConcurrentLinked Queue linked queue.

[0023] To achieve the above objective, according to another aspect of an embodiment of the present invention, a data cache processing device is provided.

[0024] A data cache processing device according to an embodiment of the present invention includes: a data writing module, a data copying module and a switching module; wherein,

[0025] The data writing module is configured to obtain data to be processed, determine a first blocking queue of two blocking queues currently pointed to by a pointer in a local cache, and write the data to be processed into the first blocking queue; wherein the local cache includes: the two blocking queues, and a pointer for indicating a switch between the two blocking queues;

[0026] The data copy module is configured to copy the data to be processed in the first blocking queue to a target thread when the data to be processed in the first blocking queue meets a preset condition, so as to process the data to be processed by the target thread;

[0027] The switching module is used to point the pointer to the second blocking queue of the two blocking queues, so as to continue writing the data to be processed through the second blocking queue.

[0028] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device for processing data is provided.

[0029] An electronic device for processing data according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a data cache processing method according to an embodiment of the present invention.

[0030] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided.

[0031] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, which, when executed by a processor, implements a data cache processing method according to an embodiment of the present invention.

[0032] One embodiment of the above invention has the following advantages or beneficial effects: by setting two blocking queues in the local cache, and switching between the two blocking queues using the pointing of a pointer. When receiving the data to be processed, the data to be processed is first written into the first blocking queue to which the pointer currently points; when the data in the first blocking queue meets the preset conditions, the data can be processed by the target thread, and at the same time, the pointer is pointed to the second blocking queue to continue to store subsequent data to be processed using the second blocking queue. In this process, due to the use of the two blocking queue switching method, while the first blocking queue copies the data to the target thread, the second blocking queue can continue to write the subsequent data, thereby reducing the waiting time for writing and thereby improving the transmission speed and throughput. In addition, due to the use of blocking queues, the process of writing data into the blocking queue does not require locking, thereby further improving the transmission speed and throughput by reducing the number of locks.

[0033] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0035] Figure 1 is a schematic diagram of main steps of a data cache processing method according to an embodiment of the present invention;

[0036] Figure 2 is a flowchart of another data cache processing method according to an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of main steps of another data cache processing method according to an embodiment of the present invention;

[0038] Figure 4 is a schematic diagram of main modules of a data cache processing device according to an embodiment of the present invention;

[0039] Figure 5 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0040] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0042] It should be pointed out that, in the absence of conflict, the embodiments of the present invention and the technical features therein may be combined with each other.

[0043] Figure 1 3 is a schematic diagram of main steps of a data cache processing method according to an embodiment of the present invention.

[0044] like Figure 1 As shown, a data cache processing method according to an embodiment of the present invention can be applied to a local cache, wherein the local cache includes two blocking queues and a pointer for indicating switching between the two blocking queues; the method mainly includes the following steps:

[0045] Step S101: Obtain data to be processed.

[0046] The data to be processed is the data to be written to the local cache. For example, when adding a local cache between an application and Redis to reduce the pressure on Redis, the data to be processed can be data generated by the application or data received by the application.

[0047] Step S102: Determine the first blocking queue of the two blocking queues to which the pointer currently points, and write the to-be-processed data into the first blocking queue.

[0048] Since two blocking queues are set up in the local cache, and pointers are used to indicate the switching between the two blocking queues, when preparing to write data to the local cache, such as when calling the put method to write data to the local cache, the data will be written to the current queue, and the current queue is the pointer. If the pointer points to the first blocking queue, the data to be processed will be directly written to the first blocking queue through the pointer of the current queue. Here, the blocking queue can be a BlockingQueue in Java. Through the characteristics of the blocking queue - when there is no data in the queue, all threads on the consumer side will be automatically blocked (suspended) until data is put into the queue, and when the queue is full of data, all threads on the producer side will be automatically blocked (suspended) until there is an empty position in the queue, and the thread will be automatically awakened. This can ensure the thread safety of the data write operation. Therefore, the data write process does not need to be locked, thereby ensuring the performance and throughput of the local cache when facing high concurrent writes.

[0049] It is understandable that there is no order between the first blocking queue and the second blocking queue in the embodiment of the present invention, and the description of the first and the second does not impose any limitation on the two blocking queues, but is only used to illustrate the switching of the pointer between the two blocking queues. When the two blocking queues are ready to write the data to be processed, the blocking queue that the pointer currently points to is the first blocking queue. For example, when a blocking queue A and a blocking queue B are set in the local cache, if the data to be processed is ready to be written, the pointer currently points to the blocking queue A, then the blocking queue A is the first blocking queue at this time. When the data to be processed is ready to be written at another moment, the pointer currently points to the blocking queue B, then the blocking queue B is the first blocking queue at this time.

[0050] Step S103: When the data to be processed in the first blocking queue meets a preset condition, the data to be processed in the first blocking queue is copied to a target thread, so that the target thread is used to process the data to be processed.

[0051] Among them, the preset conditions include: the storage volume of the data to be processed is not less than the storage space threshold, or the time length for writing the data to be processed into the first blocking queue is not less than the time length threshold. The storage space threshold and the time length threshold can be specified by the user in the callback method. For example, in Java 8 and above, the storage space threshold and the time length threshold can be specified through the Consumer in the stream, thereby handing over the user operation to the cache to control time and space through the callback method. For example, when the storage space threshold is 2M and the time length threshold is 1s, if the data to be processed stored in the first blocking queue reaches 2M or the time interval reaches 1s, the data to be processed in the first blocking queue will be copied to the target thread so that the target thread can be used to process the data to be processed.

[0052] To ensure thread safety, thread containment technology can be used to enclose the data to be processed within a target thread, allowing the target thread to write the data to the distributed cache. This prevents tampering with the data during the target thread's writing of the data to the distributed cache, ensuring thread safety. This process eliminates the need for locking, further reducing the number of locks required to maintain local cache performance and improve transmission speed and throughput.

[0053] It is understandable that in addition to writing the data to be processed into the distributed cache, the target thread can also implement other operations specified by the user. The specific operations on the data to be processed may vary according to the actual scenario. For example, the target thread executes the user operation, reads the data to be processed, and writes it to redis. The user operation is a method exposed to the user, which is used to implement specific operations and can be implemented using java Stream Custom. After the user operation is completed, the target thread and its internal held variables will be released. That is to say, after the data to be processed in the first blocking queue is written into the distributed cache, the data to be processed in the target thread is released to wait for the second blocking queue to copy the subsequent data to be processed to the target thread, and this cycle is repeated to implement data cache processing.

[0054] Step S104: point the pointer to the second blocking queue of the two blocking queues, so as to use the second blocking queue to continue storing the data to be processed.

[0055] After the data to be processed is copied to the target thread, the pointer is switched so that it points to the second blocking queue. In order to ensure that the data to be processed in the first blocking queue and the second blocking queue can be copied to the target thread when the preset conditions are met, in one embodiment of the present invention, the storage space of the first blocking queue and the second blocking queue is equal, that is, the size of the first blocking queue and the second blocking queue is exactly the same, thereby ensuring data controllability.

[0056] Since the data storage capacity of the local cache is generally small, the first blocking queue and the second blocking queue may be filled with data in a very short time, and pointer switching is required. In other words, the pointer switching operation may be relatively frequent. In one embodiment of the present invention, in order to ensure that the pointer switching is correct, the process of the pointer pointing from the first blocking queue to the second blocking queue is locked, and the lock is released after the pointer points to the second blocking queue. It is understandable that the pointer is also locked in the process of pointing from the second blocking queue to the first blocking queue, and the lock is released after the pointer points to the first blocking queue. In short, the pointer switching process is locked, and the lock is released after the pointer switching is completed, thereby ensuring the orderly switching of the pointer.

[0057] The first blocking queue and / or the second blocking queue can use the more efficient Concurrent Linked Queue as a storage medium. Since the Concurrent Linked Queue itself is implemented using CAS and head and tail pointers to ensure thread safety, and the chain structure itself has the characteristic of fast writing, the overall efficiency is very high.

[0058] Reference below Figure 2 The queue and thread diagram shown in FIG. 1 further explains the data cache processing method provided by the embodiment of the present invention. Figure 3 As shown, the method mainly includes the following steps:

[0059] Step S301: Obtain data to be processed.

[0060] Step S302: Determine the first blocking queue currently pointed to by the pointer among the two blocking queues in the local cache, and write the to-be-processed data into the first blocking queue.

[0061] When preparing to write data to the local cache, such as calling the put method to write data to the local cache, the data will be written to the current queue, and the current queue is the pointer. Figure 2 As shown in the figure, the queue currently pointed to by the pointer is queue 1, and the data to be processed is written to queue 1. Since queue 1 is a blocking queue, no locking is required during the writing process.

[0062] Step S303: When the storage amount of the data to be processed in the first blocking queue is not less than the preset space threshold, or the time for writing the data to be processed into the first blocking queue is not less than the time threshold, the data to be processed in the first blocking queue is copied to the child thread so that the child thread can be used to process the data to be processed.

[0063] Here, the data to be processed can be copied to Figure 2The child thread shown in the figure is inferior and uses thread blocking technology to ensure thread safety. Then, the child thread will perform user operations, such as reading data and writing it to Redis.

[0064] Step S304: point the pointer to the second blocking queue of the two blocking queues, so as to continue writing the data to be processed through the second blocking queue.

[0065] like Figure 2 As shown in the figure, after the data copy is completed, the pointer is switched from Queue 1 to Queue 2, and the sizes of Queue 2 and Queue 1 are exactly the same. At this time, the new write operation will store the data in Queue 2, that is, the subsequent pending data will be written to Queue 2. When the data in Queue 2 meets the preset conditions, the data in Queue 2 is copied to the child thread, just like the operation of Queue 1, and the pointer is switched back to Queue 1. This reciprocating process realizes the writing of data in the local cache through queue switching. In addition, the queue switching process is locked to ensure the orderly switching of queues, thereby improving the performance of the local cache, increasing transmission speed and throughput.

[0066] Step S305: After the child thread is used to write the data to be processed into the distributed cache, the data to be processed in the child thread is released.

[0067] Here, after the user operation is completed, the child thread and its internal held variables will be released to wait for the next data copy.

[0068] According to a data cache processing method according to an embodiment of the present invention, two blocking queues are set up in the local cache and switched between the two blocking queues using the pointer. When receiving data to be processed, the data is first written to the first blocking queue currently pointed to by the pointer. When the data in the first blocking queue meets the preset conditions, the data can be processed by the target thread, and the pointer is simultaneously pointed to the second blocking queue, so that the second blocking queue can continue to store subsequent data to be processed. During this process, due to the switching between the two blocking queues, the first blocking queue can copy data to the target thread while the second blocking queue continues to write subsequent data, thereby reducing the waiting time for writing and improving transmission speed and throughput. In addition, due to the use of blocking queues, the process of writing data to the blocking queue does not require locking, thereby further improving transmission speed and throughput by reducing the number of locks. Furthermore, the thread blocking technology is used to enclose the data to be processed within the target thread, so the process of writing data to the distributed cache also does not require locking, thereby further reducing the number of locks to ensure the performance of the local cache and improve transmission speed and throughput.

[0069] Figure 4It is a schematic diagram of main modules of a data cache processing device according to an embodiment of the present invention.

[0070] like Figure 4 As shown, a data cache processing device 400 according to an embodiment of the present invention includes: a data writing module 401, a data copying module 402 and a switching module 403; wherein,

[0071] The data writing module 401 is configured to obtain data to be processed, determine the first blocking queue of the two blocking queues currently pointed to by the pointer in the local cache, and write the data to be processed into the first blocking queue; wherein the local cache includes: the two blocking queues and a pointer for indicating a switch between the two blocking queues;

[0072] The data copy module 402 is configured to copy the data to be processed in the first blocking queue to a target thread when the data to be processed in the first blocking queue meets a preset condition, so that the target thread processes the data to be processed;

[0073] The switching module 403 is configured to point the pointer to the second blocking queue of the two blocking queues, so as to continue writing the data to be processed into the second blocking queue.

[0074] In one embodiment of the present invention, the switching module 403 is configured to lock the pointer while it is pointing from the first blocking queue to the second blocking queue, and release the lock after the pointer points to the second blocking queue.

[0075] In one embodiment of the present invention, the data replication module 402 is configured to use a thread sealing technology to seal the data to be processed inside a target thread, so as to write the data to be processed into a distributed cache using the target thread.

[0076] In one embodiment of the present invention, the data copy module 402 is further configured to release the data to be processed in the target thread.

[0077] In one embodiment of the present invention, the data writing module 401 is used to write a duration threshold or a storage space threshold using a callback method; the preset conditions include: the storage amount of the data to be processed is not less than the storage space threshold, or the duration of writing the data to be processed into the first blocking queue is not less than the duration threshold.

[0078] In one embodiment of the present invention, the storage spaces of the first blocking queue and the second blocking queue are equal.

[0079] In one embodiment of the present invention, the first blocking queue and / or the second blocking queue is a Concurrent Linked Queue chain queue.

[0080] According to an embodiment of the present invention, a data cache processing device employs two blocking queues within a local cache and switches between them using a pointer. Upon receiving data to be processed, the data is first written to the first blocking queue to which the pointer currently points. When the data in the first blocking queue meets a preset condition, the target thread processes the data, while the pointer is directed to the second blocking queue, allowing the second blocking queue to continue storing subsequent data to be processed. During this process, due to the switching between the two blocking queues, data can be copied from the first blocking queue to the target thread while the second blocking queue continues to write subsequent data, thereby reducing write latency and improving transmission speed and throughput. Furthermore, due to the use of blocking queues, the process of writing data to the blocking queues does not require locking, thereby further improving transmission speed and throughput by reducing the number of locks. Furthermore, thread confinement technology is employed to confine the data to be processed within the target thread, so that the process of writing data to the distributed cache also does not require locking, thereby further reducing the number of locks, ensuring local cache performance and improving transmission speed and throughput.

[0081] Figure 5 An exemplary system architecture 500 is shown to which the data cache processing method or data cache processing device according to the embodiment of the present invention can be applied.

[0082] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0083] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0084] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0085] Server 505 may be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 501, 502, and 503. The backend management server may analyze and process received data such as product information query requests, and feed back the processing results (e.g., target push information, product information) to the terminal device.

[0086] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0087] Reference below Figure 6 , which shows a schematic structural diagram of a computer system 600 of a terminal device suitable for implementing an embodiment of the present invention. Figure 6 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0088] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the system 600 are also stored in the RAM 603. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0089] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed into the storage section 608 as needed.

[0090] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are performed.

[0091] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, they may be described as: a processor including a data writing module, a data copying module, and a switching module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the data writing module may also be described as a "module for writing data to be processed into a first blocking queue."

[0094] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining data to be processed; determining the first blocking queue of the two blocking queues currently pointed to by the pointer, and writing the data to be processed into the first blocking queue; wherein the local cache includes two blocking queues and a pointer for indicating the switching of the two blocking queues; when the data to be processed in the first blocking queue meets a preset condition, the data to be processed in the first blocking queue is copied to the target thread, so that the target thread can be used to process the data to be processed; the pointer is pointed to the second blocking queue of the two blocking queues, so that the second blocking queue can continue to write the data to be processed.

[0095] According to the technical solution of an embodiment of the present invention, two blocking queues are set in the local cache, and the pointer is used to switch between the two blocking queues. When the data to be processed is received, the data to be processed is first written to the first blocking queue currently pointed to by the pointer; when the data in the first blocking queue meets the preset conditions, the data can be processed by the target thread, and at the same time, the pointer is pointed to the second blocking queue to continue to store subsequent data to be processed using the second blocking queue. In this process, due to the use of the two blocking queue switching method, the data can be copied to the target thread in the first blocking queue while the subsequent data is continued to be written using the second blocking queue, thereby reducing the waiting time for writing and thus improving the transmission speed and throughput. In addition, due to the use of blocking queues, the process of writing data to the blocking queue does not require locking, thereby further improving the transmission speed and throughput by reducing the number of locks. Furthermore, the thread enclosing technology is used to enclose the data to be processed inside the target thread, so the process of writing data to the distributed cache does not require locking, thereby further reducing the number of locks to ensure the performance of the local cache and improve the transmission speed and throughput.

[0096] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A data cache processing method, characterized in that: Applied to a local cache, the local cache includes two blocking queues and a pointer for indicating switching between the two blocking queues; including: Get the data to be processed; Determine a first blocking queue of the two blocking queues currently pointed to by the pointer, and write the to-be-processed data into the first blocking queue; When the data to be processed in the first blocking queue meets a preset condition, copying the data to be processed in the first blocking queue to a target thread, so as to process the data to be processed by the target thread; Pointing the pointer to the second blocking queue of the two blocking queues, so as to continue writing the data to be processed using the second blocking queue; Wherein, no lock is applied during the process of writing the to-be-processed data into the first blocking queue and the second blocking queue.

2. The method according to claim 1, characterized in that Also includes: The pointer is locked while pointing from the first blocking queue to the second blocking queue, and the lock is released after the pointer points to the second blocking queue.

3. The method according to claim 1, characterized in that The processing of the to-be-processed data by using the target thread includes: The thread enclosing technology is adopted to enclose the data to be processed inside the target thread, so that the target thread is used to write the data to be processed into the distributed cache.

4. The method according to claim 3, characterized in that After the target thread is used to write the data to be processed into a distributed cache, the method further includes: Release the data to be processed in the target thread.

5. The method according to claim 1, wherein Also includes: Use callback method to write duration threshold or storage space threshold; The preset conditions include: the storage amount of the data to be processed is not less than the storage space threshold, or the time length for writing the data to be processed into the first blocking queue is not less than the time length threshold.

6. The method according to claim 1, characterized in that The storage spaces of the first blocking queue and the second blocking queue are equal.

7. The method according to any one of claims 1 to 6, characterized in that: The first blocking queue and / or the second blocking queue is a Concurrent Linked Queue chain queue.

8. A data cache processing device, characterized in that: include: Data writing module, data replication module and switching module; wherein, The data writing module is configured to obtain data to be processed, determine a first blocking queue of two blocking queues currently pointed to by a pointer in a local cache, and write the data to be processed into the first blocking queue; wherein the local cache includes: the two blocking queues, and a pointer for indicating a switch between the two blocking queues; The data copy module is configured to copy the data to be processed in the first blocking queue to a target thread when the data to be processed in the first blocking queue meets a preset condition, so as to process the data to be processed by the target thread; The switching module is configured to point the pointer to the second blocking queue of the two blocking queues, so as to continue writing the data to be processed using the second blocking queue; Wherein, no lock is applied during the process of writing the to-be-processed data into the first blocking queue and the second blocking queue.

9. An electronic device for processing data, characterized in that include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images