Data processing method, device, equipment, storage medium and product
By introducing shared cache services into the microservice architecture, the data inconsistency problem caused by the non-supported cross-machine communication in a stand-alone device is solved, and the accuracy of data processing and storage space are improved.
Patent Information
- Application Number
- CN202111472505.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-03
- Publication Date
- 2025-03-28
- Estimated Expiration
- 2041-12-03
AI Technical Summary
In stand-alone devices, inter-process communication does not support cross-machine communication, resulting in inconsistent data in bitmap instances stored on each stand-alone device, which in turn leads to data processing errors and has low accuracy.
It adopts a microservice architecture, including storage services and shared cache services, and processes data requests from the service party. If it is a read operation, it directly performs read operations in the shared cache service; if it is a write operation, first performs write operations in the storage service, and then performs synchronous processing in the shared cache service to ensure data consistency.
Through the shared cache service, the waste of creating cache instances independently by each stand-alone device is avoided, storage space is saved, data processing accuracy is ensured, and errors caused by data inconsistency are avoided.
Smart Images

Figure CN114217986B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art
[0002] At present, indexing technology is usually used to query massive data. It can be understood that indexing technology is to quickly find data by using the storage form of key-value pairs. Among them, bitmap is a storage form of key-value pairs, which saves storage space by sharing one value element with multiple key elements.
[0003] In the related art, a process (usually including multiple threads) is run on a stand-alone device. The process running on the stand-alone device will create a bitmap instance within the process, that is, one process has one bitmap instance. Since inter-process communication does not support cross-machine communication, the files stored in the bitmap instance on each stand-alone device will face the phenomenon of being out of sync, which will lead to a series of problems such as data processing errors.
[0004] It can be seen that how to improve the accuracy of data processing is an urgent problem to be solved. Summary of the invention
[0005] To solve the above technical problems, the embodiments of the present application provide a data processing method, an apparatus, an electronic device, a computer-readable storage medium and a computer program product, which can improve the accuracy of data processing at least to a certain extent.
[0006] According to one aspect of an embodiment of the present application, a data processing method is provided, which is applied to a microservice, wherein the microservice includes a storage service and a shared cache service, and the method includes: receiving data processing requests respectively initiated by multiple business parties, wherein the data processing requests include read operation requests or write operation requests; in response to at least one read operation request received, if it is determined that the latest version of the atomic variable is cached in the shared cache service, performing a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request; in response to at least one write operation request received, performing write operations on the atomic variables stored in the storage service in sequence, and performing synchronization processing on the same atomic variable cached in the shared cache service.
[0007] According to one aspect of an embodiment of the present application, there is provided a data processing device, which is applied to a microservice, wherein the microservice includes a storage service and a shared cache service, and the device includes: a receiving module, configured to receive data processing requests respectively initiated by multiple business parties, wherein the data processing requests include read operation requests or write operation requests; a reading module, configured to respond to at least one received read operation request, and if it is determined that the latest version of the atomic variable is cached in the shared cache service, perform a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request; a writing and synchronization module, configured to respond to at least one received write operation request, perform write operations on the atomic variables stored in the storage service in sequence, and perform synchronization processing on the same atomic variable cached in the shared cache service.
[0008] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the electronic device, the electronic device implements the data processing method as described above.
[0009] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data processing method described above is implemented.
[0010] According to one aspect of an embodiment of the present application, a computer program product is provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the data processing method described above is implemented.
[0011] In the technical solution provided by the embodiments of the present application, on the one hand, since a shared cache service is created in the microservice, each stand-alone device no longer needs to create a cache instance separately, thereby saving storage space to a great extent; on the other hand, when a write operation request initiated by the business party is received, after performing a write operation on the atomic variable stored in the storage service according to the write operation request, synchronization processing is performed on the same atomic variable cached in the shared cache service, so that the atomic variables cached in the shared cache service are consistent with the atomic variables stored in the storage service. When the business party initiates a read operation request, it only needs to perform a read operation on the latest version of the atomic variables cached in the shared cache service, that is, each business party reads the latest version of the atomic variables cached in the shared cache service, thereby ensuring that the atomic variables read by each business party are consistent, and avoiding the phenomenon of low data processing accuracy caused by inconsistency of atomic variables obtained from the cache instances created by each stand-alone device.
[0012] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0014] Figure 1 It is a schematic diagram of data processing in the related art;
[0015] Figure 2 is a schematic diagram of an exemplary implementation environment to which the technical solution of the embodiments of the present application can be applied;
[0016] Figure 3 is a schematic diagram of an exemplary microservice architecture to which the technical solution of the embodiments of the present application can be applied;
[0017] Figure 4 is a flow chart of a data processing method shown in an exemplary embodiment of the present application;
[0018] Figure 5 is a flow chart of a data processing method shown in an exemplary embodiment of the present application;
[0019] Figure 6 is a flow chart of a data processing method shown in an exemplary embodiment of the present application;
[0020] Figure 7 is a schematic diagram of reading data shown in an exemplary embodiment of the present application;
[0021] Figure 8 yes Figure 4 A flowchart of step S403 in the illustrated embodiment in an exemplary embodiment;
[0022] Fig. 9 yes Figure 8 A flowchart of step S802 in the illustrated embodiment in an exemplary embodiment;
[0023] Fig.10 yes Figure 4 A flowchart of step S403 in the illustrated embodiment in an exemplary embodiment;
[0024] Fig.11 is a schematic diagram of writing data shown in an exemplary embodiment of the present application;
[0025] Fig.12 is a flow chart of a data processing method shown in an exemplary embodiment of the present application;
[0026] Fig.13 is a schematic diagram of reading data and writing data shown in an exemplary embodiment of the present application;
[0027] Fig.14 is a block diagram of a data processing device shown in an exemplary embodiment of the present application;
[0028] Fig.15 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION
[0029] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments that are the same as the present application. Instead, they are only examples of devices and methods that are the same as some aspects of the present application as detailed in the attached claims.
[0030] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0031] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0032] It should be noted that the "multiple" mentioned in this application refers to two or more than two. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship.
[0033] Before introducing the technical solutions of the embodiments of the present application, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.
[0034] Microservices: Divide an application into multiple independent small services. Each service focuses on its own independent function and runs in its own independent process. There are relatively clear boundaries between services. Services communicate with each other through lightweight communication mechanisms, such as remote procedure call (RPC) communication mechanism. Among them, microservices are very suitable for current Internet applications with frequent demand changes, fast iteration speed and short delivery cycle.
[0035] Bitmap: uses a bit to mark the value corresponding to an element, and the key is the element. Since it uses bits as units to store data, it can greatly save storage space. For example, suppose there is a need to find out whether a number m exists in 2 billion random integers. At the same time, suppose there is a 32-bit operating system and 4G memory. In Java, int occupies 4 bytes, and 1 byte = 8 bits. Therefore, if each number is stored as int, it will be 2 billion ints, so the space occupied is about (2000000000*4 / 1024 / 1024 / 1024)≈7.45G. If it is stored in bits, 2 billion numbers are 2 billion bits, and the space occupied is about (2000000000 / 8 / 1024 / 1024 / 1024)≈0.233G. It can be seen that bitmap can save storage space to a great extent.
[0036] Atomic variables: Shared data is called atomic variables, which corresponds to a mechanism to solve the security problems caused by multiple threads operating shared data in concurrent situations. That is, if one thread modifies the atomic variable, the other threads can see it immediately. Its function is similar to the lock mechanism.
[0037] Paxos: is a consensus algorithm based on message passing. The problem it solves is how a distributed system can reach a consensus on a certain value (resolution). A typical scenario is that in a distributed database system, if the initial state of each node is consistent and each node executes the same sequence of operations, then they can finally get a consistent state. To ensure that each node executes the same sequence of commands, a "consistency algorithm" needs to be executed on each instruction to ensure that the instructions seen by each node are consistent. The Paxos algorithm is applicable to several situations: multiple processes / threads in a single device reach data consistency; multiple clients concurrently read and write data in a distributed file system or distributed database; the consistency of multiple replicas in distributed storage responding to read and write requests.
[0038] Raft: is a consensus algorithm that solves the same problem of how a distributed system can reach a consensus on a certain value (resolution). Compared with the Paxos algorithm, the Raft algorithm decomposes a large number of computational problems into some simple and relatively independent sub-problems.
[0039] In the related art, a process (usually including multiple threads) runs on a stand-alone device. The process running on the stand-alone device will create a bitmap instance in the process, where the bitmap instance is used to copy the data in the storage service and cache it in the bitmap instance so that the data in the bitmap instance can be used for processing when there is a business demand. Since a process has a bitmap instance, that is, each stand-alone device corresponds to a bitmap instance, which wastes storage space to a great extent; and since inter-process communication does not support cross-machine communication, the data cached in the bitmap instance on each stand-alone device will face inconsistency / synchronization, which will lead to a series of problems such as data processing errors, and the accuracy of data processing is low.
[0040] See also Figure 1 , process 1 corresponding to stand-alone device 1 creates a bitmap_1 instance, where when there is business demand 1, cached data (i.e., data in the buffer shown in the figure, where buffer represents storage, the same below) is obtained from the bitmap_1 instance for processing; process n corresponding to stand-alone device n creates a bitmap_n instance, where when there is business demand n, cached data is obtained from the bitmap_n instance for processing. It can be seen that each stand-alone device corresponds to a bitmap instance, which wastes storage space to a great extent.
[0041] Please refer to Figure 1 , process 1 modifies the data in the storage service, and the data copied to the bitmap_1 instance is the latest, but because cross-machine communication is not supported, the data copied to the bitmap_n instance by process n is still the data before the modification, that is, not the latest data. Therefore, when there is a business demand n, the non-latest data is used for processing. It can be seen that each single-machine device corresponds to a bitmap instance, and the cached data in different bitmap instances are inconsistent / synchronous, which easily causes a series of problems such as data processing errors, and the accuracy of data processing is low.
[0042] Based on this, the embodiments of the present application propose a data processing method, device, electronic device, computer-readable storage medium and computer program product, which optimize the data processing scheme, greatly improve the accuracy of data processing, and save storage space.
[0043] See also Figure 2 , Figure 2 It is a schematic diagram of an exemplary implementation environment of the present application; the implementation environment includes a terminal device 201 and a server 202, wherein the terminal device 201 and the server 202 communicate with each other via a wired or wireless network.
[0044] It should be noted that microservices are deployed on the server 202, and the microservices include storage services and shared cache services. The storage service is used to store data, which is deployed in the database and can be simply understood as being equivalent to a disk; the shared cache service is used to cache data obtained from the storage service, which is deployed in the memory.
[0045] In one embodiment of the present application, see Figure 3 , is a schematic diagram of the architecture of an example microservice. Among them, a creation request can be received, and then a bitmap shared cache service can be created according to the received creation request. The bitmap shared cache service can interact with the storage service. Specifically, the bitmap shared cache service can obtain data from the storage service for caching. Therefore, when a data processing request sent by the business party according to the demand is received, the cached data can be directly obtained from the bitmap shared cache service for processing, thereby improving the efficiency of data processing compared to obtaining the stored data from the storage service for processing.
[0046] In one embodiment of the present application, the storage service may be a Paxos-based storage service.
[0047] In one embodiment of the present application, the storage service may be a Raft-based storage service.
[0048] The terminal device 201 may be a smart phone, a tablet, a laptop, a computer, a vehicle-mounted terminal or other electronic device, which is not limited herein. At the same time, the number of the terminal device 201 may be one or more.
[0049] The server 202 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, etc., which are not limited here. At the same time, the number of servers 202 may be one or more.
[0050] In one embodiment of the present application, an application is installed in the terminal device 201, which can be used by the user, that is, the terminal device 201, as a business party, can send a data processing request to the server 202 according to demand, wherein the data processing request includes a read operation request or a write operation request; accordingly, if the server 202 receives at least one read operation request, the server 202 responds to the at least one read operation request received, and when it is determined that the latest version of the atomic variable is cached in the shared cache service, the server 202 performs a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request; if the server 202 receives at least one write operation request, the server 202 responds to the at least one write operation request received, performs write operations on the atomic variables stored in the storage service in sequence, and performs synchronization processing on the same atomic variable cached in the shared cache service.
[0051] based on Figure 2-3 The application scenario shown, after adopting the technical solution of the embodiment of the present application, improves the accuracy of data processing and saves storage space.
[0052] The following is a detailed description of various implementation details of the technical solution of the embodiment of the present application:
[0053] See also Figure 4 , Figure 4 is a flow chart of a data processing method shown in an embodiment of the present application. The data processing method may be Figure 2 The server 202 is shown to execute, Figure 4 As shown, the data processing method at least includes steps S401 to S403, which are described in detail as follows:
[0054] Step S401: receiving data processing requests respectively initiated by multiple business parties, where the data processing requests include read operation requests or write operation requests.
[0055] In the embodiment of the present application, a read operation request is an instruction initiated by a business party to instruct a read operation to be performed on data in a cache service; wherein the read operation request corresponds to a read operation program. Optionally, the business party may initiate a read operation request by triggering a thread for reading data, and running the thread to perform a read operation on the data in the cache service to realize data reading.
[0056] In the embodiment of the present application, a write operation request is an instruction initiated by the business party to instruct the execution of a write operation on the data in the storage service; wherein the write operation request corresponds to a write operation program, and the write operation program includes but is not limited to a process and a thread. Optionally, the business party may initiate a write operation request by triggering a thread for writing data, and running the thread to execute a write operation on the data in the storage service to implement data writing.
[0057] In one embodiment of the present application, the received data processing requests respectively initiated by multiple business parties may all be read operation requests.
[0058] In one embodiment of the present application, the received data processing requests respectively initiated by multiple business parties may all be write operation requests.
[0059] In one embodiment of the present application, some of the received data processing requests respectively initiated by multiple business parties may be read operation requests, and some may be write operation requests.
[0060] Step S402, in response to at least one received read operation request, if it is determined that the latest version of the atomic variable is cached in the shared cache service, a read operation is performed on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request.
[0061] In an embodiment of the present application, if at least one read operation request is received, it is necessary to respond to the at least one read operation request received, and if it is determined that the latest version of the atomic variable is cached in the shared cache service, then a read operation can be performed on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request.
[0062] In the embodiment of the present application, since multiple business parties may initiate write operation requests concurrently, in order to ensure the security of the data stored in the storage service, an atomic variable mechanism can be adopted, and the data stored in the storage service is called atomic variables. Correspondingly, the data cached in the shared cache service is also called atomic variables.
[0063] In one embodiment of the present application, see Figure 5 Before the process of performing a read operation on the latest version of the atomic variable cached in the shared cache service according to at least one read operation request in step S402, the following steps may also be included:
[0064] Step S501, comparing the version corresponding to the atomic variable stored in the storage service with the version corresponding to the same atomic variable cached in the shared cache service;
[0065] Step S502, if the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, then the atomic variable stored in the storage service is used to replace the same atomic variable cached in the shared cache service, and the version corresponding to the replaced atomic variable is updated so that the atomic variable cached in the shared cache service is the latest version.
[0066] That is, in an optional embodiment, it can be determined whether the atomic variable cached in the shared cache service is the latest version of the atomic variable. Specifically, the version corresponding to the atomic variable stored in the storage service can be compared with the version corresponding to the same atomic variable cached in the shared cache service; wherein:
[0067] In the first case, if the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, it means that the atomic variable cached in the shared cache service at this time is not the latest version of the atomic variable; therefore, it is necessary to replace the same atomic variable cached in the shared cache service with the atomic variable stored in the storage service so that the atomic variable cached in the shared cache service is the latest version, and at the same time, update the version corresponding to the replaced atomic variable to identify the version to which the replaced atomic variable belongs, so as to facilitate comparison based on the version corresponding to the atomic variable when necessary.
[0068] In the second case, if the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is equal to the version corresponding to the same atomic variable cached in the shared cache service, it means that the atomic variable cached in the shared cache service at this time is the latest version of the atomic variable; therefore, no operation is required.
[0069] In the third case, if the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is lower than the version corresponding to the same atomic variable cached in the shared cache service, it means that the atomic variable cached in the shared cache service is now the latest version of the atomic variable; therefore, it is necessary to replace the same atomic variable stored in the storage service with the atomic variable cached in the shared cache service, and at the same time update the version corresponding to the replaced atomic variable to identify the version to which the replaced atomic variable belongs, so as to facilitate comparison based on the version corresponding to the atomic variable when necessary.
[0070] It can be understood that since the response to the write operation request is to perform a write operation on the atomic variables stored in the storage service, the atomic variables stored in the storage service are usually the latest version, that is, usually it belongs to the first case or the second case, and the probability of the third case occurring is relatively small.
[0071] In one embodiment of the present application, the number of received read operation requests is at least two; see Figure 6 Before the process of comparing the version corresponding to the atomic variable stored in the storage service with the version corresponding to the same atomic variable cached in the shared cache service in step S501, the following steps may also be included:
[0072] Step S601, forwarding a read operation request to a storage service, and obtaining a version corresponding to the stored atomic variable from the storage service;
[0073] Step S602: forward all other read operation requests to the shared cache service, and obtain the cached version corresponding to the same atomic variable from the shared cache service.
[0074] That is, in an optional embodiment, when at least two read operation requests are received, one read operation request can be forwarded to the storage service, and the version corresponding to the stored atomic variable can be obtained from the storage service, wherein the read operation request forwarded to the storage service is used to perform read operations on the atomic variables stored in the storage service; and all other read operation requests are forwarded to the shared cache service, and the version corresponding to the cached atomic variable is obtained from the shared cache service, wherein all other read operation requests forwarded to the shared service are used to perform read operations on the atomic variables cached in the shared cache service.
[0075] For example, suppose that there are 100 read operation requests received. At this time, 1 read operation request is forwarded to the storage service, and the version corresponding to the stored atomic variable is obtained from the storage service; and 99 read operation requests are forwarded to the shared cache service, and the version corresponding to the cached atomic variable is obtained from the shared cache service.
[0076] In this way, the read operation programs corresponding to multiple read operation requests concurrently access the atomic variables cached in the shared cache service, while only one read operation program corresponding to a read operation request accesses the atomic variables stored in the storage service. Since the rate of executing read operations on the atomic variables cached in the shared cache service is much higher than the rate of executing read operations on the atomic variables stored in the storage service, high concurrent reading is achieved, which improves the data reading efficiency and thus improves the data processing efficiency.
[0077] It should be noted that step S601 and step S602 can be executed in any order or in parallel. In practical applications, they can be flexibly adjusted according to specific application scenarios.
[0078] In order to better understand the data reading process, an example is given here, where:
[0079] See also Figure 7, which is a schematic diagram of an example of reading data. Specifically, first, the version corresponding to the stored atomic variable is obtained from the storage service, and the version corresponding to the same atomic variable cached is obtained from the shared cache service; then, the version corresponding to the atomic variable stored in the storage service is compared with the version corresponding to the same atomic variable cached in the shared cache service. If the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, the same atomic variable cached in the shared cache service is replaced with the atomic variable stored in the storage service, and the version corresponding to the replaced atomic variable is updated so that the atomic variable cached in the shared cache service is the latest version; then, the latest version of the atomic variable cached in the shared cache service is returned to the corresponding business party that initiated the read operation request.
[0080] Step S403, in response to at least one received write operation request, sequentially perform write operations on the atomic variables stored in the storage service, and perform synchronization processing on the same atomic variable cached in the shared cache service.
[0081] In an embodiment of the present application, if at least one write operation request is received, it is necessary to respond to the at least one write operation request received, and perform write operations on the atomic variables stored in the storage service in turn, and perform synchronization processing on the same atomic variable cached in the shared cache service.
[0082] In one embodiment of the present application, the received write operation requests are at least two; see Figure 8 The process of sequentially performing write operations on the atomic variables stored in the storage service in step S403 may include the following steps:
[0083] Step S801, storing the write operation program corresponding to each write operation request in a queue;
[0084] Step S802: Take out write operation programs from the queue in sequence to perform write operations on the atomic variables stored in the storage service.
[0085] That is, in an optional embodiment, when at least two write operation requests are received, the write operation programs corresponding to each write operation request can be stored in a queue, i.e., enqueued for storage, and then the write operation programs are taken out of the queue in turn to perform write operations on the atomic variables stored in the storage service.
[0086] It is understandable that, since the queue is a first-in, first-out mechanism, the write operation program of the first queue will first take out and write to the atomic variables stored in the storage service, and the write operation program of the last queue will take out and write to the atomic variables stored in the storage service later.
[0087] In this way, through the queue mechanism, while ensuring the security of the data stored in the storage service, each write operation program can be executed in sequence, that is, the write operation program corresponding to the write operation request received first is executed first, and the write operation program corresponding to the write operation request received later is executed later, avoiding a series of problems such as data processing errors caused by errors in the order of writing operations executed by the write operation program.
[0088] In one embodiment of the present application, each write operation program in the queue is set to a blocking state; see Fig. 9 The process of sequentially taking out the write operation programs from the queue in step S802 to perform write operations on the atomic variables stored in the storage service may include the following steps:
[0089] Step S901, taking out the write operation program at the head of the queue from the queue as the target write operation program, and switching the state of the target write operation program from the blocking state to the running state;
[0090] Step S902, write operations are performed on the atomic variables stored in the storage service through the target write operation program in the running state, and after the target write operation program in the running state completes the write operation on the atomic variables stored in the storage service, the write operation program at the head of the queue is taken out from the queue as the target write operation program to perform write operations on the atomic variables stored in the storage service, until all the write operation programs in the queue are taken out.
[0091] That is, in an optional embodiment, the write operation program at the head of the queue is taken out as the target write operation program, and the state of the target write operation program is switched from a blocking state to a running state, and then the target write operation program is used to perform write operations on the atomic variables stored in the storage service. After the target write operation program completes the write operation on the atomic variables stored in the storage service, the write operation program at the head of the queue can be taken out again as the target write operation program to perform write operations on the atomic variables stored in the storage service, until all the write operation programs in the queue are taken out, thereby realizing the execution of write operations on the atomic variables stored in the storage service by the write operation programs corresponding to all write operation requests.
[0092] Among them, in the optional embodiment, the target write operation program is the write operation program of the current write operation to be executed, so the state of the target write operation program needs to be switched from the blocking state to the running state.
[0093] For example, suppose the write operation programs from the head to the tail of the queue are a1, a2, a3, a4, and a5 respectively; specifically, firstly, take out the write operation program a1 from the queue as the target write operation program, and switch the state of the write operation program a1 from the blocking state to the running state, and execute the write operation on the atomic variables stored in the storage service through the write operation program a1. After the write operation program a1 completes the write operation on the atomic variables stored in the storage service, then take out the write operation program a2 from the queue as the target write operation program, and so on, until the write operation programs a1, a2, a3, a4, and a5 are taken out, and all of them execute the write operation on the atomic variables stored in the storage service through the write operation programs a1, a2, a3, a4, and a5.
[0094] It can be understood that in the embodiment of the present application, after each write operation is performed on the atomic variables stored in the storage service, the atomic variables after the write operation can be obtained, and at this time, the version corresponding to the atomic variables after the write operation needs to be updated so that when necessary, comparison can be performed according to the version corresponding to the atomic variables.
[0095] In one embodiment of the present application, each write operation program in the queue is set to a blocking state; see Fig.10 The process of performing synchronization processing on the same atomic variable cached in the shared cache service in step S403 may include the following steps:
[0096] Step S1001, obtaining the atomic variables after the write operation in the storage service, and the versions corresponding to the atomic variables after the write operation;
[0097] Step S1002: Replace the same atomic variable cached in the shared cache service with the atomic variable after the write operation in the storage service, and update the version corresponding to the replaced atomic variable to the version corresponding to the atomic variable after the write operation.
[0098] That is, in an optional embodiment, the atomic variables after the write operation in the storage service and the version corresponding to the atomic variables after the write operation can be obtained, and then the atomic variables after the write operation in the storage service can be used to replace the same atomic variables cached in the shared cache service, and the version corresponding to the replaced atomic variables can be updated to the version corresponding to the atomic variables after the write operation.
[0099] Among them, the atomic variable after the write operation in the storage service obtained in the optional embodiment can be the atomic variable after the last write operation program in the queue performs the write operation, and accordingly, the version corresponding to the atomic variable after the write operation obtained can be the version corresponding to the atomic variable after the last write operation program in the queue performs the write operation. In this way, after all write operation programs have completed the write operation, the same atomic variable cached in the shared cache service is replaced according to the latest version of the atomic variable stored in the storage service, and the version corresponding to the replaced atomic variable is updated to the version corresponding to the latest version of the atomic variable, thereby improving the efficiency of synchronization processing.
[0100] Among them, the atomic variables after the write operation in the storage service obtained in the optional embodiment can be the atomic variables after each write operation program in the queue performs the write operation, and accordingly, the corresponding version of the atomic variables after the write operation obtained can be the corresponding version of the atomic variables after each write operation program in the queue performs the write operation. In this way, after each write operation program completes the write operation, the same atomic variable cached in the shared cache service is replaced according to the latest version of the atomic variable stored in the storage service, and the version corresponding to the replaced atomic variable is updated to the version corresponding to the latest version of the atomic variable, thereby improving the accuracy of the synchronization processing.
[0101] To better understand the data writing process, an example is given here, where:
[0102] See also Fig.11 , which is a schematic diagram of an example of writing data. Specifically, firstly, a write operation is performed on the atomic variable stored in the storage service to obtain the atomic variable after the write operation, and the version corresponding to the atomic variable after the write operation is updated at the same time; then, the atomic variable after the write operation in the storage service and the version corresponding to the atomic variable after the write operation are obtained, and at this time, the atomic variable after the write operation in the storage service is used to replace the same atomic variable cached in the shared cache service, and the version corresponding to the replaced atomic variable is updated to the version corresponding to the atomic variable after the write operation, and at this time, the same atomic variable cached in the shared cache service is the latest version.
[0103] In the embodiment of the present application, the microservice includes a storage service and a shared cache service. Not only does each stand-alone device no longer need to create a cache instance separately, thus saving storage space; but after performing a write operation on the atomic variable stored in the storage service according to the write operation request, since the same atomic variable cached in the shared cache service is synchronized, the atomic variables cached in the shared cache service are consistent with the atomic variables stored in the storage service. When the business party initiates a read operation request, the read operation is performed on the latest version of the atomic variable cached in the shared cache service, thereby ensuring that the atomic variables read by each business party are consistent, thereby improving the accuracy of data processing.
[0104] The following is a detailed description of a specific application scenario of the embodiment of the present application:
[0105] See also Fig.12 , is a flow chart of an exemplary data processing method; the data processing method may include the following steps, which are described in detail as follows:
[0106] Step S1201: receiving data processing requests respectively initiated by multiple business parties, where the data processing requests include read operation requests or write operation requests.
[0107] Optionally, some of the multiple business parties initiate read operation requests, and some of the business parties initiate write operation requests.
[0108] First, the data reading process is introduced, including:
[0109] Step S1202: In response to the received multiple read operation requests, the version corresponding to the atomic variable stored in the storage service is compared with the version corresponding to the same atomic variable cached in the shared cache service.
[0110] Optionally, in response to multiple read operation requests received, the version corresponding to the stored atomic variable can be obtained from the storage service, and the version corresponding to the same atomic variable cached can be obtained from the shared cache service, and then the version corresponding to the atomic variable stored in the storage service is compared with the version corresponding to the same atomic variable cached in the shared cache service to determine whether the same atomic variable cached in the shared cache service is the latest version.
[0111] Step S1203, if the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, then the atomic variable stored in the storage service is used to replace the same atomic variable cached in the shared cache service, and the version corresponding to the replaced atomic variable is updated so that the atomic variable cached in the shared cache service is the latest version.
[0112] Optionally, if the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, it means that the atomic variable cached in the shared cache service at this time is not the latest version of the atomic variable; therefore, it is necessary to replace the same atomic variable cached in the shared cache service with the atomic variable stored in the storage service so that the atomic variable cached in the shared cache service is the latest version, and at the same time, the version corresponding to the replaced atomic variable is updated to identify the version to which the replaced atomic variable belongs, so as to facilitate comparison based on the version corresponding to the atomic variable when necessary.
[0113] Step S1204: performing a read operation on the latest version of the atomic variable cached in the shared cache service according to the multiple read operation requests.
[0114] Optionally, the read operation program corresponding to each read operation request may simultaneously perform a read operation on the latest version of the atomic variable cached in the shared cache service.
[0115] Next, the data reading process is introduced, including:
[0116] Step S1205 , in response to the received multiple write operation requests, storing the write operation program corresponding to each write operation request in a queue.
[0117] Optionally, considering the security of atomic variables stored in the storage service, the write operation programs corresponding to each write operation request cannot perform write operations on the atomic variables stored in the storage service at the same time. Therefore, the write operation programs corresponding to each write operation request can be stored in a queue at this time.
[0118] Step S1206, taking out the write operation programs from the queue in sequence to perform write operations on the atomic variables stored in the storage service.
[0119] Optionally, each write operation program in the queue is set to a blocking state, the write operation program at the head of the queue is taken out from the queue as the target write operation program, and the state of the target write operation program is switched from the blocking state to the running state; thereafter, the atomic variables stored in the storage service are written by the target write operation program in the running state, and after the target write operation program in the running state completes the write operation on the atomic variables stored in the storage service, the write operation program at the head of the queue is taken out from the queue as the target write operation program to write to the atomic variables stored in the storage service, until all the write operation programs in the queue are taken out.
[0120] Among them, after each write operation is performed on the atomic variables stored in the storage service, the atomic variables after the write operation will be obtained, and the version corresponding to the atomic variables after the write operation will be updated at the same time.
[0121] Step S1207, obtaining the atomic variables after the write operation in the storage service, and the versions corresponding to the atomic variables after the write operation.
[0122] Optionally, obtain the atomic variable after the last write operation program in the queue executes the write operation, that is, the latest version of the atomic variable; and obtain the version corresponding to the atomic variable after the last write operation program in the queue executes the write operation, that is, the version corresponding to the latest version of the atomic variable.
[0123] Step S1208, replace the same atomic variable cached in the shared cache service with the atomic variable after the write operation in the storage service, and update the version corresponding to the replaced atomic variable to the version corresponding to the atomic variable after the write operation.
[0124] Optionally, the atomic variable cached in the shared cache service is replaced with the latest version of the atomic variable in the storage service, and the version corresponding to the replaced atomic variable is updated to the version corresponding to the latest version of the atomic variable.
[0125] It should be noted that, according to the business logic of different application scenarios, if the read operation request precedes the write operation request, it is necessary to execute the read operation procedure of the read operation request first, and then execute the write operation procedure of the write operation request, that is, first execute steps S1202-step S1204, and then execute steps S1205-step S1208; if the write operation request precedes the read operation request, it is necessary to execute the write operation procedure of the write operation request first, and then execute the read operation procedure of the read operation request, that is, first execute steps S1205-step S1208, and then execute steps S1202-step S1204.
[0126] See also Fig.13 , which is a schematic diagram of an exemplary data processing method. For example, suppose that there are three data processing requests received, of which one write operation request corresponds to a write operation program (i.e., thread 1), and two read operation requests correspond to read operation programs (i.e., thread 2 and thread 3), respectively. Thread 1, thread 2, and thread 3 can all call the compare exchange function to perform the compare exchange operation on the atomic variable; at the same time, suppose that according to the business logic, thread 1 must be executed before thread 2 and thread 3, so at this time, thread 1 remains in the running state, and thread 2 and thread 3 are set to the blocked state. Specifically, thread 2 and thread 3 can be blocked on the condition variable (only when the condition is met, will the blocked state be switched to the running state); wherein:
[0127] Thread 1 in the running state performs a write operation on the atomic variable stored in the storage service, and after the write operation succeeds, performs synchronization processing on the same atomic variable cached in the shared cache service, that is, updates the same atomic variable cached in the shared cache service;
[0128] After updating the same atomic variable cached in the shared cache service, thread 2 and thread 3 are awakened to switch threads 2 and 3 from a blocked state to a running state. It can be understood that the completion of updating the same atomic variable cached in the shared cache service can be used as a condition for awakening threads 2 and 3; and then threads 2 and 3 in the running state respectively perform read operations on the latest version of the same atomic variable cached in the shared cache service.
[0129] In the embodiment of the present application, the microservice includes a storage service and a shared cache service. By comparing the version corresponding to the atomic variable stored in the storage service with the version corresponding to the same atomic variable cached in the shared cache service, the shared cache service is updated to ensure the consistency of the atomic variable; at the same time, when multiple concurrent threads access the shared cache service, only one thread will access the storage service, thereby achieving high concurrent reading.
[0130] Fig.14 FIG. 1 is a block diagram of a data processing device according to an embodiment of the present application. Fig.14 As shown, the data processing device is applied to microservices, the microservices include storage services and shared cache services, and the data processing device includes:
[0131] A receiving module 1401 is configured to receive data processing requests respectively initiated by multiple business parties, where the data processing requests include read operation requests or write operation requests;
[0132] The read module 1402 is configured to respond to the at least one read operation request received, and if it is determined that the latest version of the atomic variable is cached in the shared cache service, perform a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request;
[0133] The write and synchronization module 1403 is configured to perform write operations on the atomic variables stored in the storage service in sequence in response to at least one write operation request received, and perform synchronization processing on the same atomic variable cached in the shared cache service.
[0134] In the embodiment of the present application, the microservice includes a storage service and a shared cache service. Not only does each stand-alone device no longer need to create a cache instance separately, thus saving storage space; but after performing a write operation on the atomic variable stored in the storage service according to the write operation request, since the same atomic variable cached in the shared cache service is synchronized, the atomic variables cached in the shared cache service are consistent with the atomic variables stored in the storage service. When the business party initiates a read operation request, the read operation is performed on the latest version of the atomic variable cached in the shared cache service, thereby ensuring that the atomic variables read by each business party are consistent, thereby improving the accuracy of data processing.
[0135] In one embodiment of the present application, the data processing device further includes:
[0136] A comparison module, configured to compare a version corresponding to an atomic variable stored in the storage service with a version corresponding to the same atomic variable cached in the shared cache service;
[0137] The update module is configured to replace the same atomic variable cached in the shared cache service with the atomic variable stored in the storage service if the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, and update the version corresponding to the replaced atomic variable so that the atomic variable cached in the shared cache service is the latest version.
[0138] In one embodiment of the present application, at least two read operation requests are received; the data processing device further includes:
[0139] A first forwarding and obtaining module is configured to forward a read operation request to the storage service and obtain a version corresponding to the stored atomic variable from the storage service;
[0140] The second forwarding and obtaining module is configured to forward all other read operation requests to the shared cache service, and obtain the cached version corresponding to the same atomic variable from the shared cache service.
[0141] In one embodiment of the present application, after performing a write operation on an atomic variable stored in a storage service, the atomic variable after the write operation is obtained, and the version corresponding to the atomic variable after the write operation is updated; the write and synchronization module 1403 includes:
[0142] An acquisition unit, configured to acquire the atomic variables after the write operation in the storage service, and the versions corresponding to the atomic variables after the write operation;
[0143] The updating unit is configured to replace the same atomic variable cached in the shared cache service with the atomic variable after the write operation in the storage service, and update the version corresponding to the replaced atomic variable to the version corresponding to the atomic variable after the write operation.
[0144] In one embodiment of the present application, at least two write operation requests are received; the write and synchronization module 1403 includes:
[0145] A storage unit configured to store the write operation program corresponding to each write operation request in a queue;
[0146] The fetching unit is configured to fetch the write operation programs from the queue in sequence to perform write operations on the atomic variables stored in the storage service.
[0147] In one embodiment of the present application, each write operation program in the queue is set to a blocking state; the fetching unit includes:
[0148] A switching unit is configured to take out a write operation program at the head of the queue from the queue as a target write operation program, and switch the state of the target write operation program from a blocking state to a running state;
[0149] The write unit is configured to perform write operations on the atomic variables stored in the storage service through a target write operation program in a running state, and after the target write operation program in a running state completes the write operation on the atomic variables stored in the storage service, take out the write operation program at the head of the queue as the target write operation program to perform write operations on the atomic variables stored in the storage service, until all the write operation programs in the queue are taken out.
[0150] In one embodiment of the present application, the data processing device further includes:
[0151] The creation module is configured to create a shared cache service according to the received creation request; wherein the shared cache service is used to cache the atomic variables obtained from the storage service.
[0152] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here.
[0153] An embodiment of the present application also provides an electronic device, including one or more processors and a storage device, wherein the storage device is used to store one or more programs, and when the one or more programs are executed by the electronic device, the above data processing method is implemented.
[0154] Fig.15 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.
[0155] It should be noted that Fig.15 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0156] like Fig.15 As shown, the computer system includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1502 or the program loaded from the storage part 1508 to the random access memory (RAM) 1503, such as executing the method in the above embodiment. In RAM 1503, various programs and data required for system operation are also stored. CPU 1501, ROM 1502 and RAM 1503 are connected to each other through bus 1504. Input / output (I / O) interface 1505 is also connected to bus 1504.
[0157] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, etc.; an output section 1507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed so that a computer program read therefrom is installed into the storage section 1508 as needed.
[0158] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1509, and / or installed from a removable medium 1511. When the computer program is executed by a central processing unit (CPU) 1501, various functions defined in the system of the present application are executed.
[0159] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0160] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0161] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0162] Another aspect of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the data processing method as described above is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.
[0163] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data processing method provided in each of the above embodiments.
[0164] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.
Claims
1. A data processing method, characterized in that: Applied to a microservice, the microservice includes a storage service and a shared cache service, the storage service is deployed in a disk, and the shared cache service is deployed in a memory, the method includes: Receiving data processing requests respectively initiated by multiple business parties, wherein the data processing requests include read operation requests or write operation requests; In response to at least one received read operation request, if it is determined that the shared cache service has a latest version of the atomic variable cached, performing a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request; In response to at least one received write operation request, sequentially perform write operations on the atomic variables stored in the storage service, and perform synchronization processing on the same atomic variable cached in the shared cache service; Before performing a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request, the method further includes: Compare the version corresponding to the atomic variable stored in the storage service with the version corresponding to the same atomic variable cached in the shared cache service; If the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, the atomic variable stored in the storage service is used to replace the same atomic variable cached in the shared cache service, and the version corresponding to the replaced atomic variable is updated so that the atomic variable cached in the shared cache service is the latest version.
2. The method according to claim 1, characterized in that At least two read operation requests are received; before comparing the version corresponding to the atomic variable stored in the storage service with the version corresponding to the same atomic variable cached in the shared cache service, the method further includes: Forward a read operation request to the storage service, and obtain the version corresponding to the stored atomic variable from the storage service; All other read operation requests are forwarded to the shared cache service, and the cached version corresponding to the same atomic variable is obtained from the shared cache service.
3. The method according to claim 1, characterized in that After performing a write operation on the atomic variable stored in the storage service, obtaining the atomic variable after the write operation, and updating the version corresponding to the atomic variable after the write operation; performing synchronization processing on the same atomic variable cached in the shared cache service, including: Obtain the atomic variables after the write operation in the storage service, and the versions corresponding to the atomic variables after the write operation; The atomic variable after the write operation in the storage service is used to replace the same atomic variable cached in the shared cache service, and the version corresponding to the replaced atomic variable is updated to the version corresponding to the atomic variable after the write operation.
4. The method according to claim 1, characterized in that At least two write operation requests are received; and sequentially performing write operations on the atomic variables stored in the storage service includes: The write operation program corresponding to each write operation request is stored in a queue; The write operation programs are taken out from the queue in sequence to perform write operations on the atomic variables stored in the storage service.
5. The method according to claim 4, characterized in that Each write operation program in the queue is set to a blocking state; and taking out the write operation programs from the queue in sequence to perform write operations on the atomic variables stored in the storage service includes: Taking out the write operation program at the head position of the queue from the queue as the target write operation program, and switching the state of the target write operation program from a blocking state to a running state; A write operation is performed on the atomic variables stored in the storage service through a target write operation program in a running state. After the target write operation program in a running state completes the write operation on the atomic variables stored in the storage service, the write operation program at the head of the queue is taken out from the queue as the target write operation program to perform a write operation on the atomic variables stored in the storage service, until all the write operation programs in the queue are taken out.
6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: A shared cache service is created according to the received creation request; wherein the shared cache service is used to cache the atomic variables obtained from the storage service.
7. A data processing device, characterized in that: Applied to microservices, the microservices include storage services and shared cache services, the storage services are deployed in a disk, and the shared cache services are deployed in a memory, the device includes: A receiving module, configured to receive data processing requests respectively initiated by multiple business parties, wherein the data processing requests include read operation requests or write operation requests; a reading module configured to respond to at least one received read operation request, and if it is determined that the shared cache service has a latest version of the atomic variable cached, perform a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request; A write and synchronization module, configured to, in response to at least one received write operation request, sequentially perform write operations on the atomic variables stored in the storage service, and perform synchronization processing on the same atomic variable cached in the shared cache service; Before performing a read operation on the latest version of the atomic variable cached in the shared cache service according to the at least one read operation request, the method further includes: Compare the version corresponding to the atomic variable stored in the storage service with the version corresponding to the same atomic variable cached in the shared cache service; If the comparison result indicates that the version corresponding to the atomic variable stored in the storage service is higher than the version corresponding to the same atomic variable cached in the shared cache service, the atomic variable stored in the storage service is used to replace the same atomic variable cached in the shared cache service, and the version corresponding to the replaced atomic variable is updated so that the atomic variable cached in the shared cache service is the latest version.
8. The device according to claim 7, characterized in that The number of received read operation requests is at least two; the device further comprises: A first forwarding and obtaining module, configured to forward a read operation request to the storage service, and obtain a version corresponding to the stored atomic variable from the storage service; The second forwarding and obtaining module is configured to forward all other read operation requests to the shared cache service, and obtain the cached version corresponding to the same atomic variable from the shared cache service.
9. The device according to claim 7, characterized in that After performing a write operation on the atomic variables stored in the storage service, the atomic variables after the write operation are obtained, and the corresponding version of the atomic variables after the write operation is updated; The write and synchronization module comprises: an acquisition unit, configured to acquire the atomic variables after the write operation in the storage service, and the versions corresponding to the atomic variables after the write operation; The updating unit is configured to replace the same atomic variable cached in the shared cache service with the atomic variable after the write operation in the storage service, and update the version corresponding to the replaced atomic variable to the version corresponding to the atomic variable after the write operation.
10. The device according to claim 7, characterized in that The number of received write operation requests is at least two; the write and synchronization module comprises: A storage unit configured to store the write operation program corresponding to each write operation request in a queue; The fetching unit is configured to fetch the write operation programs from the queue in sequence to perform write operations on the atomic variables stored in the storage service.
11. The device according to claim 10, characterized in that Each write operation program in the queue is set to a blocking state; the fetching unit includes: a switching unit configured to take out the write operation program at the head position of the queue from the queue as the target write operation program, and switch the state of the target write operation program from a blocking state to a running state; A write unit is configured to perform a write operation on the atomic variables stored in the storage service through a target write operation program in a running state, and after the target write operation program in a running state completes the write operation on the atomic variables stored in the storage service, take out the write operation program at the head of the queue from the queue as the target write operation program to perform a write operation on the atomic variables stored in the storage service, until all the write operation programs in the queue are taken out.
12. The device according to claim 7, characterized in that The device also includes: A creation module is configured to create a shared cache service according to a received creation request; wherein the shared cache service is used to cache atomic variables obtained from the storage service.
13. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the electronic device, enables the electronic device to implement the data processing method as described in any one of claims 1 to 6.
14. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.
15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Method, system and device for processing shared cache synchronization message and medium
CN111221826A
Data synchronization method and device, electronic equipment and readable storage medium
CN111767296A