Data Processing Method, Apparatus, Server, and Storage Medium
By aggregating write data requests in the same technical field and generating data snapshots, the problems of transaction consistency and performance in high concurrency scenarios are solved, and efficient data processing and consistency guarantees are achieved.
Patent Information
- Application Number
- CN202110128043.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-29
- Publication Date
- 2025-06-13
- Estimated Expiration
- 2041-01-29
AI Technical Summary
In high concurrency scenarios, it is difficult for the prior art to ensure transaction consistency and performance efficiency at the same time, especially when the number of requests increases, locking or database transactions will lead to performance degradation.
By aggregating write data requests from the same technical field and generating data snapshots, recording the current time data information in the technical field is avoided, and each operation needs to be recorded separately, improving processing performance, and ensuring data consistency in high concurrency scenarios.
It realizes the guarantee of transaction consistency in high concurrency scenarios, while improving the performance of data processing, avoiding the negative impact of locking and database transactions on performance.
Smart Images

Figure CN113779083B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of Internet technologies, and in particular, to a data processing method, apparatus, server, and storage medium. Background Art
[0002] In recent years, with the development of Internet technologies, the increasing number of Internet users presents both opportunities and challenges to every Internet company. Only applications with strong performance can meet the challenges. Transaction processing in high-concurrency scenarios is one of the most important parts of applications.
[0003] Currently, in order to ensure transaction consistency in high-concurrency scenarios, there are the following methods: in the case of a single machine, it can be implemented with locks, through database transactions, or by using optimistic locks.
[0004] However, in the case of a low request volume, transactions are easily satisfied. However, as the number of requests increases, the requirements for transaction consistency and performance also become higher and higher. Although locking or database transactions can meet the consistency requirements, they have a greater impact on the performance of transaction processing. In summary, there is currently no technical solution that can both ensure transaction consistency in high-concurrency scenarios and have high performance. Summary of the Invention
[0005] Embodiments of the present invention provide a data processing method, apparatus, server, and storage medium, and provide a technical solution that can both ensure transaction consistency in high-concurrency scenarios and have high performance.
[0006] In a first aspect, an embodiment of the present invention provides a data processing method, including:
[0007] Receiving a plurality of write data requests sent by a plurality of client devices, each write data request including operation information;
[0008] Determining the technical field to which each write data request belongs according to the operation information;
[0009] Aggregating at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs, and generating a data snapshot, where the data snapshot is used to record the data information of the technical field at the current moment.
[0010] In a specific embodiment, the method further includes:
[0011] Sending the data snapshot corresponding to each technical field to a database for storage.
[0012] In a specific embodiment, the aggregating at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs, and generating a data snapshot includes:
[0013] Obtain at least one write data request for each technical field according to the technical field to which each write data request belongs;
[0014] For each technical field, generate the data snapshot after at least one write data request corresponding to the technical field is executed, and the data snapshot is used to record the data information after the execution of at least one write data request in the technical field.
[0015] In a specific embodiment, the method further includes:
[0016] Receive a read data request sent by any client;
[0017] Determine the technical field for which data needs to be read according to the read data request;
[0018] Obtain data information from the data snapshot of the technical field at the current moment, and return the data information to the client.
[0019] In a specific embodiment, the obtaining data information from the data snapshot of the technical field at the current moment includes:
[0020] According to the technical field, obtain the data snapshot corresponding to the technical field at the current moment from the database;
[0021] Obtain the data information from the data snapshot.
[0022] In a second aspect, an embodiment of the present invention provides a data processing device, including:
[0023] A receiving module, configured to receive multiple write data requests sent by multiple clients, and each write data request includes operation information;
[0024] A processing module, configured to determine the technical field to which each write data request belongs according to the operation information;
[0025] The processing module is further configured to aggregate at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs, and generate a data snapshot, and the data snapshot is used to record the data information of the technical field at the current moment.
[0026] In a specific embodiment, the device further includes:
[0027] A sending module, configured to send the data snapshot corresponding to each technical field to the database for storage.
[0028] In a specific embodiment, the processing module is specifically configured to:
[0029] Obtain at least one write data request for each technical field according to the technical field to which each write data request belongs;
[0030] For each technical field, generate the data snapshot after at least one write data request corresponding to the technical field is executed, and the data snapshot is used to record the data information after the execution of at least one write data request in the technical field.
[0031] In a specific embodiment, the receiving module is further configured to receive a read data request sent by any client;
[0032] The processing module is further configured to:
[0033] Determine the technical field for which data needs to be read according to the read data request;
[0034] Obtain data information from the data snapshot of the technical field at the current moment, and return the data information to the client.
[0035] In a specific embodiment, the processing module is specifically configured to:
[0036] According to the technical field, obtain the data snapshot corresponding to the technical field at the current moment from the database;
[0037] Obtain the data information from the data snapshot.
[0038] In a third aspect, an embodiment of the present invention provides a server, including:
[0039] A receiver, a processor, a transmitter; and,
[0040] A memory for storing executable instructions of the processor;
[0041] Wherein, the processor is configured to execute the data processing method according to any one of the first aspects by executing the executable instructions.
[0042] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the data processing method according to any one of the first aspects is implemented.
[0043] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, and when the computer program is executed by a processor, it is used to implement the data processing method according to any one of the first aspects.
[0044] The data processing method, device, server, and storage medium provided by the embodiments of the present invention. In this solution, in a server that responds to data read / write operation requests, when the server receives multiple write data requests sent by multiple clients, each write data request includes operation information. According to the operation information, the technical field to which each write data request belongs is determined. According to the technical field to which each write data request belongs, at least one write data request belonging to the same technical field is aggregated, and a data snapshot is generated. The data snapshot records the data information of this technical field at the current moment. For multiple operations in the same field, the current data information can be obtained by accumulating them, without recording each operation event one by one, which can ensure data consistency in a high-concurrency scenario and has good processing performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0046] Figure 1 It is a schematic diagram of the application scenario of the data processing method provided by the present invention;
[0047] Figure 2 It is a schematic flowchart of Embodiment 1 of a data processing method provided by the present invention;
[0048] Figure 3 It is a schematic flowchart of Embodiment 2 of a data processing method provided by the present invention;
[0049] Figure 4 It is a schematic flowchart of Embodiment 3 of a data processing method provided by the present invention;
[0050] Figure 5 It is a schematic diagram of a data processing method provided by the present invention;
[0051] Figure 6 It is a schematic diagram of the concept of a data processing method provided by the present invention;
[0052] Figure 7 It is a schematic structural diagram of an embodiment of a data processing device provided by the present invention;
[0053] Figure 8 It is a schematic structural diagram of an embodiment of a server provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0054] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention under the inspiration of this embodiment belong to the scope of protection of the present invention.
[0055] In the description of the present invention and the claims, as well as in the above drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.
[0056] In the prior art, when the request volume is not high, transactions are easily satisfied. However, as the number of requests increases, the requirements for transaction consistency and performance also become higher and higher. Although locking or database transactions can meet the consistency requirements, they have a greater impact on the performance of transaction processing. Based on the above needs, the main problems existing in the prior art mainly include:
[0057] 1. Using locks to prevent concurrent modification: By locking the program, the impact of concurrent transactions can be prevented to a certain extent. However, this method is only suitable for use in a single-machine scenario and cannot guarantee transaction consistency in a cluster environment. Although with the continuous improvement of computer performance, the time and cost of acquiring locks are also continuously reduced, the existence of locks means that only one thread can operate at the same time, which greatly reduces the concurrent performance of the program.
[0058] 2. Implementing through database transactions: Database transactions are a prerequisite for the database to ensure transaction consistency. When operations are running in a transaction environment, the database will ensure that only one modification can be made to the same record. However, if there are many records to be modified and the database cannot evaluate the impact, it may lead to a full-table lock of the database, making all operations unavailable. The implementation of database transactions requires a relatively high level of developers.
[0059] 3. Implement transactions using the optimistic locking approach: In addition to database transactions, the optimistic locking approach can be used to ensure transaction consistency in most cases. In the case of optimistic locking, only one of the concurrent modification requests can be successfully modified, and subsequent requests will fail to modify because they do not meet the conditions. However, in some cases, we actually expect all concurrent requests to be successfully operated. For example, when the inventory is sufficient, the out-of-stock operation can be performed.
[0060] 4. Only focus on data and cannot reflect operation behavior: The traditional implementation is a bloodless model where data and behavior are separated. Data is stored in the database, loaded into the application server when used, processed, and then stored back in the database.
[0061] In view of the above problems, an embodiment of the present invention provides a data processing method for implementing data operations in a cluster environment, reducing operation failures, improving operation performance, and ensuring the integration of data and behavior.
[0062] Figure 1 is a schematic diagram of the application scenario of the data processing method provided by the present invention; as Figure 1 shown, this scenario includes multiple clients running on a device. The device can be a computer, a personal computer, a smart phone, or other devices that can send data read and write requests to the server through clients such as application programs or browsers. The server is used to process and respond to the write data requests and read data requests sent by the clients, and the database is used to store data.
[0063] In view of the above technical problems and scenarios, the core idea of the data processing method provided by the present invention is as follows: For a concurrent scenario, when the server receives a large number of write data requests, according to the existing method, each request needs to be processed one by one, resulting in fewer processed requests. And after each request is processed, the change of the data needs to be recorded. However, sometimes the operations of multiple data requests will interfere with each other, and different operations are performed at the same time, resulting in inconsistent recorded data results, which may lead to inaccurate query results when querying data. Therefore, the inventor found in the process of implementing event sourcing that based on the idea of domain-driven, the write data operations of the same domain type can be recorded after completion. That is to say, multiple write data operations have no transaction dependencies. After multiple concurrent write data operations are executed, a data snapshot is generated, and the data information at the moment after the operation is recorded. That is to say, the change record of the data is reflected by recording the behavior events, and a snapshot is generated at a specific event to ensure data consistency and query efficiency.
[0064] Figure 2 is a schematic flowchart of Embodiment 1 of a data processing method provided by the present invention; as Figure 2 shown, in the above scenario, this data processing method is mainly applied on the server side and specifically includes the following steps:
[0065] S101: Receive multiple write data requests sent by multiple clients, where each write data request includes operation information.
[0066] In this step, for the server, the server may receive multiple write data requests sent by multiple clients simultaneously. Each write data request includes specific operation information, which can be the specific operation to be performed and the data for the operation, etc. For example, if the server receives multiple write data requests for a user's account data, including the operation information that the user is being transferred money by several people simultaneously, or that while money is being transferred in, there are also transfer-out operations to other people at the same time. Then the operation information included in different write data requests can include the transfer-in operation and the transfer-in amount, or include the transfer-out operation and the transfer-out amount, etc. Or, if the received write data request is an operation after placing an order for online shopping, then the write data request can include the order placement operation, the information of the ordered goods, and the quantity of the ordered goods, etc.
[0067] In this solution, it should be understood that the multiple write data requests sent by multiple clients received can be received simultaneously, or can be received within a period of time, and the length of the time period can be set according to the actual implementation of the solution.
[0068] S102: Determine the technical field to which each write data request belongs according to the operation information.
[0069] In this step, the server determines the technical field of the write data request according to the operation information in each write data request. Here, the technical field is used to indicate the type of the write data request, for example: transfer, purchase of goods, etc.
[0070] S103: Aggregate at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs, and generate a data snapshot, where the data snapshot is used to record the data information of the technical field at the current moment.
[0071] A specific implementation manner of this step is: The server obtains at least one write data request for each technical field (which can be two or more write data) according to the technical field to which each write data request belongs. For each technical field, after the at least one write data request corresponding to the technical field is executed, the data snapshot is generated, and the data snapshot is used to record the data information after the execution of the at least one write data request of the technical field.
[0072] In this step, the server can aggregate multiple write data requests belonging to the same technical field. For example, if they are all transfer operations for a certain account or a certain user, these transfer operations can be aggregated and executed, and finally the account data of the account or user can be obtained, so as to generate a data snapshot of this technical field at this moment. That is to say, only one data snapshot is generated after executing multiple parallel write data requests in the same field, and the data information at this moment is recorded in the data snapshot.
[0073] If multiple write data requests in multiple fields are received simultaneously, then the requests for each field can be aggregated separately to obtain the corresponding data snapshot of each field at this moment.
[0074] The data processing method provided in this embodiment, in the server that responds to data read and write operation requests, when the server receives multiple write data requests sent by multiple clients, each write data request includes operation information, determines the technical field to which each write data request belongs according to the operation information, and aggregates at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs, and generates a data snapshot. The data information of this technical field at the current moment is recorded in the data snapshot. For multiple operations in the same field, the current data information can be obtained by accumulation, and each operation event does not need to be recorded one by one, which can ensure data consistency in a high-concurrency scenario and has better processing performance.
[0075] Figure 3 It is a schematic flowchart of the second embodiment of a data processing method provided by the present invention; as Figure 3 shown, on the basis of the above embodiment, the data processing method further includes:
[0076] S104: Send the data snapshot corresponding to each technical field to the database for storage.
[0077] In this step, the server obtains the data snapshot of each technical field at the current moment according to the solution of the foregoing embodiment, and sends the data snapshot at the current moment to the database for storage, so that subsequent queries can be based on the data snapshot when reading data.
[0078] Figure 4 It is a schematic flowchart of the third embodiment of a data processing method provided by the present invention; as Figure 4 shown, on the basis of any of the above embodiments, the data processing method further includes:
[0079] S201: Receive a read data request sent by any client.
[0080] In this step, when the user needs to query data, a read data request can be sent through the client. Similarly, the information in the read data request can determine the specific technical field. For example, the read data request can be to query the account balance, inventory, or order information, etc. This solution does not impose any restrictions on this.
[0081] S202: Determine the technical field in which data needs to be read according to the data read request.
[0082] S203: Acquire data information from the data snapshot of the technical field at the current moment, and return the data information to the client.
[0083] In the above steps, after receiving the data read request, the server determines the technical field where data needs to be read based on the information carried in the data read request, and then obtains the data snapshot at the current moment corresponding to the technical field from the database according to the technical field. Specifically, the server can send a snapshot acquisition request to the database based on the technical field, and the database returns the data snapshot to the server based on the technical field. The server continues to obtain the data information from the data snapshot.
[0084] After obtaining the data information, the server returns the data information to the client, completing the data reading process.
[0085] Based on any of the above embodiments, the data processing method of the present invention is illustrated below.
[0086] Figure 5 A schematic diagram of the data processing method provided by the present invention, Figure 6 It is a conceptual schematic diagram of a data processing method provided by the present invention, such as Figure 5 and 6 As shown, when the server receives multiple write data requests, one of which adds (500) to a certain data, another adds (100) to the data, and another sub (100) to the data, then these three write data requests are all modifications to the data in the same field. In the specific operation process, they belong to input 500, input 100, and output 100. When generating a data snapshot, the specific processes of these data operations are recorded respectively and the final execution result at this time is obtained, that is, total (500) in the figure. After executing the operations corresponding to the above three write data requests, the final data information is recorded in the snapshot.
[0087] The next time two write data requests sub(200) and add(100) are received, the operations corresponding to these two requests are output 200 and input 100. When generating a data snapshot, the specific processes of these data operations are recorded respectively and the final execution result at this time is obtained, that is, total(400) in the figure. After executing the operations corresponding to the foregoing two write data requests, the final data information is recorded in the snapshot.
[0088] The core of the data processing method provided by the present invention is a system based on domain-driven design event sourcing. The core content is events. Event sourcing is generally used together with the Command Query Responsibility Segregation pattern (CQRS). The event sourcing pattern calculates the current state of data not by recording the current state of the data but by accumulating the events that occur to the data (such as Figure 5 the example shown). Event sourcing can reflect the change records of data with respect to data operations. More data and operations also provide considerable convenience for data mining. Event sourcing records individual events and does not require locking the data for modification. The overall service has higher processing capacity and throughput for write data requests. The CQRS pattern separates the behaviors of modification (Command, addition, deletion, modification, which will modify the system state) and query (Query, query, which will not modify the system state) from a business perspective. This makes the logic clearer and facilitates targeted optimization of different parts.
[0089] It should be understood that event sourcing uses the concept of domain events to achieve the persistence of aggregates, and each aggregate is persisted as a series of events in the database. The application retrieves and replays events from the event store to load the aggregate. The core of event sourcing is to record the events of each operation, the change situation of the data before and after the event occurs. It can even only record the occurrence of the event and the impact of the event occurrence. The occurrence of multiple events will not affect each other. As long as it is ensured that each event operation is accurately recorded, the consistency of the transaction can be ensured, and the consistency of the transaction is of utmost importance. In this solution, a transaction is a sequence of database operations that access and may operate on various data items. These operations are either all executed or all not executed, and it is an indivisible unit of work. A transaction consists of all database operations executed between the start and end of the transaction.
[0090] Event sourcing and CQRS are closely related and complementary. By recording events, it can be ensured that the C (Command) records of the data do not interfere with each other. And through the aggregation of the domain model, a snapshot of the data can be generated to ensure the query (Query) performance of the data, separating the read and write of the data, and both the read and write are in the most suitable model.
[0091] Under the idea of domain-driven design, combined with the characteristics of the model itself, according to the read model and write model for reading and modifying design data respectively, the write model ensures that operations on data will not be lost by recording events, and there is no transaction dependency and sequence for multiple operations to ensure write performance. For reading, snapshots of multiple events are generated according to the domain model to ensure query efficiency.
[0092] In a truly object-oriented design, data and behavior are together (such as the Actor model (an Actor refers to the most basic computing unit. It can receive a message and perform calculations based on it. This concept is very similar to an object-oriented language, where an object receives a message (method call) and then operates according to the received message (which method is called))), that is, the service layer that executes the logic also holds data. Each operation request for data is also an event, and the change record of the data is reflected by recording the behavior event, and a snapshot is generated at a specific time to ensure query efficiency.
[0093] Figure 7 It is a schematic structural diagram of an embodiment of a data processing device provided by the present invention; as Figure 7 shown, the data processing device 10 includes:
[0094] A receiving module 11, configured to receive multiple write data requests sent by multiple clients, and each write data request includes operation information;
[0095] A processing module 12, configured to determine the technical field to which each write data request belongs according to the operation information;
[0096] The processing module 12 is further configured to aggregate at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs, and generate a data snapshot, where the data snapshot is used to record the data information of the technical field at the current moment.
[0097] In a specific implementation manner, the data processing device 10 further includes:
[0098] A sending module 13, configured to send the data snapshot corresponding to each technical field to a database for storage.
[0099] In a specific implementation manner, the processing module 12 is specifically configured to:
[0100] Obtain at least one write data request for each technical field according to the technical field to which each write data request belongs;
[0101] For each technical field, a data snapshot is generated after at least one write data request corresponding to the technical field is executed, and the data snapshot is used to record data information after the at least one write data request in the technical field is executed.
[0102] In a specific embodiment, the receiving module 11 is further configured to receive a read data request sent by any client;
[0103] The processing module 12 is further configured to:
[0104] Determine the technical field for which data needs to be read according to the read data request;
[0105] Obtain data information from the data snapshot of the technical field at the current moment, and return the data information to the client.
[0106] Optionally, the processing module 12 is specifically configured to:
[0107] Obtain the data snapshot corresponding to the technical field at the current moment from the database according to the technical field;
[0108] Obtain the data information from the data snapshot.
[0109] The data processing device provided in any of the above embodiments is used to execute the technical solutions in any of the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be elaborated here.
[0110] Figure 8 It is a schematic structural diagram of an embodiment of a server provided by the present invention. As Figure 8 shown, the server 20 includes:
[0111] A receiver 21, a processor 22, a transmitter 23; and,
[0112] A memory 24 for storing executable instructions of the processor;
[0113] Wherein, the processor 22 is configured to execute the data processing method provided in any of the foregoing method embodiments by executing the executable instructions.
[0114] Optionally, the memory 24 can be either independent or integrated with the processor 22.
[0115] When the memory 24 is a device independent of the processor 22, the server may further include:
[0116] A bus 25 for connecting the above devices.
[0117] This server is used to execute the data processing method provided by any of the foregoing method embodiments. The implementation principles and technical effects are similar and will not be elaborated here.
[0118] An embodiment of the present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the data processing method provided by any of the foregoing method embodiments.
[0119] An embodiment of the present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it is used to implement the data processing method provided by any of the foregoing method embodiments.
[0120] Those of ordinary skill in the art can understand that all or part of the steps of implementing the foregoing method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the foregoing method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disk that can store program codes.
[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A data processing method, characterized in that, it includes: Receiving multiple write data requests sent by multiple clients, where each write data request includes operation information; Determining the technical field to which each write data request belongs according to the operation information; According to the technical field to which each write data request belongs, aggregating at least one write data request belonging to the same technical field and generating a data snapshot, where the data snapshot is used to record the data information of the technical field at the current moment; The step of aggregating at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs and generating a data snapshot includes: Obtaining at least one write data request for each technical field according to the technical field to which each write data request belongs; For each technical field, generating the data snapshot after at least one write data request corresponding to the technical field is executed, where the data snapshot is used to record the data information after the execution of the at least one write data request of the technical field; The method further includes: Receiving a read data request sent by any one of the clients; Determining the technical field for which data needs to be read according to the read data request; Obtaining data information from the data snapshot of the technical field at the current moment and returning the data information to the client.
2. The method according to claim 1, characterized in that, the method further includes: Sending the data snapshot corresponding to each technical field to a database for storage.
3. The method according to claim 1, characterized in that, the step of obtaining data information from the data snapshot of the technical field at the current moment includes: Obtaining the data snapshot of the current moment corresponding to the technical field from the database according to the technical field; Obtaining the data information from the data snapshot.
4. A data processing device, characterized in that, it includes: A receiving module, configured to receive multiple write data requests sent by multiple clients, where each write data request includes operation information; A processing module, configured to determine the technical field to which each write data request belongs according to the operation information; The processing module is further configured to aggregate at least one write data request belonging to the same technical field according to the technical field to which each write data request belongs and generate a data snapshot, where the data snapshot is used to record the data information of the technical field at the current moment; Specifically, the processing module is configured to: Obtain at least one write data request for each technical field according to the technical field to which each write data request belongs; For each technical field, generate the data snapshot after at least one write data request corresponding to the technical field is executed, where the data snapshot is used to record the data information after the execution of the at least one write data request of the technical field; The receiving module is further configured to receive a read data request sent by any one of the clients; The processing module is further configured to: Determine the technical field for which data needs to be read according to the read data request; Obtain data information from the data snapshot of the technical field at the current moment and return the data information to the client.
5. The device according to claim 4, characterized in that, The device further comprises: a sending module, configured to send data snapshots corresponding to each technical field to a database for storage.
6. The device according to claim 4, wherein, the processing module is specifically configured to: obtain, according to the technical field, a data snapshot corresponding to the technical field at the current moment from the database; obtain the data information from the data snapshot.
7. A server, wherein, it comprises: a receiver, a processor, and a transmitter; and, a memory, configured to store executable instructions of the processor; wherein, the processor is configured to execute the data processing method according to any one of claims 1 to 3 by executing the executable instructions.
8. A readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, it implements the data processing method according to any one of claims 1 to 3.
9. A computer program product, wherein, it comprises a computer program, and when the computer program is executed by a processor, it is used to implement the data processing method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Scalable event sourcing datastore
US20200250172A1