Data processing method, device, system and storage medium

Abstract

The present application provides a data processing method, device, system and storage medium, the method comprising: performing timestamp packaging on the data to be updated to obtain the packaged data; storing the packaged data in a cache space; traversing the cache space according to a preset cycle to obtain the time corresponding to the timestamp of the packaged data in the cache space; if the time corresponding to the timestamp of the packaged data is less than the current time, writing the packaged data into a database. The present application can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a data processing method, device, system and storage medium. Background Art

[0002] As the amount of data processing continues to increase, it is necessary to store and manage data through a database. In some scenarios where the business is more complex or there are many users, data often needs to be updated frequently. Every time an update is made, the data is submitted to the database, which then cleans and stores the submitted data.

[0003] At present, the common approach is to improve the database configuration itself to adapt to the frequent data updates, or to suppress the database data processing peak through asynchronous processing.

[0004] However, the above method cannot reduce the frequency of data updates, resulting in higher requirements for database configuration and throughput. Summary of the invention

[0005] The present application provides a data processing method, device, system and storage medium, which can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0007] Timestamp the data to be updated and obtain the packaged data;

[0008] Storing the packaged data in a cache space;

[0009] According to a preset period, traverse the cache space to obtain the time corresponding to the timestamp of the packaged data in the cache space;

[0010] If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database.

[0011] In an optional design, the step of performing timestamp packaging on the data to be updated to obtain the packaged data includes:

[0012] Add a starting time point and a valid time period to the data to be updated to obtain the packaged data; wherein the starting time point includes: the data generation time, or the last update time of the data; the valid time period includes: the upper limit time during which the data does not change.

[0013] In an optional design, storing the packaged data in a cache space includes:

[0014] Obtaining the data identifier of the data to be updated;

[0015] If the data corresponding to the data identifier exists in the cache space, overwriting the data corresponding to the data identifier in the cache space with the packaged data;

[0016] If the data corresponding to the data identifier does not exist in the cache space, the packaged data is stored in the cache space.

[0017] In an optional design, obtaining the time corresponding to the timestamp of the packaged data in the cache space includes:

[0018] Obtaining the starting time point and valid time period of the packaged data in the cache space;

[0019] The sum of the starting time point and the valid time period is calculated to obtain the time corresponding to the timestamp of the packaged data.

[0020] In an optional design, it also includes:

[0021] If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data in the cache space is deleted.

[0022] In an optional design, it also includes:

[0023] If the time corresponding to the timestamp of the packaged data is not less than the current time, the packaged data is not written into the database.

[0024] In a second aspect, an embodiment of the present application provides a data processing device, including:

[0025] The packaging module is used to package the updated data with a timestamp to obtain the packaged data;

[0026] A storage module, used for storing the packaged data into a cache space;

[0027] A traversal module, used to traverse the cache space according to a preset period, and obtain the time corresponding to the timestamp of the packaged data in the cache space;

[0028] The processing module is used to write the packaged data into a database when the time corresponding to the timestamp of the packaged data is less than the current time.

[0029] In an optional design, the packaging module is specifically used for:

[0030] Add a starting time point and a valid time period to the data to be updated to obtain the packaged data; wherein the starting time point includes: the data generation time, or the last update time of the data; the valid time period includes: the upper limit time during which the data does not change.

[0031] In an optional design, the storage module is specifically used for:

[0032] Obtaining the data identifier of the data to be updated;

[0033] If the data corresponding to the data identifier exists in the cache space, overwriting the data corresponding to the data identifier in the cache space with the packaged data;

[0034] If the data corresponding to the data identifier does not exist in the cache space, the packaged data is stored in the cache space.

[0035] In an optional design, the traversal module is specifically used to:

[0036] Obtaining the starting time point and valid time period of the packaged data in the cache space;

[0037] The sum of the starting time point and the valid time period is calculated to obtain the time corresponding to the timestamp of the packaged data.

[0038] In an optional design, the method further includes: a deletion module, which is used to:

[0039] When the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data in the cache space is deleted.

[0040] In an optional design, the processing module is further used for:

[0041] When the time corresponding to the timestamp of the packaged data is not less than the current time, the packaged data is not written into the database.

[0042] In a third aspect, an embodiment of the present application provides a data processing system for a data warehouse, comprising: a memory and a processor, wherein the memory stores executable instructions of the processor; wherein the processor is configured to execute the data processing method described in any one of the first aspects by executing the executable instructions.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method described in any one of the first aspects.

[0044] In a fifth aspect, an embodiment of the present application provides a program product, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of a server can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the server executes any data processing method described in the first aspect.

[0045] The present application provides a data processing method, device, system and storage medium, which obtains the packaged data by timestamp packaging of the data to be updated; stores the packaged data in a cache space; traverses the cache space according to a preset period to obtain the time corresponding to the timestamp of the packaged data in the cache space; if the time corresponding to the timestamp of the packaged data is less than the current time, writes the packaged data into a database. The present application can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 This is a schematic diagram of the principle of an application scenario of this application;

[0048] Figure 2 A flowchart of a data processing method provided in Example 1 of the present application;

[0049] Figure 3 A flowchart of a data processing method provided in Embodiment 2 of the present application;

[0050] Figure 4 A schematic diagram of the structure of a data processing device provided in Embodiment 3 of the present application;

[0051] Figure 5 A schematic diagram of the structure of a data processing device provided in Embodiment 4 of the present application;

[0052] Figure 6 This is a structural diagram of the data processing system provided in Example 5 of the present application.

[0053] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0056] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0057] As the amount of data processing continues to increase, it is necessary to store and manage data through a database. In some scenarios where the business is more complex or there are more users, it is often necessary to update the data frequently. For example, the attribute of game coins in the game system is frequently updated because other system modules (such as level systems, reward systems, task systems, daily sign-ins, etc.) will increase or decrease this attribute. Every time an update is made, the data will be submitted to the database, and then the database will clean and store the submitted data. At present, the general method is to improve the configuration of the database itself to adapt to the frequent updates of data, or to suppress the data processing peak of the database through asynchronous processing. However, the above method cannot reduce the frequency of data updates, resulting in higher requirements for database configuration and throughput.

[0058] In response to the above technical problems, the present application provides a data processing method, device, system and storage medium, which can be applied to a data storage device configured with a cache. It can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database. Figure 1 This is a schematic diagram of the principle of an application scenario of this application, such as Figure 1 As shown, when the data is updated, a timestamp is added to the data to be updated to obtain the packaged data. A space can be opened in the memory to place a collection structure for storing the packaged data, that is, to implement the Map interface, using key-value (in the form of key-value pairs) to facilitate search. The key of the Map structure is the key of the package class object, and the value of the Map structure is the package class object itself. According to the characteristics of the Map data structure, the data identifier of the data to be updated is obtained when storing; if the data corresponding to the data identifier exists in the cache space, the data is updated, and the packaged data overwrites the data corresponding to the data identifier in the cache space; if the data corresponding to the data identifier does not exist in the cache space, the packaged data is stored in the cache space. A daemon thread can also be started to traverse the data in the cache space Map according to a preset cycle to obtain the time corresponding to the timestamp of the packaged data in the cache space. Then, the time corresponding to the timestamp is compared with the current system time. If it is less than the current system time, the packaged data is written into the database, and the packaged data in the cache space is deleted to avoid repeated data writing. If the time corresponding to the timestamp of the packaged data is not less than the current time, the packaged data is not written into the database.

[0059] The above method can be applied to adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

[0060] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0061] Figure 2 This is a flowchart of a data processing method provided in Example 1 of the present application, such as Figure 2 As shown, the method in this embodiment may include:

[0062] S101. Timestamp the data to be updated to obtain the packaged data.

[0063] In this embodiment, when the data is updated, the starting time point and the valid time period are added to the data to be updated to obtain the packaged data. The starting time point includes: the data generation time, or the last update time of the data; the valid time period includes: the upper limit time of the data not changing.

[0064] Specifically, a packaging class can be added, including attributes such as lastActiveTime, ttl, key, and value. Among them, the starting time point lastActiveTime indicates the data generation time or the last update time, which is the timestamp when the original data is packaged, accurate to milliseconds. The effective time period ttl indicates the upper limit time of data unchanged, accurate to milliseconds. The keyword key is the unique identifier key of the original data, such as the id of the original data, or a string composed of multiple fields, to ensure that the key of each data is not repeated with other data. The original data is stored in the data value. When the data is updated, the data is packaged once to generate an entity of the packaging class. For example, taking Java as an example, when the data entity object A is updated, a packaging class object is generated. Among them, the lastActiveTime of this packaging class object is the time when the packaging class is generated, which can be the current system time; ttl is the upper limit time of data unchanged, which can be set according to the type of entity A, and different upper limits of data unchanged can be set for different data types; key is the id of data entity A; data entity object A is stored in the data value.

[0065] S102: Store the packaged data into the cache space.

[0066] In this embodiment, the data identifier of the data to be updated is obtained; if the data corresponding to the data identifier exists in the cache space, the packaged data overwrites the data corresponding to the data identifier in the cache space; if the data corresponding to the data identifier does not exist in the cache space, the packaged data is stored in the cache space.

[0067] Specifically, a space can be opened in the memory to place a collection structure to store the packaged data, that is, to implement the Map interface, using key-value (in the form of key-value pairs) to facilitate search. The key of the Map structure is the key of the packaged object, and the value of the Map structure is the packaged object itself. For example, you can consider using ConcurrentHashMap in concurrent application scenarios. The Key of the object is the key of ConcurrentHashMap, and the object itself is stored in the collection structure as the Value. According to the characteristics of the Map data structure, if there is already data with the same key in the Map, the new data will overwrite the original data. If there is no data with the same key in the Map, a new key-value pair record will be inserted.

[0068] S103: traverse the cache space according to a preset period to obtain the time corresponding to the timestamp of the packaged data in the cache space.

[0069] In this embodiment, the cache space can be traversed according to a preset period to obtain the starting time point and valid time period of the packaged data in the cache space; the sum of the starting time point and the valid time period is calculated to obtain the time corresponding to the timestamp of the packaged data.

[0070] Specifically, when the data structure is initialized, a daemon thread can be started to traverse the data in the cache space Map according to a preset period. For example, a timer Timer is set, and the time interval is set to 1 minute to traverse all the data in the Map. The time corresponding to the timestamp of the packaged data is obtained according to the sum of the starting time point lastActiveTime in the packaging class and the effective time period ttl.

[0071] S104: If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database.

[0072] In this embodiment, the time corresponding to the timestamp of the packaged data is compared with the current time. If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database. If the time corresponding to the timestamp of the packaged data is not less than the current time, the packaged data is not written into the database.

[0073] Specifically, the time obtained in S103 can be compared with the current system time. If it is less than the current system time, the packaged data will be written into the database. For example, suppose that in the database, there is data with id abc, and this data has an attribute value=100. Due to business operations, the value of value is constantly +1, and it increases to 105. If the database is operated directly, the data with id abc will be updated 5 times, and then the data will remain at 105. The number of operations on the database is 5 times. If the method provided by the present application is adopted, each time the data changes by +1, it will be temporarily updated to the memory without performing any operation on the database. After 5 updates, the value in the memory will be updated to the latest 105. When the daemon process determines that it has timed out, it executes the operation of writing to the database, and the value in the database will be updated to 105. The entire process only performs one database operation. The pressure on the database is reduced, and the configuration of the database can be reduced to achieve the purpose of low cost.

[0074] In this embodiment, the packaged data is obtained by timestamp packaging of the data to be updated; the packaged data is stored in the cache space; according to the preset period, the cache space is traversed to obtain the time corresponding to the timestamp of the packaged data in the cache space; if the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database. This application can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

[0075] Figure 3 This is a flow chart of a data processing method provided in Example 2 of the present application, such as Figure 3 As shown, the method in this embodiment may include:

[0076] S201. Timestamp the data to be updated to obtain the packaged data.

[0077] S202: Store the packaged data into the cache space.

[0078] S203: traverse the cache space according to a preset period to obtain the time corresponding to the timestamp of the packaged data in the cache space.

[0079] S204: If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database.

[0080] In this embodiment, the specific implementation process and technical principles of steps S201 to S204 can be found in Figure 2 The relevant descriptions in steps S101 to S104 in the method shown are not repeated here.

[0081] S205: If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data in the cache space is deleted.

[0082] In this embodiment, after the packaged data is written into the database, the packaged data in the cache space may be deleted to avoid duplicate writing of data, thereby ensuring the stability of the database and reducing the hardware cost of the database.

[0083] In this embodiment, the packaged data is obtained by timestamp packaging of the data to be updated; the packaged data is stored in the cache space; according to the preset period, the cache space is traversed to obtain the time corresponding to the timestamp of the packaged data in the cache space; if the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database. This application can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

[0084] In addition, this embodiment can also delete the packaged data in the cache space after writing the packaged data into the database to avoid repeated writing of data, ensure the stability of the database, and reduce the hardware cost of the database.

[0085] Figure 4 This is a schematic diagram of the structure of the data processing device provided in Example 3 of the present application, such as Figure 4 As shown, the data processing device of this embodiment may include:

[0086] The packaging module 31 is used to perform timestamp packaging on the data to be updated to obtain the packaged data;

[0087] The storage module 32 is used to store the packaged data into the cache space;

[0088] A traversal module 33, used to traverse the cache space according to a preset period, and obtain the time corresponding to the timestamp of the packaged data in the cache space;

[0089] The processing module 34 is used to write the packaged data into the database when the time corresponding to the timestamp of the packaged data is less than the current time.

[0090] In an optional design, the packaging module 31 is specifically used for:

[0091] Add a starting time point and a valid time period to the data to be updated to obtain packaged data; wherein the starting time point includes: the data generation time, or the last update time of the data; the valid time period includes: the upper limit time during which the data does not change.

[0092] In an optional design, the storage module 32 is specifically used for:

[0093] Get the data identifier of the data to be updated;

[0094] If the data corresponding to the data identifier exists in the cache space, the packaged data will overwrite the data corresponding to the data identifier in the cache space;

[0095] If the data corresponding to the data identifier does not exist in the cache space, the packaged data is stored in the cache space.

[0096] In an optional design, the traversal module 33 is specifically used to:

[0097] Get the starting time point and valid time period of the packaged data in the cache space;

[0098] Calculate the sum of the starting time point and the valid time period to obtain the time corresponding to the timestamp of the packaged data.

[0099] In an optional design, the processing module 34 is further configured to:

[0100] When the time corresponding to the timestamp of the packaged data is not less than the current time, the packaged data is not written into the database.

[0101] The data processing device of this embodiment can execute Figure 2 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 2 The relevant descriptions in the method shown will not be repeated here.

[0102] In this embodiment, the packaged data is obtained by timestamp packaging of the data to be updated; the packaged data is stored in the cache space; according to the preset period, the cache space is traversed to obtain the time corresponding to the timestamp of the packaged data in the cache space; if the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database. This application can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

[0103] Figure 5 A structural diagram of a data processing device provided in Embodiment 4 of the present application, such as Figure 5 As shown, the data processing device of this embodiment is Figure 4 Based on the device shown, it can also include:

[0104] Deletion module 35, used to:

[0105] When the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data in the cache space is deleted.

[0106] The data processing device of this embodiment can execute Figure 2 , Figure 3 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 2 , Figure 3 The relevant descriptions in the method shown will not be repeated here.

[0107] In this embodiment, the packaged data is obtained by timestamp packaging of the data to be updated; the packaged data is stored in the cache space; according to the preset period, the cache space is traversed to obtain the time corresponding to the timestamp of the packaged data in the cache space; if the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database. This application can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

[0108] In addition, this embodiment can also delete the packaged data in the cache space after writing the packaged data into the database to avoid repeated writing of data, ensure the stability of the database, and reduce the hardware cost of the database.

[0109] Figure 6 A structural diagram of a data processing system provided in Embodiment 5 of the present application is shown in FIG. Figure 6 As shown, the data processing system 40 of this embodiment may include: a processor 41 and a memory 42 .

[0110] The memory 42 is used to store programs; the memory 42 may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory 42 is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories 42. And the above computer programs, computer instructions, data, etc. can be called by the processor 41.

[0111] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories 42 in partitions. And the above-mentioned computer programs, computer instructions, data, etc. may be called by the processor 41.

[0112] The processor 41 is used to execute the computer program stored in the memory 42 to implement the various steps in the method involved in the above embodiment.

[0113] For details, please refer to the relevant description in the previous method embodiment.

[0114] The processor 41 and the memory 42 may be independent structures or integrated structures. When the processor 41 and the memory 42 are independent structures, the memory 42 and the processor 41 may be coupled and connected via a bus 43 .

[0115] The data processing system of this embodiment can execute Figure 2 , Figure 3For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 2 , Figure 3 The relevant descriptions in the method shown will not be repeated here.

[0116] In this embodiment, the packaged data is obtained by timestamp packaging of the data to be updated; the packaged data is stored in the cache space; according to the preset period, the cache space is traversed to obtain the time corresponding to the timestamp of the packaged data in the cache space; if the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database. This application can adapt to high-frequency data updates without changing the database configuration, ensure the stability of the database, and reduce the hardware cost of the database.

[0117] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When at least one processor of a user device executes the computer-executable instructions, the user device executes the above-mentioned various possible methods.

[0118] Among them, computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.

[0119] The present application also provides a program product, which includes a computer program, which is stored in a readable storage medium. At least one processor of the server can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the server implements any method of the above-mentioned embodiments of the present application.

[0120] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: include: Timestamp the data to be updated and obtain the packaged data; Storing the packaged data in a cache space; According to a preset period, traverse the cache space to obtain the time corresponding to the timestamp of the packaged data in the cache space; If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data is written into the database; The time stamp packaging of the data to be updated is performed to obtain the packaged data, including: Add a packaging class, the attributes of which include a starting time point, a valid time period, a keyword, and data; the starting time point indicates the data generation time or the last update time; the valid time period indicates the upper limit time during which the data is not changed; and the keyword is the unique identifier of the data; When the data is updated, the data is packaged and an entity of the packaging class is generated; Storing the packaged data in a cache space includes: Obtaining the data identifier of the data to be updated; If the data corresponding to the data identifier exists in the cache space, overwriting the data corresponding to the data identifier in the cache space with the packaged data; If the data corresponding to the data identifier does not exist in the cache space, the packaged data is stored in the cache space.

2. The method according to claim 1, characterized in that Obtaining the time corresponding to the timestamp of the packaged data in the cache space includes: Obtaining the starting time point and valid time period of the packaged data in the cache space; The sum of the starting time point and the valid time period is calculated to obtain the time corresponding to the timestamp of the packaged data.

3. The method according to any one of claims 1 to 2, characterized in that: Also includes: If the time corresponding to the timestamp of the packaged data is less than the current time, the packaged data in the cache space is deleted.

4. The method according to any one of claims 1 to 2, characterized in that Also includes: If the time corresponding to the timestamp of the packaged data is not less than the current time, the packaged data is not written into the database.

5. A data processing device, characterized in that: include: The packaging module is used to package the updated data with a timestamp to obtain the packaged data; A storage module, used for storing the packaged data into a cache space; A traversal module, used to traverse the cache space according to a preset period, and obtain the time corresponding to the timestamp of the packaged data in the cache space; A processing module, configured to write the packaged data into a database when the time corresponding to the timestamp of the packaged data is less than the current time; The packaging module is specifically used to: add a packaging class, the attributes of the packaging class include a starting time point, a valid time period, a keyword and data; the starting time point indicates the data generation time or the last update time; the valid time period indicates the upper limit time of the data not changing; the keyword is the unique identifier of the data; when the data is updated, the data is packaged once to generate an entity of the packaging class; The storage module is specifically used to obtain the data identifier of the data to be updated; if the data corresponding to the data identifier exists in the cache space, the packaged data will overwrite the data corresponding to the data identifier in the cache space; if the data corresponding to the data identifier does not exist in the cache space, the packaged data will be stored in the cache space.

6. A data processing system for a data warehouse, characterized in that: include: A memory and a processor, wherein executable instructions of the processor are stored in the memory; wherein the processor is configured to execute the data processing method of claims 1-4 by executing the executable instructions.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data processing method described in claims 1 to 4 is implemented.

Images