Data storage method, server and storage medium
Through dual database architecture and data classification, combined with regular response events and hashing operations, traditional databases solve the challenges of data immutability, integrity, auditability and performance, and achieve more efficient and secure data storage and management.
Patent Information
- Application Number
- CN202510533988.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-27
- Publication Date
- 2025-05-27
- Estimated Expiration
- 2045-04-27
AI Technical Summary
Traditional databases have challenges in data immutability, integrity, auditability, and concurrency and read-write performance, and cannot meet complex and diverse business needs.
The dual database architecture is adopted to divide the data into state data and non-state data, and stored in a state database that supports addition, deletion, and correction, and a non-state database that only supports addition, investigation, and the data is not tampered with and efficient storage by regularly responding to data persistence and immutability events.
It improves the immutability, integrity and auditability of data, while improving concurrent processing capabilities and read and write performance, adapting to complex and diverse business needs.
Smart Images

Figure CN120045574A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the technical field of data storage, and in particular, to a data storage method, a server, and a storage medium. Background Art
[0002] In the digital information age, data has become the core asset of various organizations and enterprises. As a key tool for data storage and management, the performance and security of databases are crucial. However, traditional databases face many challenges in terms of data immutability, integrity, auditability, as well as concurrency and read / write performance.
[0003] In terms of data immutability, traditional databases usually lack effective mechanisms to ensure that data cannot be tampered with once stored. Due to the limitations of data storage structures and access controls, internal personnel or attackers with certain permissions may maliciously modify the data in the database, and such modifications are often difficult to trace and detect. This may bring serious consequences to some scenarios with high requirements for data authenticity and integrity, such as financial transaction records and medical record files.
[0004] In terms of data integrity, although traditional databases have some constraint mechanisms, such as primary key constraints and foreign key constraints, these mechanisms mainly verify the format and relevance of data and cannot fundamentally ensure that data is not tampered with during storage and transmission. Once the data is attacked externally or operated incorrectly internally, the integrity of the data may be damaged, thereby affecting the accuracy of business decisions and analyses based on this data.
[0005] Auditability is also a weak link in traditional databases. When problems occur with the data, it is difficult to quickly and accurately trace the change history of the data, the personnel who made the changes, and the reasons for the changes. Although traditional logging methods can record some operations, for complex database operations and large-scale data changes, the management and analysis of logs become extremely difficult and cannot meet the requirements of efficient auditing.
[0006] In terms of concurrency and read / write performance, as the amount of data continues to grow and the business complexity increases, traditional databases often struggle to handle high-concurrency read / write requests. For example, when processing a large number of concurrent transactions, relational databases are prone to lock contention problems, resulting in low transaction processing efficiency and a serious decline in read / write performance. This not only affects the user experience but also limits the expansion and operational efficiency of enterprise businesses.
[0007] Although the emergence of blockchain technology provides good solutions in terms of data immutability, rights confirmation, traceability, and auditing, it also has some limitations. The data processing method of blockchain is relatively complex, and its performance is far inferior to that of traditional databases when performing operations such as joined table queries and efficient reading and writing, which traditional databases are good at. Moreover, the architecture design of blockchain makes its storage and computing resources consume huge amounts when dealing with large-scale data, making it difficult to meet some business scenarios with high requirements for real-time performance.
[0008] Some business scenarios require real-time recording of the latest status of users. Usually, the database of the server corresponding to the business scenario is a relational database. The real-time status data is saved on the relational database. The server receives the status change information sent by the client and records the latest status of the user in the relational database.
[0009] The data or records in traditional relational databases can be updated or deleted. People can modify, update, or delete the real-time status data or records on the relational database, resulting in some key records being rewritten continuously, historical information being lost, and the system losing its traceability ability. When the situation of data being tampered occurs, it becomes difficult to track which data has been tampered with. Even more seriously, at some critical moments, it becomes impossible to restore the original data, which has become a major pain point in such applications.
[0010] In summary, neither the existing traditional databases nor the blockchain data structure can well meet the requirements of both data immutability, integrity, auditability, and high concurrent and reading / writing performance. Therefore, there is an urgent need for a new technical solution that can enhance the immutability ability of traditional databases on the basis of traditional databases while improving concurrent and reading / writing performance to adapt to the increasingly complex and diverse business needs. Summary of the Invention
[0011] An object of an embodiment of this application is to provide a data storage method, a server, and a storage medium to improve the technical problem that data in related technologies is easily tampered with.
[0012] In a first aspect, an embodiment of the present application provides a data storage method applied to a server, including: obtaining user data, parsing status data and non-status data from the user data, where the status data is data that is updated in real time, and the non-status data is data other than the status data in the user data. A preset status database is configured with a real-time status data table, and a preset non-status database is configured with a non-status data table, a non-real-time status data table, and a summary table. The status database is a database that supports addition, deletion, query, and modification functions, and the non-status database is a database that only supports query and addition functions. Writing the status data into the real-time status data table, writing the non-status data into the non-status data table, periodically responding to a status data persistence event, and writing the latest status data saved in the real-time status data table as a first new record into the non-real-time status data table of the non-status database. Periodically responding to a data immutability event, performing a hashing operation on the new data saved in the non-status database to obtain a first operation result, where the new data is data that has been newly added since the last periodic data immutability event. If the current data immutability event is the first data immutability event, the new data is all the current data in the non-status database. Generating a second new record based on the first operation result and writing the second new record into the summary table of the non-status database.
[0013] Optionally, the performing a hashing operation on the new data saved in the non-status database to obtain a first operation result includes: screening a target data table containing the new data in the non-status database, where the target data table is one of the non-status data table, the non-real-time status data table, and the summary table; forming a set of data tables to be hashed with all the target data tables; and performing a hashing operation on each target data table in the set of data tables to be hashed to obtain a first operation result corresponding to each target data table.
[0014] Optionally, the performing a hashing operation on each target data table in the set of data tables to be hashed to obtain a first operation result corresponding to each target data table includes: obtaining the new data in the target data table; putting the new data into a pre-hashing data set; concatenating the data in the pre-hashing data set into a pre-hashing string; and performing a hashing operation on the pre-hashing string to obtain a first operation result corresponding to each target data table.
[0015] Optionally, the pre-hashing data set further includes the previous result of the hashing operation on the target data table in the summary table; correspondingly, the concatenating the data in the pre-hashing data set into a pre-hashing string includes: concatenating the data in the pre-hashing data set with the previous result into a pre-hashing string.
[0016] Optionally, the information identifier of the last piece of historical information is included in the most recent record of the summary table regarding the target data table, where the last piece of historical information is the last piece of information included during the non-tampering event in the target data table, and the most recent record is a record whose time is before the second newly added record. Obtaining the newly added data in the target data table includes: collecting, based on the information identifier of the last piece of historical information, the information arranged after the last piece of historical information in the target data table to obtain the newly added data.
[0017] Optionally, generating the second newly added record based on the first operation result includes: determining the table name identifier of the target data table, where the table name identifier is used to identify the target data table; and forming the second newly added record regarding the target data table by combining the first operation result and the table name identifier.
[0018] Optionally, determining the table name identifier of the target data table includes: obtaining the table name of the target data table; and performing desensitization processing on the table name of the target data table to obtain the table name identifier.
[0019] Optionally, the newly added data includes multiple pieces of newly added information arranged in chronological order. Forming the second newly added record regarding the target data table by combining the first operation result and the identifier includes: finding the last piece of newly added information from the newly added data; and forming the second newly added record regarding the target data table by combining the first operation result, the table name identifier, and the last piece of newly added information.
[0020] Optionally, the method further includes: publicly announcing the summary table in real time.
[0021] Optionally, the method further includes: periodically responding to the summary chain-up event, performing a hashing operation on the new summary data in the summary table stored in the non-state database to obtain a second operation result; and writing the second operation result into the blockchain, where the new summary data is the newly added summary data since the last periodic summary chain-up event, and if this event is the first event, the new summary data is all the current data in the summary table.
[0022] In a second aspect, an embodiment of the present application provides a server, including a memory and a processor, where the memory is connected to the processor, and the processor is configured to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the server implements the above data storage method.
[0023] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to execute the above data storage method.
[0024] The embodiments of the present application can achieve the following technical effects: The data to be stored in the database in the embodiments of the present application are clearly classified into status data and non-status data, so that data with different characteristics can be adapted to different storage and management methods, which is the basis for achieving immutability. The embodiments of the present application adopt a dual-database architecture, where the status database is used to store status data and the non-status database is used to store non-status data. In terms of operation permission settings, the status database supports regular add, delete, query, and modification operations to meet the requirements of frequent updates of status data. The non-status database only supports add and query operations and prohibits delete and modification operations, thus ensuring the stability of non-status data and providing a basis for immutability. Status data is periodically transferred to the non-status database as new records to become non-real-time status data, retaining the data change history, which is crucial for data integrity and auditability. The embodiments of the present application adopt a summary table to periodically record summaries of new data in the non-status database, which can improve the immutability effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.
[0026] Figure 1 It is a schematic structural diagram of a data storage system provided by an embodiment of the present application; Figure 2 It is a schematic flowchart of a data storage method provided by an embodiment of the present application; Figure 3 It is a schematic diagram of a data storage scenario provided by an embodiment of the present application; Figure 4 It is a schematic structural diagram of a data storage device provided by an embodiment of the present application; Figure 5 It is a schematic structural diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0027] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts fall within the protection scope of the present application.
[0028] It should be noted that if there is no conflict, the various features in the embodiments of the present application can be combined with each other and all fall within the protection scope of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the flowchart. Furthermore, the terms "first", "second", "third", etc. adopted in the present application do not limit the data and the execution order, but only distinguish the same items or similar items with basically the same functions and effects.
[0029] Hereinafter, an embodiment of the present application provides a data storage system. Please refer to Figure 1 , the data storage system 100 includes an electronic device 11 and a server 12, and the electronic device 11 is communicatively connected to the server 12.
[0030] The electronic device 11 is installed with an application APP. After the user starts the application APP on the electronic device 11 and completes relevant operations on the interaction interface provided by the application APP, the application APP sends user data to the server 12 through the electronic device 11. The server 12 stores the user data by using the data storage methods provided in the following various embodiments.
[0031] Hereinafter, an embodiment of the present application provides a data storage method, which is applied to the server. Please refer to Figure 2 , the data storage method includes steps S21 to S28.
[0032] Step S21, obtain user data.
[0033] The user data is the data sent by the client to the server. For example, the client sends integral data, daily consumption data, user login data, etc. to the server.
[0034] Step S22, parse the status data and non-status data from the user data.
[0035] The status data is the data that is updated in real time. Among them, the real-time update frequency can be customized by the designer according to business requirements. For example, the status data is the inventory quantity of goods in an e-commerce system, the real-time status of an order, etc., and the real-time update frequency is 1 second or 2 seconds or when an update event arrives, etc.
[0036] Non-status data refers to the data in user data excluding status data. Generally, non-status data is data that does not change within a preset time period or system data, etc. For example, non-status data includes the basic information of a commodity, the fixed information when a user registers, etc.
[0037] For another example, when a user plays an online game, the client sends user data to the server. The user data consists of the following types of data: user information (nickname, avatar, etc.), score data, equipment acquisition data, current equipment bonus data, player position coordinates, data for task challenges, player resource status, etc. The user's own information, such as nickname and avatar, is usually fixed within a period of time. Therefore, the server does not update the user information in real time. In the embodiments of the present application, such data is regarded as non-status data. Usually, a player needs to complete certain tasks or meet certain trigger conditions to obtain equipment in the game. Similarly, when a player challenges a task, they also need to put in effort and time, which is not a high-frequency event. And for the fairness of the game, a traceable record needs to be provided at any time for auditing to prevent cheating. In the embodiments of the present application, such data is also regarded as non-status data. On the other hand, score data, current equipment bonus data, player position coordinates, player resource status, etc. usually record a certain current real-time status. Due to various factors, the frequency of update and reading may be very high, and it is necessary to update in real time and provide accurate perception to the player in real time. Therefore, in the embodiments of the present application, such data is regarded as status data.
[0038] The status database is a database that supports functions such as addition, deletion, query, and modification. The status database is configured with a real-time status data table, and the real-time status data table is used to record status data. It can be understood that the number of real-time status data tables can be one or more than two. Different types of real-time status data tables are used to record status data of different business types.
[0039] The non-status database is a database that only supports addition and query functions. The non-status database is configured with non-status data tables, non-real-time status data tables, and summary tables. It can be understood that the number of non-status data tables can be one or more than two. Different types of non-status data tables are used to record non-status data of different business types. The number of non-real-time status data tables can be the same as or different from the number of real-time status data tables. The non-real-time status data table can correspond to the real-time status data table in terms of data structure, or can be redesigned to optimize storage or support other purposes, or can be missing or partially missing. The number of summary tables can be one or more than two, usually one. When there are more than two, each summary table can be configured according to different rules to store summary data of different business types; when there is only one, different rules can be used to use custom fields to distinguish and store summary data records of different business types, such as the "table name" field. The fields of the summary table usually contain record identifiers, record timestamps, and record values. In addition, new fields can be added according to specific needs to optimize the usage effect.
[0040] In the embodiments of the present application, the data to be stored in the database is clearly classified into status data and non-status data. This classification method is the basis for all subsequent improvement measures, and can adopt more appropriate storage and management strategies according to the characteristics of different types of data.
[0041] In the embodiments of the present application, a method of storing data in two databases is adopted. The status database is specifically used to store status data, while the non-status database is responsible for storing non-status data. In terms of operation permission settings, the status database supports regular addition, deletion, query, and modification operations to meet the needs of frequent updates of status data. The non-status database only supports addition and query operations, and deletion and modification operations are prohibited, thus ensuring the stability of non-status data and providing a common rule basis for non-tamperability.
[0042] It should be noted that the common rule here refers to the common specification to be followed in accordance with the embodiments of the present application. Those who violate this specification will become a special minority and will be detected and falsified by the functional mechanism formed by the embodiments of the present application.
[0043] Step S23, write the status data into the real-time status data table.
[0044] The real-time status data table is used to record the real-time status of the recognized content. For example, when recording the status data with the user as the recognition object, there is one record corresponding to one user's status data. The server can assign an id to each record as the unique identifier of the record. In the above embodiment with the user as the recognition object, the server needs to assign a unique identifier to the user, and there should be no duplicate users in the record set of the real-time status data table. When the user generates corresponding status data, if there is no previous record for the user, the server will add a new record for the user after the current last record. When there is already a record for the user, the server will update the value of the record on the existing record instead of adding a new record. The server can also update other information that may need to be updated. Please refer to Table 1: Table 1: playerStatus
[0045] Table 1 demonstrates an embodiment of a real-time status data table with the table name playerStatus. There are three fields in the table: id, name, and status. Among them, id is the unique identifier of the record, status is the value of the record, and name is the unique identifier of the user. The data in the field with the unique identifier attribute cannot have duplicate records. This setting here can ensure that each user has a definite real-time status.
[0046] Step S24, write the non-status data into the non-status data table.
[0047] The non-status data table is used to record non-status data. Please refer to Table 2: Table 2: Player
[0048] Table 2 demonstrates an embodiment of a non-status data table. Under the solution of this embodiment of the present application, the operation rules of only allowing addition and query and not allowing deletion and modification need to be followed. Therefore, if there is a second data write for the same user, a new record must be added instead of updating the old record.
[0049] The table name of the data table in Table 2 is Player. There are four fields in the table: id, name, data, and timestamp. Among them, id is the unique identifier of the record; data is the value of the record; name is the user's name and is not a unique identifier; non-status data tables usually contain a timestamp field to form traceability. As can be seen from Table 2, for User A, there are two records with ids 0 and 2 respectively, and the data values of the records are different, indicating that the user has updated the avatar, with an interval of about 7 days; for User B, there are two records with ids 1 and 4 respectively, and the data values of the records are different, indicating that the user has updated the nickname, with an interval of about 3 months; for User C, there are two records with ids 3 and 5 respectively, and the data values of the records are different, indicating that the user has updated the avatar, with an interval of about 10 months. In this demonstration, the function of traceability is reflected.
[0050] Step S25, regularly respond to the status data persistence event, and write the latest status data saved in the real-time status data table as the first new record into the non-real-time status data table of the non-status database.
[0051] In some embodiments, to improve the performance of high-frequency updates and simple numerical queries, the status database usually uses a NOSQL type in-memory database. To prevent data loss in this type of database when the computer powers off, the data needs to be transferred to a more durable medium. The status data persistence event is an event that transfers and solidifies the status data to a persistent storage device (such as a hard disk). Note that the transfer described in the embodiments of the present application does not involve the part of deleting the original data, but is a backup storage operation. The server triggers the status data persistence event according to a preset transfer time interval, and the transfer time interval can be 6 hours or 24 hours or 7 days or any other time interval.
[0052] The latest status data is the data in the real-time status data table. The meaning of "latest" is to illustrate that the data version at this time is in the latest position on the time line. Since the data in the real-time status data table is always the latest data at that time, there is no need to make a special selection in actual operation, and only need to transfer it on the spot.
[0053] When the data is transferred to the non-real-time status data table of the non-status database, the data structure may change because the unique identification restriction on a certain identification content will be removed. As shown in Table 3, continuing the embodiment scenario of Table 1 before, in the non-real-time status data table, the field of the user name no longer has a uniqueness restriction, indicating that the user can have multiple records in this table. In fact, the user not only appears repeatedly, but also in an infinite loop mode; in a more optimized embodiment solution, some users who have not had relevant information updates recently may be skipped.
[0054] Table 3: PlayerStatus
[0055] The embodiment demonstrated in Table 3 is a non-real-time status data table in a non-status database, with the table name PlayerStatus, serving as the corresponding persistent transfer table of Table 1. As can be seen from Table 3, the data records generated by the latest round of persistent events will appear at the bottom of the data table. For the convenience of subsequent work, such as tracing and auditing, this type of non-real-time status data table will also include a timestamp field.
[0056] As mentioned above, the non-status data saved in the non-status database is not updated in real time, while the status data saved in the status database is updated in real time. To record the historical status data of the real-time status data table, the embodiments of the present application periodically transfer the latest status data of the real-time status data table to the non-real-time status data table, which can reduce the high-frequency access to the non-status database and improve the data reading and writing efficiency.
[0057] Step S26, periodically respond to the data non-tampering event, perform a hashing operation on the newly added data saved in the non-status database, and obtain the first operation result.
[0058] The data non-tampering event is an event for performing non-tampering processing on the newly added data in the non-status database. The server triggers the data non-tampering operation event at a preset frequency. For example, every 24 hours or 7 days, the server performs the data non-tampering operation.
[0059] In this embodiment, the newly added data is the data newly added after the previous periodic data non-tampering event. If the current data non-tampering event is the first data non-tampering event, the newly added data is all the current data.
[0060] In some embodiments, the newly added data may include old data that has been included in previous non-tampering events, which does not affect the effectiveness of the overall method framework.
[0061] Generally, the newly added data in the non-status database mainly comes from the business data of the non-status data table and the non-real-time status data table (business data refers to data that does not come from the data generated for implementing the embodiments of the present application, such as the data in the summary table), and a small amount comes from the new records generated by the non-tampering event in the summary table. In special cases, when there is no newly added data in both the non-status data table and the non-real-time status data table, the summary table may or may not generate new records, depending on whether the preset rules of the data non-tampering event allow the newly added data to form records.
[0062] New records in the non-status database. For the non-status data table, compared with Table 2, in the embodiments of the present application, non-status data of User Ding, User Wu, and User Jia are newly added to the non-status data table to obtain Table 4.
[0063] Table 4: Player
[0064] In Tables 3 and 4, the newly added data part is distinguished by using italic font style.
[0065] In the embodiments of the present application, hashing operations are used to reduce the amount of large data and retain its specificity for storage and later differentiation.
[0066] In some embodiments, the steps of obtaining the first operation result by performing hashing operations on the newly added data saved in the non-status database include: combining all the latest data of the non-status data table and the non-real-time status data table, and the newly added records generated when the summary table performed the non-tampering operation last time to obtain combined data, and performing hashing operations on the combined data to obtain the first operation result. For example, in the embodiments of the present application, the italic-style data in Table 3, the italic-style data in Table 4, and the previous record in the summary table are combined to obtain combined data, and hashing operations are performed on the combined data based on the hash algorithm to obtain the first operation result.
[0067] In some other embodiments, the hashing operations are processed separately for each data table. The steps of obtaining the first operation result by performing hashing operations on the newly added data saved in the non-status database include: screening the target data tables containing the newly added data in the non-status database, where the target data table is one of the non-status data table, the non-real-time status data table, and the summary table, forming a set of data tables to be hashed with all the target data tables, and performing hashing operations on each target data table in the set of data tables to be hashed to obtain the first operation result corresponding to each target data table.
[0068] For example, the non-status database is configured with a first non-status data table Player, a second non-status data table F2, a third non-status data table F3, a first non-real-time status data table X1, a second non-real-time status data table PlayerStatus, a third non-real-time status data table X3, and a summary table Digest.
[0069] Compared with the previous record operation, there are new non-status data added to both the first non-status data table Player and the second non-status data table F2. There is no new non-status data added to the third non-status data table F3. There are new status data added to both the second non-real-time status data table PlayerStatus and the third non-real-time status data table X3. There is no new status data added to the first non-real-time status data table X1.
[0070] Therefore, the first non-status data table Player, the second non-status data table F2, the second non-real-time status data table PlayerStatus, and the third non-real-time status data table X3 are all target data tables.
[0071] In addition, since there are new status data added to the first non-status data table Player, the second non-status data table F2, the second non-real-time status data table PlayerStatus, and the third non-real-time status data table X3, and the second new records corresponding to the first non-status data table Player, the second non-status data table F2, the second non-real-time status data table PlayerStatus, and the third non-real-time status data table X3 are all written into the summary table Digest, there are also new data in the summary table Digest. The new data in the summary table Digest are the second new records corresponding to the first non-status data table Player, the second non-status data table F2, the second non-real-time status data table PlayerStatus, and the third non-real-time status data table X3. Therefore, the summary table Digest is also a target data table.
[0072] Therefore, the first non-status data table Player, the second non-status data table F2, the second non-real-time status data table PlayerStatus, the third non-real-time status data table X3, and the summary table Digest are all target data tables.
[0073] In the embodiment of the present application, the first non-status data table Player, the second non-status data table F2, the second non-real-time status data table PlayerStatus, and the third non-real-time status data table X3 are added to the set of data tables to be hashed. Therefore, the set of data tables to be hashed Q = {Player, F2, PlayerStatus, X3, Digest}.
[0074] The embodiments of the present application sequentially perform hash operation processing on the first non-status data table Player, the second non-status data table F2, the second non-real-time status data table PlayerStatus, the third non-real-time status data table X3, and the summary table Digest, and respectively obtain the first operation result corresponding to the first non-status data table Player, the first operation result corresponding to the second non-status data table F2, the first operation result corresponding to the second non-real-time status data table PlayerStatus, the first operation result corresponding to the third non-real-time status data table X3, and the first operation result corresponding to the summary table Digest.
[0075] The summary table includes a record identifier, a record timestamp, and record content. The record identifier is the record id (unique identifier) and is used to identify this record. The record timestamp is the time when this record is written into the summary table, and the record content is the first operation result corresponding to the target data table.
[0076] The embodiments of the present application sequentially generate a new record for each target data table in the summary table. Its record id is automatically generated by the preset rule for adding new records in the database, usually the numerical value of the previous id is increased by 1, and the first operation result and the record timestamp of the target data table are written into the corresponding fields of the new record.
[0077] Table 5: Digest
[0078] As shown in the summary table embodiment of Table 5, the table name is Digest, the id is the field name of the record unique identifier, and the record id is stored here; hash is the field name of the record content, and the first operation result is stored here; created is the field name of the record timestamp, and the exact time at this moment is stored here. In addition, in the scenario of this embodiment, there is also a table field that stores the name of the target data table as a table name identifier. This identifier has no uniqueness restriction because records will be cyclically generated for each target data table here.
[0079] It can be understood that in Table 5, the ids from 0 to 67 are all historical records of the summary table. For the sake of brief display, Table 5 does not show the records with record ids from 0 to 67, but only shows the latest records with ids from 68 to 72. Among them, record 68 is the record corresponding to the first non-status data table Player, record 69 is the record corresponding to the second non-status data table F2, record 70 is the record corresponding to the second non-real-time status data table PlayerStatus, record 71 is the record corresponding to the third non-real-time status data table X3, and record 72 is the record corresponding to the summary table Digest.
[0080] The embodiments of the present application generate a first operation result for each target data table, and lock and make each target data table non-tamperable, which is conducive to more accurately and reliably tracking the tampering changes of each target data table subsequently, thereby improving the auditability of the target data table.
[0081] Performing a hashing operation process on each target data table in the hash data table set to obtain a first operation result corresponding to each target data table includes the following steps: obtaining the newly added data in the target data table, putting the newly added data into a pre-hash data set, concatenating the data in the pre-hash data set into a pre-hash string, and performing a hashing operation on the pre-hash string to obtain a first operation result corresponding to each target data table.
[0082] For example, referring to Table 3, the newly added data in the target data table PlayerStatus in this embodiment includes the first data with id = 3, the second data with id = 4, and the third data with id = 5.
[0083] The embodiments of the present application put the first data D1, the second data D2, and the third data D3 into the pre-hash data set W, and concatenate the data in the pre-hash data set W into a pre-hash string Sn according to a preset rule. For example, D1 + D2 + D3 is concatenated in order to form the pre-hash string Sn, or D2 + D1 + D3 is concatenated in order to form the pre-hash string Sn, or D3 + D2 + D1 is concatenated in order to form the pre-hash string Sn, etc. The embodiments of the present application perform a hashing operation on the pre-hash string Sn to obtain a first operation result corresponding to the target data table.
[0084] In some embodiments, the pre-hash data set further includes the previous result of the last hashing operation on the target data table in the summary table. Correspondingly, concatenating the data in the pre-hash data set into a pre-hash string includes the following steps: concatenating the data in the pre-hash data set with the previous result into a pre-hash string.
[0085] For example, the pre-hash data set W contains the first data D1, the second data D2, the third data D3, and the previous result H01. The data in the pre-hash data set W is concatenated into a pre-hash string Sn according to a preset rule, that is, Sn = D1 + D2 + D2 + H01. The embodiments of the present application perform a hashing operation on the pre-hash string to obtain a first operation result corresponding to each target data table.
[0086] In some embodiments, the information identifier of the last piece of historical information is included in the most recent record of the summary table regarding the target data table. The last piece of historical information is the last piece of information included when the non-tamperable event occurs in the target data table, and the most recent record is the record in the summary table whose time is before the second newly added record, as shown in Table 6: Table 6: Digest
[0087] As shown in Table 6, the digest table is named Hash, and the LRID is the information identifier of the last historical information.
[0088] Obtaining the newly added data in the target data table includes the following steps: Based on the information identifier of the last historical information, collect the information arranged after the last historical information in the target data table to obtain the newly added data. For example, please refer to Table 3 and Table 6. The latest record in the digest table Digest regarding the target data table PlayerStatus is the record with id = 70. When the last non-tampering event occurred in the target data table PlayerStatus recorded by it, the last historical information in the newly added data is the data of user C with id = 2. Therefore, the value of the LRID field in record 70 of the digest table is 2. In the embodiment of the present application, the information identifier of the last historical information corresponding to each target data table is added to the digest table, which is beneficial to quickly identifying the newly added data in the target data table and improving the data storage efficiency.
[0089] Step S27, generating a second newly added record based on the first operation result.
[0090] Generating a second newly added record based on the first operation result includes the following steps: determining the table name identifier of the target data table, where the table name identifier is used to identify the target data table, and forming a second newly added record regarding the target data table by combining the first operation result with the table name identifier.
[0091] Determining the table name identifier of the target data table includes the following steps: obtaining the table name of the target data table and performing desensitization processing on the table name of the target data table to obtain the table name identifier. In the embodiment of the present application, by performing desensitization processing on the table name of the target data table, for example, converting the table name Player to F1, converting the table name PlayerStatus to X2, converting the table name Digest to Z0, etc., please refer to the difference between Table 6 and Table 7. Users cannot distinguish which part of the content belongs to the target data table through the digest table, so as to improve the privacy of the content of the target data table recorded in the digest table.
[0092] The newly added data includes multiple newly added information arranged in chronological order. Combining the first operation result with the identifier to form a second newly added record regarding the target data table includes the following steps: finding the last newly added information from the newly added data, and combining the first operation result, the table name identifier, and the last newly added information to form a second newly added record regarding the target data table.
[0093] Please combine with Table 6. In the records of the non-status data table Player(F1) in the Digest of the embodiments of the present application, retrieve the most recent record, that is, the record with id = 68. Based on the last piece of historical information, that is, the value of LRID is 5, return to the non-status data table Player(F1), query the record with id = 5. Based on the records after this record, determine that the information with id = 6, id = 7, and id = 8 are all newly added data. Then, generate a new first operation result based on the information with id = 6, id = 7, and id = 8. Among them, the id of the last record, 8, is the value of the LRID field of the new record written into the Digest table. Update Table 6 to obtain Table 7 as shown below: Table 7: Digest
[0094] As shown in Table 7, the Digest table, with the table name Digest, after the desensitization process of the table name field as described above, the embodiments of the present application generate a new first operation result Hash6 based on the information with id = 6, id = 7, and id = 8. Among them, the information identifier LRID of the last piece of historical information of this non-tamperable event is 8.
[0095] Step S28, write the second newly added record into the Digest table of the non-status database.
[0096] As shown in Table 7, write the new first operation result Hash6 into the hash field of the second newly added record, write 8 into the information identifier LRID field of the last newly added information, write the time stamp T6 into the time stamp field created, write the desensitized non-status data table name F1 into the table field. The system automatically assigns a new id = 73 as the record id of the second newly added record, and write this second newly added record into the Digest table of the non-status database.
[0097] It can be understood that data non-tamperability is one of the core improvements of the embodiments of the present application. The non-status database will regularly perform non-tamperable operations on newly added data, including a Digest table for recording each non-tamperable operation. As described above, the embodiments of the present application perform hash digest calculation on the newly added data after the last operation together with the Digest recorded in the last operation, and then record the calculation result together with the time stamp and the new id into the Digest table.
[0098] To further optimize security and management convenience, the newly added data can be recorded separately for each data table, including the data table storing non-real-time status data, the data table storing non-status data, and the Digest table itself. At this time, the records of the Digest table will also include the table name identifier of the data table being recorded. And, for greater security, the relevant data can be desensitized first to avoid leakage of sensitive information.
[0099] In some embodiments, the method further includes the following steps: publishing the abstract table in real time. To ensure non-tamperability, the embodiments of the present application publish the content of the abstract table on the Internet in real time to form a distributed record. This distributed record method makes it extremely difficult for any party to tamper with the data because tampering requires modifying the data on multiple distributed nodes simultaneously, greatly increasing the cost and difficulty of tampering.
[0100] In some embodiments, the method further includes the following steps: regularly responding to the event of the abstract being uploaded to the blockchain, performing a hashing operation on the new abstract data in the abstract table stored in the non-state database to obtain a second operation result, and writing the second operation result into the blockchain. The new abstract data is the newly added abstract data since the last regular abstract upload event. If this event is the first event, the new abstract data is all the current data in the abstract table.
[0101] The embodiments of the present application can regularly upload the abstract table to the blockchain. By leveraging the non-tamperable feature of the blockchain itself, the difficulty of data tampering is further enhanced, ensuring data security.
[0102] To elaborate in detail on the data storage method provided by the embodiments of the present application, the following is a detailed description in conjunction with Figure 3 as follows: Please refer to Figure 3 , the server is configured with a state database 31 and a non-state database 32. The state database 31 is configured with a first real-time status data table Y1, a second real-time status data table Y2, and a third real-time status data table Y3. The non-state database is configured with a first non-state data table F1, a second non-state data table F2, a third non-state data table F3, a first non-real-time status data table X1, a second non-real-time status data table X2, a third non-real-time status data table X3, and an abstract table Z0. The first real-time status data table Y1 corresponds to the first non-real-time status data table X1, the second real-time status data table Y2 corresponds to the second non-real-time status data table X2, and the third real-time status data table Y3 corresponds to the third non-real-time status data table X3.
[0103] ① The server writes the first latest non-state data into the first non-state data table F1, writes the second latest non-state data into the second non-state data table F2, and writes the third latest non-state data into the third non-state data table F3.
[0104] ② The server responds to the first state data persistence event 33 and writes the first latest status data stored in the first real-time status data table Y1 as a first new record into the first non-real-time status data table X1.
[0105] The server responds to the second status data persistence event 34, and writes the second latest status data saved in the second real-time status data table Y2 as the first new record into the second non-real-time status data table X2.
[0106] The server responds to the third status data persistence event 35, and writes the third latest status data saved in the third real-time status data table Y3 as the first new record into the third non-real-time status data table X3.
[0107] ③ The server responds to the first data non-tampering event 36, performs a hashing operation on the new data saved in the first non-real-time status data table X1 to obtain the first operation result, generates the second new record 37 based on the first operation result, and writes the second new record 37 into the summary table Z0.
[0108] The server responds to the second data non-tampering event 38, performs a hashing operation on the new data saved in the second non-real-time status data table X2 to obtain the first operation result, generates the second new record 39 based on the first operation result, and writes the second new record 39 into the summary table Z0.
[0109] The server responds to the third data non-tampering event 310, performs a hashing operation on the new data saved in the third non-real-time status data table X3 to obtain the first operation result, generates the second new record 311 based on the first operation result, and writes the second new record 311 into the summary table Z0.
[0110] The server responds to the fourth data non-tampering event 312, performs a hashing operation on the new data saved in the first non-status data table F1 to obtain the first operation result, generates the second new record 313 based on the first operation result, and writes the second new record 313 into the summary table Z0.
[0111] The server responds to the fifth data non-tampering event 314, performs a hashing operation on the new data saved in the second non-status data table F2 to obtain the first operation result, generates the second new record 315 based on the first operation result, and writes the second new record 315 into the summary table Z0.
[0112] The server responds to the sixth data non-tampering event 316, performs a hashing operation on the new data saved in the third non-status data table F3 to obtain the first operation result, generates the second new record 317 based on the first operation result, and writes the second new record 317 into the summary table Z0.
[0113] The server responds to the seventh data non-tampering event 318, performs a hashing operation on the newly added data stored in the summary table Z0 to obtain a first operation result, generates a second new record 319 based on the first operation result, and writes the second new record 319 into the summary table Z0.
[0114] Through the above series of improvement measures, the embodiment of the present application not only adds the non-tampering ability to the traditional database, realizes the same data right confirmation, traceability, auditing and natural credibility as the blockchain, but also retains the highly developed data processing capabilities of traditional databases such as relational databases, such as joining tables for query, efficient reading and writing, etc. Moreover, due to the separation of state data and other data and the adoption of the design of only increment and query, the parallel processing transaction ability and reading and writing efficiency of relational databases are significantly improved, effectively solving the problems existing in traditional databases in many aspects.
[0115] In order to elaborate in detail on the data storage method provided by the embodiment of the present application, the embodiment of the present application combines the specific steps and methods of the following examples to implement the improvement of the traditional database, so as to achieve the goals of enhancing the non-tampering ability, improving data integrity and auditability, and optimizing concurrency and reading and writing performance, as follows: ① Data classification and database initialization a. When the system starts, the data classification module is initialized. When user data enters the storage process, the data classification module determines the data type of the user data according to preset rules. For example, in the e-commerce business scenario, order status (such as pending payment, paid, shipped, etc.), real-time sales volume of goods, etc. will be determined as state data; while product name, description, merchant information, etc. are classified as non-state data.
[0116] b. At the same time, the state database and the non-state database are initialized. The state database is configured to support add, delete, query and modify operations, while the non-state database is set to only support add and query operations, restricting the delete and modify permissions for non-state data at the database level.
[0117] ② State data processing flow c. During daily business operations, the state data is updated in the state database in real time. For example, in an online shopping mall system, when a new order is generated, the "pending payment" status record of the order will be added to the state database; when the user completes the payment and the order status is updated to "paid", the corresponding record in the state database is modified.
[0118] d. The state database transfers the state data to the non-state database in the form of new records according to the set time period (such as every hour, every day, etc., and the specific period can be flexibly adjusted according to business requirements). For example, at 2 o'clock in the morning every day, the system will add the latest status of all orders with status updates in the previous day as non-real-time state data to the non-state database.
[0119] ③Non-status data processing flow e. Non-status data is directly stored in the non-status database when it is generated. For example, the merchant information submitted when a new merchant enters the platform is directly saved to the non-status database. Since the non-status database only supports addition and query operations, the entered data cannot be directly modified or deleted. When the existing merchant information changes, the system will add new records to the relevant non-status data tables in the non-status database.
[0120] f. The non-status database periodically (such as at regular time intervals or when a certain data volume is reached) performs an immutability operation on the newly added data. Taking the product information table (i.e., the target data table) of a certain e-commerce platform as an example, assume that an immutability operation is performed every time 100 pieces of product information are newly added. The system will obtain the 100 pieces of product information data newly added after the last operation, as well as the digest value recorded in the digest table during the last operation, and perform a hash digest calculation together (a mature hash algorithm such as SHA - 256 can be selected). After calculating the new hash value (i.e., the first operation result), a unique new record identifier is generated, and the timestamp of the current operation is recorded. The new hash value, timestamp, new record identifier, and the table name identifier of the product information table are recorded in the digest table together.
[0121] ④Digest table management and distributed guarantee g. When the digest table records each immutability operation, in addition to the information mentioned above, it separately records for different types of target data tables (data tables storing non-real-time status data, data tables storing non-status data, and the digest table itself). For example, for the order status change table storing non-real-time status data, the digest table will clearly identify the target data table when recording its relevant operation information.
[0122] h. The content of the digest table is published online in real time, which can be achieved by building a distributed server network. Each distributed node synchronizes the digest table information in real time to ensure data consistency and wide dissemination. For example, server nodes are deployed in multiple regions around the world, and each node can obtain and store the latest content of the digest table in real time.
[0123] i. To further strengthen immutability, the digest table can be chained on the blockchain according to a set time period (such as weekly, monthly, etc.). Select a suitable blockchain platform (such as a consortium chain, etc., selected according to the business's requirements for privacy and performance), encapsulate the digest data of the digest table according to the format and rules of the blockchain, add it to the blocks of the blockchain, and use the consensus mechanism and encryption technology of the blockchain to ensure data immutability.
[0124] ⑤Data query process j. When a query request is initiated, the system first determines the type of data to be queried. If it is status data, the query is directly performed in the status database. For example, to query the real-time status of a certain order currently, the result can be quickly obtained from the status database.
[0125] k. If the data to be queried is non-status data or non-real-time status data, the query operation is performed in the non-status database. Since the non-status database only supports addition and query and the data has the property of non-tamperability, the query operation is more efficient and stable. For example, to query the historical price record of a certain commodity (non-real-time status data) or the basic information of a commodity (non-status data), the result can be quickly returned from the non-status database.
[0126] Through the above specific implementation manners, each module and process cooperate closely, achieving an overall improvement in aspects such as non-tamperability, data integrity, auditability, and concurrent and read-write performance of the traditional database.
[0127] Based on the discussions of the above various embodiments, the embodiments of the present application at least have the following differentiating points: 1) Data classification strategy: Accurately distinguishing status data from non-status data is the cornerstone of the entire solution. This classification enables data with different characteristics to be adapted to different storage and management methods, which is a prerequisite for achieving goals such as non-tamperability and performance optimization.
[0128] 2) Dual-database architecture and operation permission setting: The separate design of the status database and the non-status database, and the respectively granted permissions of addition, deletion, query, modification and only addition and query, ensure the stability and non-tamperability of data from the architectural level, while improving the concurrent processing ability.
[0129] 3) Status record persistence mechanism: Status data is periodically transferred and stored as new records in the non-status database to become non-real-time status data, retaining the history of data changes, which is crucial for data integrity and auditability.
[0130] 4) Data non-tamperability process: The non-tamperability operation of the non-status database on newly added data, including steps such as calculating a new hash value in combination with the previous digest, timestamp, and new record identifier into the digest table, is the core process for achieving data non-tamperability.
[0131] 5) Digest table management and distributed guarantee: The digest table details the non-tamperability operation information, which is publicly announced in real time on the Internet and periodically uploaded to the blockchain, strengthening the non-tamperability of data and ensuring data security.
[0132] 6) Performance optimization means: Utilizing a NoSQL key-value pair in-memory database such as Redis to improve the performance of the status database further optimizes the overall read-write performance.
[0133] 7) Data classification and dual-database construction method: covering the classification and determination rules for status data and non-status data, as well as the initialization and configuration methods for the status database and non-status database, preventing others from adopting similar data classification and database construction strategies without authorization.
[0134] 8) Status data processing flow: including the update method of status data in the status database, as well as the specific steps and time period settings for periodically archiving non-real-time status data to the non-status database, protecting this process from being copied.
[0135] 9) Non-status data non-tampering operation: from the selection of new data, combining with the previous digest to calculate the hash value, to the complete operation process of digest table records, preventing others from using the same non-tampering implementation means.
[0136] 10) Digest table management and the combined mechanism of distributed and blockchain: including the composition of the digest table record information, the real-time publication method, and the process and cycle settings for blockchain on-chain, protecting this management and security mechanism from being imitated.
[0137] 11) Performance optimization implementation method: the specific integration method of using a NoSQL in-memory database (such as Redis) to improve the performance of the status database, data caching strategy, etc., preventing others from using similar performance optimization means without permission.
[0138] The dual-database collaborative management solution proposed in the embodiments of this application brings significant beneficial effects to database applications from multiple dimensions, specifically as follows: In terms of data security and integrity: A. Enhancement of non-tampering ability: By storing non-status data in a non-status database that only supports addition and query operations, and performing regular non-tampering processing on its new data, combined with digest table records, distributed publication, and blockchain on-chain, etc., a multi-level data non-tampering guarantee system is constructed. This makes it almost impossible for critical data in the database to be maliciously tampered with once recorded, greatly improving the security and credibility of the data. For example, in the scenario of financial transaction records, after each transaction data is stored as non-status data, its integrity and authenticity can be effectively guaranteed for a long time, preventing financial risks caused by data tampering.
[0139] B. Improvement of data integrity and auditability: Status data is periodically archived as non-real-time status data and updated in the form of new records, retaining the complete historical track of data changes. Combining with the digest table to record detailed information of each non-tampering operation, including timestamp, data table identifier, etc., enables precise tracing of each data change when auditing is required, clarifying key information such as the change time and the data tables involved, providing strong support for data compliance review and problem troubleshooting.
[0140] C. Enhanced concurrent processing ability: The data is divided into state data and non-state data and stored in databases with different characteristics respectively, reducing resource competition during data operations. The state database focuses on processing frequently updated state data, and the non-state database processes relatively static non-state data. This separation method enables the database to allocate resources more efficiently when processing concurrent transactions and reduces the occurrence of lock contention. For example, during high-concurrency e-commerce promotion activities, the update of order status (state data) and the query of basic product information (non-state data) can be carried out simultaneously and efficiently without interference, significantly improving the system's processing ability in high-concurrency scenarios.
[0141] D. Improved read and write performance: The non-state database only supports insert and query operations, and its data structure and operation logic are relatively simple, providing a more efficient execution environment for query operations. At the same time, the centralized management and optimized update strategy of the state database for state data also make the read and write operations of state data more fluent. In addition, NoSQL key-value pair in-memory databases such as Redis can be used to further improve the performance of the state database. Redis is based on in-memory storage and has extremely high read and write speeds. For frequently updated and queried state data, using Redis as part of the state database can cache hot data in it, greatly reducing disk I / O operations, thus significantly improving the read and write performance of state data. For example, in scenarios such as real-time statistics of the number of online users and real-time sales volume of products, using Redis's fast read and write capabilities can quickly respond to data update and query requests, bringing a qualitative leap to the overall performance of the system. This way of combining relational databases with NoSQL in-memory databases gives full play to the advantages of both and comprehensively improves the read and write performance of the database, providing users with a faster data access experience.
[0142] F. Combining the advantages of traditional databases and blockchain: The present invention not only retains the powerful capabilities of traditional databases such as relational databases in data processing, such as complex join queries and efficient data read and write, but also integrates the characteristics of blockchain technology in data immutability, rights confirmation, and traceability. This combination enables the database system to not only meet the requirements of daily business for data processing efficiency but also reach a high standard in terms of data security and credibility, and is applicable to various business scenarios with strict data requirements, such as medical and government affairs fields.
[0143] G. Good scalability: Whether it is a state database or a non-state database, their architecture designs have good scalability. As the volume of business data grows, the system performance can be easily extended by adding server nodes, optimizing the storage structure, etc. At the same time, the distributed storage of the summary table and the blockchain on-chain mechanism can also be flexibly adjusted and extended according to business needs to adapt to business development of different scales.
[0144] In summary, through innovative design and technology integration, the embodiments of the present application have brought significant improvements in multiple aspects to traditional databases, providing more powerful, secure, and efficient support for various business systems that rely on databases.
[0145] It should be noted that in the above various embodiments, there is not necessarily a certain order among the above steps. Those of ordinary skill in the art can understand according to the description of the embodiments of the present application that in different embodiments, the above steps can have different execution orders, that is, they can be executed in parallel or exchanged, etc.
[0146] As another aspect of the embodiments of the present application, the embodiments of the present application provide a data storage device. Among them, the data storage device can be a software module, and the software module includes several instructions, which are stored in a memory. The processor can access this memory and call the instructions for execution to complete the data storage methods described in the above various embodiments.
[0147] In some embodiments, the data storage device can also be built by hardware devices. For example, the data storage device can be built by one or more than two chips, and each chip can work in coordination with each other to complete the data storage methods described in the above various embodiments. For another example, the data storage device can also be built by various logic devices, such as being built by a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.
[0148] Please refer to Figure 4 , the data storage device 400 includes: a data acquisition module 41, a data parsing module 42, a first writing module 43, a second writing module 44, a first adding module 45, an operation module 46, a second adding module 47, and a summary update module 48.
[0149] The data acquisition module 41 is used to acquire user data. The data parsing module 42 is used to parse status data and non-status data from the user data. The status data is data that is updated in real time, and the non-status data is data other than the status data in the user data. The preset status database is configured with a real-time status data table, and the preset non-status database is configured with a non-status data table, a non-real-time status data table, and a summary table. The status database is a database that supports functions of addition, deletion, query, and modification, and the non-status database is a database that only supports functions of query and addition. The first writing module 43 is used to write the status data into the real-time status data table. The second writing module 44 is used to write the non-status data into the non-status data table. The first adding module 45 is used to periodically respond to the status data persistence event, and write the latest status data saved in the real-time status data table as a first added record into the non-real-time status data table of the non-status database. The operation module 46 is used to periodically respond to the data immutability event, perform a hashing operation on the newly added data saved in the non-status database to obtain a first operation result. The newly added data is data that has been newly added since the last periodic data immutability event. If the current data immutability event is the first data immutability event, the newly added data is all the current data in the non-status database. The second adding module 47 is used to generate a second added record based on the first operation result. The summary update module 48 is used to write the second added record into the summary table of the non-status database.
[0150] In some embodiments, the operation module 46 is specifically configured to: screen a target data table containing newly added data in the non-status database, where the target data table is one of the non-status data table, the non-real-time status data table, and the summary table, form a set of data tables to be hashed with all the target data tables, and perform a hashing operation on each target data table in the set of data tables to be hashed to obtain a first operation result corresponding to each target data table.
[0151] In some embodiments, the operation module 46 is further specifically configured to: obtain the newly added data in the target data table, put the newly added data into a pre-hashing data set, splice the data in the pre-hashing data set into a pre-hashing string, and perform a hashing operation on the pre-hashing string to obtain a first operation result corresponding to each target data table.
[0152] In some embodiments, the pre-hashing data set further includes the previous result of the hashing operation on the target data table in the summary table. Correspondingly, the operation module 46 is further specifically configured to: splice the data in the pre-hashing data set with the previous result into a pre-hashing string.
[0153] In some embodiments, the information identifier of the last piece of historical information is included in the most recent record of the summary table regarding the target data table, where the last piece of historical information is the last piece of information included during the non-tamperable event in the target data table, and the most recent record is a record whose time is before the second newly added record. The operation module 46 is further specifically configured to: based on the information identifier of the last piece of historical information, collect the information arranged after the last piece of historical information in the target data table to obtain newly added data.
[0154] In some embodiments, the second newly added module 47 is specifically configured to: determine the table name identifier of the target data table, where the table name identifier is used to identify the target data table, and form the second newly added record regarding the target data table by combining the first operation result and the table name identifier.
[0155] In some embodiments, the second newly added module 47 is further specifically configured to: obtain the table name of the target data table, and perform desensitization processing on the table name of the target data table to obtain the table name identifier.
[0156] In some embodiments, the newly added data includes multiple pieces of newly added information arranged in chronological order. The second newly added module 47 is further specifically configured to: find the last piece of newly added information from the newly added data, and form the second newly added record regarding the target data table by combining the first operation result, the table name identifier, and the last piece of newly added information.
[0157] In some embodiments, the second newly added module 47 is further specifically configured to: publicly announce the summary table in real time.
[0158] In some embodiments, the second newly added module 47 is further specifically configured to: periodically respond to the summary on-chain event, perform a hashing operation on the new summary data in the summary table stored in the non-state database to obtain a second operation result, and write the second operation result into the blockchain. The new summary data is the newly added summary data since the last periodic summary on-chain event. If this event is the first event, the new summary data is all the current data in the summary table.
[0159] It should be noted that the above data storage device can execute the data storage method provided by the embodiments of the present application, and has the corresponding functional modules and beneficial effects for executing the method. For the technical details not described in detail in the embodiments of the data storage device, reference can be made to the data storage method provided by the embodiments of the present application.
[0160] See Figure 5 , Figure 5Schematic structural diagram of a server provided by an embodiment of the present application. The server 500 includes one or more processors 51 and a memory 52. The memory 52 is connected to one or more processors 51, for example, connected to the processor through a bus.
[0161] The processor 51 is configured to support the server to execute the corresponding functions in the methods in the above method embodiments. The processor may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The above hardware chip may be an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
[0162] The memory 52 is used to store program codes, etc. The memory may include volatile memory (VM), such as random access memory (RAM); the memory may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory may also include a combination of the above types of memories.
[0163] The memory 52 can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the program instructions / modules corresponding to the data storage method in the embodiment of the present application. The processor executes various functional applications and data processing of the data storage method and the data storage device by running the non-volatile software programs, instructions, and modules stored in the memory, that is, implements the functions of each module or unit of the data storage method and the data storage device provided in the above method embodiments.
[0164] The memory 52 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created according to the use of the data storage device, etc. In some embodiments, the memory may optionally include a memory remotely provided with respect to the processor, and these remote memories may be connected to the data storage device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0165] The one or more modules are stored in the memory and, when executed by the one or more processors, perform the data storage method in any of the above method embodiments. For example, the method steps described in the above method embodiments are executed to implement the functions of the modules described in the above device embodiments.
[0166] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a computer, cause the computer to execute the method as described in the foregoing embodiments.
[0167] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, may include the processes of the above method embodiments. Among them, the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.
[0168] The foregoing disclosure is only for the preferred embodiments of the present application, and of course cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.
Claims
1. A data storage method, applied to a server, characterized in that: include: Get user data; Parse the state data and non-state data from the user data, the state data is data updated in real time, the non-state data is data in the user data other than the state data, the preset state database is configured with a real-time state data table, the preset non-state database is configured with a non-state data table, a non-real-time state data table and a summary table, the state database is a database supporting addition, deletion, query and modification functions, and the non-state database is a database supporting only addition and query functions; Writing the status data into the real-time status data table; Writing the non-state data into the non-state data table; In response to a state data persistence event on a regular basis, the latest state data stored in the real-time state data table is written as a first newly added record into the non-real-time state data table of the non-state database; In response to a data immutability event on a regular basis, a hash operation is performed on the newly added data stored in the non-state database to obtain a first operation result, wherein the newly added data is the data newly added since the last regular data immutability event. If the current data immutability event is the first data immutability event, the newly added data is all current data in the non-state database; Generate a second newly added record based on the first operation result; The second newly added record is written into the summary table of the non-state database.
2. The data storage method according to claim 1, characterized in that: The performing a hash operation on the newly added data stored in the non-state database to obtain a first operation result includes: Screening a target data table containing newly added data in the non-state database, wherein the target data table is one of the non-state data table, the non-real-time state data table, and the summary table; All the target data tables are combined into a set of data tables to be hashed; A hash operation is performed on each target data table in the to-be-hash data table set to obtain a first operation result corresponding to each target data table.
3. The data storage method according to claim 2, characterized in that: The performing a hash operation on each target data table in the set of data tables to be hashed to obtain a first operation result corresponding to each target data table includes: Obtaining newly added data in the target data table; Putting the newly added data into a pre-hashed data set; splicing the data in the pre-hashed data set into a pre-hashed string; A hash operation is performed on the pre-hashed character string to obtain a first operation result corresponding to each of the target data tables.
4. The method according to claim 3, characterized in that The pre-hashed data set also includes the last result of the hash operation on the target data table in the summary table; Correspondingly, the splicing the data in the pre-hashed data set into a pre-hashed character string includes: splicing the data in the pre-hashed data set and the previous result into a pre-hashed character string.
5. The method according to claim 3, characterized in that: The latest record of the target data table in the summary table includes an information identifier of the last piece of historical information, the last piece of historical information is the last piece of information included in the target data table when the event of tamper-proofing is performed, the latest record is a record whose time is before the second newly added record, and the acquiring of the newly added data in the target data table includes: Based on the information identifier of the last piece of historical information, information arranged after the last piece of historical information is collected in the target data table to obtain newly added data.
6. The data storage method according to claim 2, characterized in that: The generating a second newly added record based on the first operation result includes: Determine a table name identifier of the target data table, wherein the table name identifier is used to identify the target data table; The first operation result and the table name identifier are combined to form a second newly added record of the target data table.
7. The data storage method according to claim 6, characterized in that: The step of determining a table name identifier of the target data table includes: Get the table name of the target data table; Desensitizing the table name of the target data table to obtain a table name identifier.
8. The data storage method according to claim 6, characterized in that: The newly added data includes a plurality of newly added information arranged in chronological order, and the first operation result and the identifier are combined to form a second newly added record about the target data table, including: Find the last piece of new information from the new data; The first operation result, the table name identifier and the last piece of newly added information are combined into a second newly added record about the target data table.
9. The method according to any one of claims 1 to 8, characterized in that: Also includes: The summary table is published in real time.
10. The method according to any one of claims 1 to 8, characterized in that: Also includes: In response to the summary chaining event on a regular basis, a hash operation is performed on the new summary data stored in the summary table of the non-state database to obtain a second operation result; The second operation result is written into the blockchain, and the new summary data is the summary data newly added since the last regular summary on-chain event. If this event is the first event, the new summary data is all the current data in the summary table.
11. A server, characterized in that: It includes a memory and a processor, the memory is connected to the processor, the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the server implements the data storage method according to any one of claims 1 to 10.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the data storage method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Database auditing method and device based on block chain
CN108446407A
Block data processing method and device
CN111327676A
Transaction data tracing method and system based on block chain
CN111861461A
Method and system for real-time incremental synchronous backup of database files
CN111966529A
Data hash processing method and device, CPU, system and electronic equipment
CN113342530A