Data storage method, device, electronic device and readable medium

Abstract

The present application relates to a method, device, electronic device and readable medium for data storage. The method includes: obtaining a data insertion instruction, wherein the data insertion instruction includes a table identifier of a target data table in a target database and data to be added; obtaining a first primary key value corresponding to the data last inserted in the target database, and obtaining a step value and a modulus value corresponding to the target database, wherein the step value is used to indicate the increment of the primary key value, and the modulus value is used to indicate the remainder of the primary key value and the step value; generating a primary key value to be inserted based on the first primary key value, the step value and the modulus value; inserting the primary key value to be inserted and the data to be added into the target data table based on the table identifier. In this method, the process of obtaining a new primary key value can be completed by the database storing the data itself, and an additional numbering database is no longer needed to generate a new primary key, thereby simplifying the system structure and avoiding the impact of a failure of the numbering database on the entire system, thereby improving the stability of the system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device, and readable medium for data storage. Background Art

[0002] A distributed database system stores data on geographically dispersed physical nodes and accesses them over a network. This type of database is often used by enterprises with geographically dispersed offices or for users with access from all over the place. Generating unique keys for new data tables is a core function of this type of database.

[0003] Currently, the unique key generation method in distributed databases usually uses a number issuing server. When a client needs a new unique key, it needs to apply to the issuing server. The issuing server uses the backend serial number database to generate a new unique key for the client to use.

[0004] However, in existing solutions, the generation of unique keys depends on a serial number database. When the serial number database crashes or responds slowly, the entire application system will become unavailable or have reduced efficiency, thereby affecting the stability of the system. Summary of the Invention

[0005] Based on the above technical problems, the present application provides a data storage method to simplify the system structure, avoid the impact of the failure of the number issuance database on the entire system, and thus improve the stability of the system.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0007] According to one aspect of an embodiment of the present application, a data storage method is provided, including:

[0008] Obtaining a data insertion instruction, wherein the data insertion instruction includes a table identifier of a target data table in a target database and data to be added;

[0009] Obtaining a first primary key value corresponding to the last inserted data in the target database, and obtaining a step value and a modulus value corresponding to the target database, wherein the step value is used to indicate an increment of the primary key value, and the modulus value is used to indicate a remainder between the primary key value and the step value;

[0010] Generate a primary key value to be inserted according to the first primary key value, the step value and the modulus value;

[0011] The primary key value to be inserted and the data to be added are inserted into the target data table according to the table identifier.

[0012] According to one aspect of an embodiment of the present application, there is provided a data storage device, including:

[0013] An instruction acquisition module is used to acquire a data insertion instruction, wherein the data insertion instruction includes a table identifier of a target data table in a target database and data to be added;

[0014] a primary key acquisition module, configured to acquire the first primary key value corresponding to the last inserted data in the target database, and to acquire the step value and modulus value corresponding to the target data table, wherein the step value is used to indicate the increment of the primary key value, and the modulus value is used to indicate the remainder of the primary key value and the step value;

[0015] a primary key generating module, configured to generate a primary key value to be inserted according to the first primary key value, the step value, and the modulus value;

[0016] A data insertion module is used to insert the primary key value to be inserted and the data to be added into the target data table according to the table identifier.

[0017] In some embodiments of the present application, based on the above technical solution, the data storage device further includes:

[0018] A primary key value calculation modulus is used to calculate a second primary key value according to the user-specified primary key, the step value, and the modulus value if the primary key value to be inserted is smaller than the user-specified primary key;

[0019] The primary key value updating module is used to update the primary key value to be inserted according to the second primary key value and the amount of the data to be added.

[0020] In some embodiments of the present application, based on the above technical solution, the primary key acquisition module includes:

[0021] an identifier obtaining unit, configured to obtain, based on the table identifier, an insert identifier of the target database and a pending primary key value corresponding to a most recent insert operation in the target database, wherein the insert identifier is used to indicate whether the most recent update operation on the target database is successful or failed;

[0022] a maximum value obtaining unit, configured to obtain the maximum value of all primary key values ​​in the target data table as the first primary key value if the insert flag indicates that the most recent update operation failed or the to-be-processed primary key value is zero;

[0023] A primary key value determining unit is configured to determine the to-be-processed primary key value as the first primary key value if the insertion flag indicates that the most recent update operation is successful or the to-be-processed primary key value is greater than zero.

[0024] In some embodiments of the present application, based on the above technical solution, the data storage device further includes:

[0025] A response receiving module, configured to receive insert response information of an insert operation on the target data table;

[0026] The identifier updating module is configured to update the insert identifier of the target database if the insert response information indicates that the insert operation on the target data table fails.

[0027] In some embodiments of the present application, based on the above technical solution, the data insertion module includes:

[0028] A request generating unit, configured to generate a data insert request in a database access layer according to the table identifier, the primary key value to be inserted, and the data to be added;

[0029] a request sending unit, configured to send the data insertion request to the target database;

[0030] The database access layer includes a first access node and a second access node, the step value of the first access node is the same as the step value of the second access node, and the module value of the first access node is different from the module value of the second access node.

[0031] In some embodiments of the present application, based on the above technical solution, the primary key generation module includes:

[0032] A locking unit, configured to perform a locking operation on the first primary key value;

[0033] an intermediate value calculation unit, configured to calculate an intermediate value according to the first primary key value and the step value;

[0034] a primary key determining unit, configured to determine the sum of the intermediate value and the modulus value as the primary key value to be inserted;

[0035] An unlocking unit is used to perform an unlocking operation on the first primary key value.

[0036] In some embodiments of the present application, based on the above technical solution, the primary key generation module includes:

[0037] an updated primary key calculation unit, configured to calculate a first updated primary key value according to the first primary key value, the step value, and the modulus value;

[0038] A sequence calculation unit is used to calculate a sequence of primary key values ​​to be inserted corresponding to the multiple data to be added according to the first updated primary key value, the number of data to be added in the data insertion instruction and the step value.

[0039] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the data storage method in the above technical solution by executing the executable instructions.

[0040] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for storing data in the above technical solution is implemented.

[0041] According to one aspect of an embodiment of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for providing data storage in the various optional implementations described above.

[0042] In an embodiment of the present application, based on a data insertion instruction, the most recently inserted primary key value is retrieved from the database. A new primary key value corresponding to the new data is then calculated based on the database's corresponding step size and modulus. The new primary key value and the data to be updated are then inserted into the database to store the new data. The process of obtaining the new primary key value can be performed by the database storing the data itself, eliminating the need for a separate numbering database to generate the new primary key. This simplifies the system structure, prevents failures in the numbering database from impacting the entire system, and improves system stability.

[0043] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0045] In the attached figure:

[0046] Figure 1 The following schematically shows an exemplary physical architecture diagram of the technical solution of the present application in an application scenario;

[0047] Figure 2 A schematic diagram of the system architecture in an embodiment of the present application;

[0048] Figure 3 A schematic diagram showing a flow chart of a data storage method in an embodiment of the present application is shown;

[0049] Figure 4 A flowchart of the primary key value generation process including exception handling in the embodiment of the application;

[0050] Figure 5 This is a schematic diagram of the structure for generating a primary key value in an embodiment of the present application;

[0051] Figure 6 This is a flowchart of the primary key value generation process of the thread lock in the embodiment of the present application;

[0052] Figure 7 The following schematically shows a block diagram of a data storage device according to an embodiment of the present application;

[0053] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0055] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0056] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0057] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0058] It should be understood that the solution of the present application can be applied to a distributed database system. In such scenarios, data is also stored on different physical devices, and usually, different physical devices are also placed in different geographical locations. Each device has a local database and also has corresponding database management functions. Multiple devices communicate through the network and together constitute a complete distributed database. In the database of each device, it is necessary to ensure the uniqueness of the primary key of the data in the data table. The application of the solution of the present application to such scenarios allows devices in different geographical locations to manage their own local databases. When new data needs to be inserted, the local database access device directly generates a new primary key based on the content of the local business database, and the various data bureaus in different geographical locations do not need to communicate with each other to ensure that the generated primary keys do not conflict.

[0059] The application scenario of this application is usually an online system including multiple devices or servers. For the sake of introduction, please refer to Figure 1 , Figure 1 The following schematically shows an exemplary physical architecture diagram of the technical solution of the present application in an application scenario.

[0060] Depend on Figure 1 As can be seen, this scenario includes access devices, databases, and various terminal devices such as terminals and laptops. Various terminal devices can access business data through access devices. When a terminal device needs to insert data into a certain data table, it sends an insert request to the access device. The access device generates a primary key for the insert content in the insert request according to the solution of this application, and sends the primary key and insert content to the database. The database stores new data in the data table based on the received primary key and insert content. The result of the insert operation is then sent back to the terminal device, thus completing the overall data storage process.

[0061] Figure 1 The access device shown in the figure may include multiple access devices, and the corresponding database may also include multiple databases. Each access device corresponds to one or more databases. The same database is usually managed by one access device.

[0062] The communication method between the client and the access device and the communication method between the access device and the database can adopt any wired or wireless communication method, and this application does not impose any restrictions on this.

[0063] It is understandable that Figure 1 The scenarios shown in the figure are only examples of scenarios in which the solution of this application is applied. The actual application scenarios may adopt other suitable network structures, such as adding proxy servers and multi-level networks, etc. This application does not impose any restrictions on this.

[0064] based on Figure 1 The physical architecture shown in , the solution of this application can be applied to the cluster logical system architecture. Specifically, for the convenience of introduction, please refer to Figure 2 , Figure 2 Schematic diagram of the system architecture in the embodiment of this application. Figure 2 As shown in the figure, the system structure mainly consists of the application layer, access layer, access layer and data layer. The application layer is used to communicate with the client and process business logic. The access layer is used to allow requests from the application layer to access the data layer. The access layer is used to support the routing of access statements of the data definition language type to various databases. The data layer corresponds to each database and is mainly responsible for the actual storage of data. Figure 2 As shown, the access layer includes multiple access devices, while the data layer also includes multiple local databases. Each access device is responsible for managing one or more local databases. The access devices in the access layer rewrite the database structured query statements in received data access requests according to the configured data routing rules, and then distribute them to the corresponding local databases for execution. After executing the corresponding operations of the query statements, the local databases feed the results upward to the application layer, thus completing operations such as adding, deleting, updating, and querying data.

[0065] The technical solution provided in this application is described in detail below in conjunction with specific implementation methods.

[0066] See also Figure 3 , Figure 3 The following is a flow chart of a method for storing data in accordance with an embodiment of the present application. The method of this embodiment can be applied to the server described above and executed by a background program on the server. The method of this embodiment can include the following steps S201 to S204:

[0067] Step S201: Obtain a data insert instruction, wherein the data insert instruction includes a table identifier of a target data table in a target database and data to be added.

[0068] The data insertion instruction originates from a request from a client at the application layer. In a specific implementation, the client determines the data table to be queried based on business requirements and, therefore, the table identifier. The table identifier is typically the table name. Specifically, the data insertion instruction includes database instructions for adding new data. The specific format of the database instructions depends on the specific implementation of the database at the data layer.

[0069] Step S202, obtain the first primary key value corresponding to the data last inserted in the target database, and obtain the step value and modulus value corresponding to the target data table, wherein the step value is used to indicate the incremental amount of the primary key value, and the modulus value is used to indicate the remainder of the primary key value and the step value.

[0070] Specifically, the first primary key value corresponding to the last inserted data in the target database is retrieved from the database. The retrieved first primary key value is independent of the target data table's update process. That is, if data is first inserted into table A and then into table B, the first primary key value retrieved from the database is the primary key value inserted into table B. For a single local database, the step value and modulus are fixed. The step value and modulus can be configured in devices in the access layer, used as database configuration parameters, or stored directly in the database's data tables.

[0071] The distributed storage system of the present application is configured with relevant parameters to facilitate the control of the generation process of the primary key value. Specific parameters include a mode switching parameter, an incremental step parameter, and a modulus parameter. The switching parameter is used to switch between the primary key generation method of this scheme and the primary key generation method of the database default. The incremental step parameter and the modulus parameter are used to store the step value and the modulus value, respectively. The above parameters can be configured at the access layer and the data layer, and the access device has the same parameter configuration as the local database it manages. For the mode switching parameters, the parameter configurations in different local databases or different access devices may be different. For the incremental step parameter and the modulus parameter, the values ​​of the incremental step parameters of all access devices in the same cluster should be the same, while the values ​​of the modulus parameters are different from each other, so as to ensure that the generated primary key values ​​are not repeated.

[0072] Step S203: Generate a primary key value to be inserted according to the first primary key value, the step value and the modulus value.

[0073] In this embodiment, the calculation method for the primary key value to be inserted can be a method of summing the first primary key value and the step value. When this calculation method is used, the first primary key value generated by each database can directly use the modulus value.

[0074] When the target data table is created, specific setting rules or trigger rules may be added to the primary key column corresponding to the primary key, depending on the specific implementation method adopted by the database. For example, the primary key column can be declared as auto-incremental. If the insert statement in the data storage request does not add the primary key of the primary key column or sets the primary key to 0 or a null value, the database can automatically generate a primary key value for this insert operation according to the preset step size and modulus. In this way, the increment process of the primary key value can be performed in the database, thereby ensuring that higher-level applications do not need to be modified and can be directly integrated with the primary key value generation process, so that the application layer logic does not need to be modified due to the switch of the primary key generation method, avoiding additional work and reducing production efficiency.

[0075] Step S204: inserting the to-be-inserted primary key value and the to-be-added data into the target data table according to the table identifier.

[0076] Specifically, the server can directly insert the primary key value to be inserted and the data to be added into the target data table through a database instruction. Specifically, the server can directly generate a corresponding insert instruction based on the format of the database insert instruction, using the table identifier, the primary key value to be inserted and the data to be added, and insert the primary key value to be inserted and the data to be added into the target data table according to the insert instruction. It can be understood that the insert operation on the data table can be understood as adding new data to the data table, and the location of the addition in the data table is not limited. It can be added to the last row of the data table or inserted between two existing rows.

[0077] In an embodiment of the present application, based on a data insertion instruction, the most recently inserted primary key value is retrieved from the database. A new primary key value corresponding to the new data is then calculated based on the step size and modulus corresponding to the database. The new data is then stored in a data table containing the new primary key value and the data to be updated. The process of obtaining the new primary key value can be accomplished within the database storing the data itself, eliminating the need for a separate numbering database to generate the new primary key. This simplifies the system structure, prevents failures in the numbering database from impacting the entire system, and improves system stability.

[0078] In one embodiment of the present application, a user may specify a primary key value for the data to be added. The data insertion instruction includes the user-specified primary key. After the above step S203 generates the primary key value to be inserted based on the first primary key value, the step value, and the modulus value, the method further includes:

[0079] If the primary key value to be inserted is smaller than the user-specified primary key, calculating a second primary key value according to the user-specified primary key, the step value, and the modulus value;

[0080] The to-be-inserted primary key value is updated according to the second primary key value and the quantity of the to-be-added data.

[0081] Among them, the user-specified primary key needs to comply with the primary key value generation procedure of the corresponding server. For example, if the step value of the server is 4 and the modulus value is 1, the primary key value sequence it generates should be represented by 4x+1, where x is an integer, that is, a number in the sequence of 1, 5, 9, ..., 4x+1. If the user specifies a primary key value outside the sequence, it will not be added to the data table. In one embodiment, when the server receives a data insertion instruction, it will check the user-specified primary key therein. If it does not meet the conditions, it will directly report an error to the client. The rules for user-specified primary keys can also be controlled on the client. For example, the client obtains the step value and modulus value of the server in advance. When the user enters the primary key value on the client, it directly prompts the input rule and checks the input value, or directly provides options to the user for the user to choose.

[0082] For user-specified primary keys that meet the conditions, after the primary key value to be inserted is calculated, the user-specified primary key and the primary key value to be inserted will be compared. If the primary key value to be inserted is smaller than the user-specified primary key, it means that the user-specified primary key does not exist in the target data table and can be inserted into the database. In this embodiment, the server calculates the second primary key value based on the user-specified primary key, the step value and the modulus value. The calculation process of the second primary key value is the same as the calculation process of the primary key value to be inserted. Subsequently, the value of the primary key value to be inserted can be determined based on the number of data to be added. Specifically, in the case where there is only one data to be added, the second primary key value becomes the next value of the primary key value to be inserted in the primary key value sequence. At this time, the primary key to be inserted can be directly replaced with the user-specified primary key. Correspondingly, when there are multiple data to be added, the primary key value to be inserted is a sequence including multiple primary key values, and the second primary key value is the maximum value of the sequence. According to the second primary key value, a value range greater than the primary key value to be inserted can be determined. At this time, the values ​​in the sequence of primary key values ​​to be inserted are replaced in sequence with the values ​​in the sequence of primary key values ​​with the second primary key value as the maximum value. For example, if the primary key values ​​to be inserted are 13, 17 and 21, and the user specifies the primary key as 25, then since 25 is greater than 21, the second primary key values ​​that can be calculated are 25, 29, and 33. Therefore, the primary key to be inserted is replaced with 25, 29, and 33.

[0083] In this embodiment, the primary key value to be inserted is updated according to the primary key value provided by the user, allowing the user to define the primary key value by himself, thereby improving the flexibility of the solution.

[0084] In one embodiment, in order to cope with abnormal situations during the operation of the server, the above step S202 of obtaining the first primary key value corresponding to the last inserted data in the target database may include the following steps:

[0085] According to the table identifier, obtaining an insert identifier of the target database and a pending primary key value corresponding to a most recent insert operation in the target database, wherein the insert identifier is used to indicate whether the most recent update operation on the target data table is successful or failed;

[0086] If the insert flag indicates that the most recent update operation failed or the to-be-processed primary key value is zero, obtaining the maximum value of all primary key values ​​in the target database as the first primary key value;

[0087] If the insertion flag indicates that the most recent update operation is successful or the to-be-processed primary key value is greater than zero, the to-be-processed primary key value is determined as the first primary key value.

[0088] Specifically, when obtaining the to-be-processed primary key value corresponding to the most recent insert operation in the target database, the insert identifier of the target database is also obtained. The insert identifier is used to indicate whether the most recent update operation on the target data table is successful or failed.

[0089] During server operation, abnormal scenarios often include duplicate primary key values ​​leading to insertion failures, server restarts or downtime, and table deletions resulting in the most recent primary key value being 0. Therefore, it is necessary to determine whether the currently obtained pending primary key value can be used for primary key value calculation based on the insertion identifier and the pending primary key value.

[0090] For ease of introduction, see Figure 4 , Figure 4 This is a flowchart of the primary key value generation process including exception handling in the embodiment of the application itself. In this embodiment, in addition to obtaining the first primary key value, the access device also obtains the insert identifier of the target data table, and the insert identifier is used to indicate whether the most recent update operation on the target data table is successful or failed. The insert identifier can be stored in the database. In addition, if the obtained primary key value is zero, it can be determined that a failure has occurred on the database side or a restart has been performed. The access device determines whether the current first primary key value can be used to calculate the next primary key value based on the obtained first primary key value and the insert identifier. If the obtained first primary key value is 0, or the insert identifier indicates that the last insertion failed, the maximum primary key value will be obtained from the target data table as the first primary key value to continue the calculation. In addition, after obtaining the maximum primary key value, the insert identifier of the target data table will also be updated to a successful insertion.

[0091] In one embodiment, the target database includes a primary key table. The primary key values ​​for all tables in the target database that use auto-increment rules are obtained from the primary key table. All primary key values ​​for the target data table and other tables in the database are stored in the primary key table. If the first primary key value obtained is 0, or the insert flag indicates that the previous insert failed, the maximum value in the primary key table is obtained and used as the first primary key value for subsequent calculations.

[0092] In an embodiment of the present application, the obtained primary key value to be processed and the insertion identifier of the database are used to determine whether an error occurred in the previous insertion process, thereby avoiding the error in the insertion process that leads to an error in the calculated primary key value and the inability to perform subsequent insertion processes, thereby improving the stability of the system.

[0093] In one embodiment, after the above step S204, in which the primary key value to be inserted and the data to be added are inserted into the target data table according to the table identifier, the method may further include the following steps:

[0094] receiving insert response information of an insert operation on the target data table;

[0095] If the insert response information indicates that the insert operation on the target data table fails, the insert flag of the target data table is updated.

[0096] In this embodiment, after the server performs the insertion process, it will receive the insertion response information of the insertion operation on the target data table. The insertion response information is the insertion result feedback generated by the database after the insertion process is completed. If the insertion process is completed correctly, the insertion response information will include the insertion success information. If an error occurs during the insertion process, the insertion response information may include error details, such as error code, etc. If the insertion response information indicates that the insertion operation on the target data table fails, the insertion flag of the target data table is updated. For details, please refer to Figure 4 After sending a data storage request to the database, the database will perform an insert operation on the target data table according to the data storage request. If an error occurs in the insert operation, for example, due to various situations such as operation timeout, duplicate primary key, or target data table lock, the database will feedback an error prompt to the access device as the update result. After receiving the error prompt, the access device will update the insert flag of the target data table to a failed state, so that the next time data is inserted, it will trigger the process of obtaining the maximum primary key value as the first primary key value to ensure the continuity and uniqueness of the primary key. The default state of the insert flag is the success state. Therefore, when the insertion is successful, the insert flag can be left unchanged.

[0097] In this embodiment, a method for updating the insertion identifier of the database is provided, in which the insertion identifier of the database is updated through the successful result of the insertion operation fed back by the database. This is conducive to timely fault response during the next insertion when an error occurs during the insertion process, thereby improving the operability of the solution.

[0098] In one embodiment of the present application, the above step S204, inserting the to-be-inserted primary key value and the to-be-added data into the target data table according to the table identifier, may include:

[0099] Generate a data insertion request in the database access layer according to the table identifier, the primary key value to be inserted and the data to be added;

[0100] Sending the data insertion request to the database;

[0101] The database access layer includes a first access node and a second access node, the step value of the first access node is the same as the step value of the second access node, and the module value of the first access node is different from the module value of the second access node.

[0102] Specifically, in this embodiment, the server can be a distributed system including multiple devices. The distributed system includes a database access layer and a database layer. The database access layer can include multiple access nodes, and the database layer side includes multiple databases. The distributed system generates a data insertion request in the database access layer based on the table identifier, the primary key value to be inserted, and the data to be added. The data insertion request includes a data insertion instruction for database execution. The data insertion instruction can be written in a database language. The database access layer sends the data insertion request to the target database. The step value is shared between all access nodes in the database access layer, but the modulus value must be different. It can be understood that due to the characteristics of the modulus operation, the value of the step value usually depends on the number of access nodes used, that is, when the step value is 10, a maximum of 10 access devices can be supported, and their modulus values ​​are 0 to 9 respectively.

[0103] For ease of introduction, see Figure 5 , Figure 5 This is a schematic diagram of the structure of generating primary key values ​​in the embodiment of this application. Figure 5As shown, the entire distributed database system includes 4 access devices. Each access device can independently receive the data storage instruction of the terminal device, and the step size of each access device is the same, while the modulus value is different. For example, when the step size is 4, the four access devices each generate a primary key value sequence with a modulus value of 0, 1, 2, and 3, and the sequence increases with time. Taking the access device with a modulus value of 1 as an example, when it receives the data storage instruction from the client, it obtains the most recently inserted primary key value from the database, assuming it is 9. Then, based on 9, it calculates the next primary key value to be 13, and then rewrites the database statement in the data storage instruction, adds the primary key value to the statement, and then sends the statement to the local database for execution. It can be seen from the above process that the step size is kept the same and the modulus is different between each access device, so that the primary key values ​​generated by any two access devices will not be repeated.

[0104] In the embodiment of the present application, multiple access devices can ensure the uniqueness of the generated primary key value in the entire distributed system without having to communicate with each other, thereby reducing system communication overhead and improving operating efficiency.

[0105] In one embodiment of the present application, the terminal device may communicate with the server in a multi-threaded manner. The above step S203, generating the primary key value to be inserted according to the first primary key value, the step value and the modulus value, may include:

[0106] Performing a locking operation on the first primary key value;

[0107] Calculate an intermediate value according to the first primary key value and the step value;

[0108] Determine the sum of the intermediate value and the modulus value as the primary key value to be inserted;

[0109] An unlock operation is performed on the first primary key value.

[0110] In this embodiment, the application layer client operates in multiple threads. This may result in a single access device being requested to insert data by multiple threads simultaneously, or a single thread being requested to store data multiple times simultaneously. To avoid conflicts during the primary key generation process, a thread mutex lock is added to the relevant data during the primary key calculation process to synchronize the threads. After the lock is applied, the calculation and update of the primary key value to be inserted can be resumed.

[0111] Specifically, the primary key value to be inserted can be generated using the following formula:

[0112]

[0113] Where K is the first primary key value, S is the step value, and M is the modulus value. When calculating, first calculate the quotient of (K+S-1) / S as the intermediate value. Then, round the intermediate value down to an integer, and then combine the step value and modulus value to calculate the intermediate value to be checked. For example, assuming the step value S is 7, the modulus value M is 2, and the first primary key value obtained is 9, then the intermediate value = (10+4-1) / 4 = 3.25, rounded down to 3, and then substituted into the formula to calculate the primary key value to be inserted is 14.

[0114] It is understandable that in the overall process of generating primary key values ​​using thread locks, thread lock control can be performed in all processes involving updating the primary key values ​​to be inserted. Figure 6 , Figure 6 This is a flow chart of the primary key value generation process using a thread lock in the embodiment of this application. Figure 6 As shown, the detailed steps for generating the primary key value are the same as those in the above embodiment and will not be repeated here. At the beginning of the process, a mutex lock is first added to the primary key variable used to obtain the first primary key value. Subsequently, the most recently inserted primary key value in the data table is obtained from the corresponding local database and assigned to the primary key variable. After obtaining the primary key value, the primary key variable for the first primary key value can be unlocked. Subsequently, before the access device prepares to calculate the primary key value, the primary key variable is locked again. Then, a new primary key value is calculated according to the method described in the above embodiment, and the primary key variable is updated to the calculated new primary key value. After completion, the primary key variable is unlocked again. In the subsequent process, if the user-specified primary key value is valid and the primary key variable needs to be updated again, the primary key variable is locked and updated according to the user-specified primary key value according to the rules. The result of the primary key variable is then calculated. It will be understood that in this embodiment, a mutex lock is added for synchronization control only when the primary key variable needs to be updated. Before each lock and after each unlock, various data operations such as calculation of data sharding, information verification, and status filling are interspersed.

[0115] In an embodiment of the present application, access control between multiple threads is performed by adding thread locks, thereby avoiding data conflicts caused by multi-thread access and improving system stability.

[0116] In one embodiment of the present application, a data insertion instruction includes multiple pieces of data to be added. The above step S203, generating the primary key value to be inserted according to the first primary key value, the step value and the modulus value, may include the following steps:

[0117] Calculate a first updated primary key value according to the first primary key value, the step value and the modulus value;

[0118] A sequence of primary key values ​​to be inserted corresponding to the plurality of data to be added is calculated according to the first updated primary key value, the number of the data to be added in the data insertion instruction, and the step value.

[0119] This embodiment provides a method for batch generating multiple primary key values ​​to be inserted. Specifically, first, based on the first primary key value, the step value and the modulus value, the first updated primary key value is calculated. The specific calculation process is the same as the process described in the above embodiment. The calculated first updated primary key value can be understood as the primary key value of the first data among multiple data to be added. In the subsequent data table insertion process, multiple data to be added in a data insertion instruction will be inserted into the data table together in one insertion operation, and their primary key values ​​can be linear. In this way, after calculating the first updated primary key value corresponding to the first data to be added, the sequence of primary key values ​​to be inserted can be directly calculated based on the number of data to be added and the step value. For example, the calculated first updated primary key value is 10, and the step value is 3. If the number of data to be added is 4, the sequence of primary key values ​​to be inserted is 10, 10+3=13, 13+3=16, and 16+3=19.

[0120] In an embodiment of the present application, multiple primary key values ​​to be inserted for multiple data to be added are calculated in batches, thereby allowing multiple data to be inserted into a data table in one insert operation, thereby reducing the system's overhead in processing data addition and improving data storage efficiency.

[0121] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0122] The following introduces the device implementation of the present application, which can be used to execute the data storage method in the above embodiments of the present application. Figure 7 The following schematically shows a block diagram of the data storage device in an embodiment of the present application. Figure 7 As shown, the data storage device 300 may mainly include:

[0123] An instruction acquisition module 310 is configured to acquire a data insertion instruction, wherein the data insertion instruction includes a table identifier of a target data table in a target database and data to be added;

[0124] A primary key acquisition module 320 is configured to acquire the first primary key value corresponding to the last inserted data in the target database, and to acquire the step value and modulus value corresponding to the target data table, wherein the step value indicates the increment of the primary key value, and the modulus indicates the remainder of the primary key value and the step value;

[0125] A primary key generating module 330, configured to generate a primary key value to be inserted according to the first primary key value, the step value, and the modulus value;

[0126] The data inserting module 340 is configured to insert the primary key value to be inserted and the data to be added into the target data table according to the table identifier.

[0127] In some embodiments of the present application, based on the above technical solution, the data storage device 300 further includes:

[0128] A primary key value calculation modulus is used to calculate a second primary key value according to the user-specified primary key, the step value, and the modulus value if the primary key value to be inserted is smaller than the user-specified primary key;

[0129] The primary key value updating module is used to update the primary key value to be inserted according to the second primary key value and the amount of the data to be added.

[0130] In some embodiments of the present application, based on the above technical solution, the primary key acquisition module 320 includes:

[0131] an identifier obtaining unit, configured to obtain, based on the table identifier, an insert identifier of the target database and a pending primary key value corresponding to a most recent insert operation in the target database, wherein the insert identifier is used to indicate whether the most recent update operation on the target database is successful or failed;

[0132] a maximum value obtaining unit, configured to obtain the maximum value of all primary key values ​​in the target data table as the first primary key value if the insert flag indicates that the most recent update operation failed or the to-be-processed primary key value is zero;

[0133] A primary key value determining unit is configured to determine the to-be-processed primary key value as the first primary key value if the insertion flag indicates that the most recent update operation is successful or the to-be-processed primary key value is greater than zero.

[0134] In some embodiments of the present application, based on the above technical solution, the data storage device 300 further includes:

[0135] A response receiving module, configured to receive insert response information of an insert operation on the target data table;

[0136] The identifier updating module is configured to update the insert identifier of the target database if the insert response information indicates that the insert operation on the target data table fails.

[0137] In some embodiments of the present application, based on the above technical solution, the data insertion module 340 includes:

[0138] A request generating unit, configured to generate a data insert request in a database access layer according to the table identifier, the primary key value to be inserted, and the data to be added;

[0139] a request sending unit, configured to send the data insertion request to the target database;

[0140] The database access layer includes a first access node and a second access node, the step value of the first access node is the same as the step value of the second access node, and the module value of the first access node is different from the module value of the second access node.

[0141] In some embodiments of the present application, based on the above technical solution, the primary key generation module 330 includes:

[0142] A locking unit, configured to perform a locking operation on the first primary key value;

[0143] an intermediate value calculation unit, configured to calculate an intermediate value according to the first primary key value and the step value;

[0144] a primary key determining unit, configured to determine the sum of the intermediate value and the modulus value as the primary key value to be inserted;

[0145] An unlocking unit is used to perform an unlocking operation on the first primary key value.

[0146] In some embodiments of the present application, based on the above technical solution, the primary key generation module 330 includes:

[0147] an updated primary key calculation unit, configured to calculate a first updated primary key value according to the first primary key value, the step value, and the modulus value;

[0148] A sequence calculation unit is used to calculate a sequence of primary key values ​​to be inserted corresponding to the multiple data to be added according to the first updated primary key value, the number of data to be added in the data insertion instruction and the step value.

[0149] It should be noted that the apparatus provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0150] Figure 8A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown.

[0151] It should be noted that Figure 8 The computer system 400 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0152] like Figure 8 As shown, computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 402 or programs loaded from storage unit 408 into random access memory (RAM) 403. Various programs and data required for system operation are also stored in RAM 403. CPU 401, ROM 402, and RAM 403 are connected to each other via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0153] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read therefrom can be installed into the storage section 408 as needed.

[0154] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 409, and / or installed from a removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the various functions defined in the system of the present application are executed.

[0155] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0158] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0159] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0160] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data storage method, characterized in that: The method is applied to a first access node in a distributed system, the distributed system including a database access layer, the database access layer including the first access node and a second access node, and includes: Obtaining a data insertion instruction, wherein the data insertion instruction includes a table identifier of a target data table in a target database and data to be added, and the data insertion instruction includes a database instruction for adding new data; According to the table identifier, obtaining an insert identifier of the target database and a pending primary key value corresponding to a most recent insert operation in the target database, wherein the insert identifier is used to indicate success or failure of the most recent update operation on the target database, and obtaining a step value and a modulus value corresponding to the target database, wherein the step value is used to indicate an increment of the primary key value, and the modulus value is used to indicate a remainder between the primary key value and the step value; If the insertion flag indicates that the most recent update operation is successful or the to-be-processed primary key value is greater than zero, determining the to-be-processed primary key value as the first primary key value; Generate a primary key value to be inserted according to the first primary key value, the step value and the modulus value; rewriting, in the database access layer, a database statement in the data insertion instruction, adding the primary key value to be inserted into the database statement, wherein the step value of the first access node is the same as the step value of the second access node, and the modulus value of the first access node is different from the modulus value of the second access node; The rewritten data insertion instruction is sent to the target database, so that the target database inserts the to-be-inserted primary key value and the to-be-added data into the target data table according to the table identifier.

2. The method according to claim 1, characterized in that The data insertion instruction also includes a user-specified primary key; after generating the primary key value to be inserted according to the first primary key value, the step value, and the modulus value, the method further includes: If the primary key value to be inserted is smaller than the user-specified primary key, calculating a second primary key value according to the user-specified primary key, the step value, and the modulus value; The to-be-inserted primary key value is updated according to the second primary key value and the quantity of the to-be-added data.

3. The method according to claim 1, characterized in that The method further comprises: If the insert flag indicates that the most recent update operation failed or the to-be-processed primary key value is zero, the maximum value of all primary key values ​​in the target data table is obtained as the first primary key value.

4. The method according to claim 3, characterized in that After inserting the to-be-inserted primary key value and the to-be-added data into the target data table according to the table identifier, the method further includes: receiving insert response information of an insert operation on the target data table; If the insert response information indicates that the insert operation on the target data table fails, the insert flag of the target database is updated.

5. The method according to claim 1, wherein The generating the primary key value to be inserted according to the first primary key value, the step value, and the modulus value includes: Performing a locking operation on the first primary key value; Calculate an intermediate value according to the first primary key value and the step value; Determine the sum of the intermediate value and the modulus value as the primary key value to be inserted; An unlock operation is performed on the first primary key value.

6. The method according to claim 1, characterized in that The data insertion instruction includes a plurality of data to be added; and generating the primary key value to be inserted according to the first primary key value, the step value, and the modulus value includes: Calculate a first updated primary key value according to the first primary key value, the step value and the modulus value; A sequence of primary key values ​​to be inserted corresponding to the plurality of data to be added is calculated according to the first updated primary key value, the number of the data to be added in the data insertion instruction, and the step value.

7. A data storage device, characterized in that A first access node applied to a distributed system, the distributed system including a database access layer, the database access layer including the first access node and a second access node, including: An instruction acquisition module is used to acquire a data insertion instruction, wherein the data insertion instruction includes a table identifier of a target data table in a target database and data to be added, and the data insertion instruction includes a database instruction for adding new data; a primary key acquisition module, configured to acquire, based on the table identifier, an insert identifier of the target database and a pending primary key value corresponding to a most recent insert operation in the target database, wherein the insert identifier is used to indicate whether the most recent update operation on the target database is successful or failed, and to acquire a step value and a modulus value corresponding to the target data table, wherein the step value is used to indicate an increment of the primary key value, and the modulus value is used to indicate a remainder between the primary key value and the step value; if the insert identifier indicates that the most recent update operation is successful or the pending primary key value is greater than zero, then determining the pending primary key value as the first primary key value; a primary key generating module, configured to generate a primary key value to be inserted according to the first primary key value, the step value, and the modulus value; A data insertion module is used to rewrite the database statement in the data insertion instruction in the database access layer, add the primary key value to be inserted into the database statement, the step value of the first access node is the same as the step value of the second access node, and the modulus value of the first access node is different from the modulus value of the second access node; and send the rewritten data insertion instruction to the target database, so that the target database inserts the primary key value to be inserted and the data to be added into the target data table according to the table identifier.

8. An electronic device, characterized in that: include: processor; a memory for storing executable instructions of the processor; The processor is configured to execute the data storage method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data storage method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the data storage method according to any one of claims 1 to 6.

Images