Data storage method and device, equipment, medium and program product
By dynamically selecting the full update operation based on the attribute fields in the access interface and the target message in the Elasticsearch cluster, the IO resource consumption problem caused by custom data primary keys is solved, and the system performance and resource utilization are improved.
Patent Information
- Application Number
- CN202411328363.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-02-25
AI Technical Summary
When using an Elasticsearch cluster, frequent query and delete operations caused by custom primary keys increase IO resource consumption, affecting system performance and resource usage.
By determining the access interface type and the target attribute field in the target message, the system dynamically selects whether to perform a full update operation, thus avoiding unnecessary routine update operations.
It improved system performance, reduced resource consumption, and optimized resource utilization.
Smart Images
Figure CN119513190A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a data storage method, device, equipment, medium and program product. Background Art
[0002] Elasticsearch is an open source search and analysis engine based on the Lucene library, which is well-known for its powerful search function, real-time performance and wide application in the field of big data. It is not just a simple search engine, but also a powerful tool that can realize full-text retrieval, structured data retrieval and analysis. Therefore, the current financial, Internet, communication and other industries usually use elasticsearch clusters as the basic service components for real-time index queries (response requirements in seconds or less) to handle a large number of highly concurrent real-time query requests.
[0003] When using elasticsearch as a data storage or search solution, in order to optimize performance and improve resource utilization, batch writing or updating is usually used to process data regularly. However, when the application uses a custom data primary key, each time the data corresponding to the custom data primary key is written or updated, the elasticsearch cluster will automatically trigger query, delete and other operations for the write or update operation to ensure the uniqueness of the data. When the amount of data is large, it will cause a large IO resource occupation, affecting the performance of the system and causing resource consumption.
[0004] Therefore, when an application uses a custom data primary key, how to ensure data uniqueness while improving system performance and reducing resource consumption is a technical problem that needs to be solved urgently. Summary of the invention
[0005] In order to solve the above technical problems, the present disclosure provides a data storage method, device, equipment, medium and program product.
[0006] A first aspect of an embodiment of the present disclosure provides a data storage method, including:
[0007] In response to a data storage request from a user, when it is determined that the data storage request uses a custom data primary key, determining an access interface corresponding to the data storage request;
[0008] When it is determined that the access interface is the first interface, executing a full update operation corresponding to the data storage request;
[0009] When it is determined that the access interface is the second interface, a target message corresponding to the data storage request is obtained, and it is determined whether the target message contains a target attribute field, where the target attribute field is a field used to determine whether to perform a full update operation on the target message;
[0010] When it is determined that the target message contains the target attribute field, a full update operation corresponding to the data storage request is performed.
[0011] A second aspect of an embodiment of the present disclosure provides a data storage device, including:
[0012] An interface determination module, for responding to a data storage request from a user, and when determining that the data storage request uses a custom data primary key, determining an access interface corresponding to the data storage request;
[0013] A first update module, configured to execute a full update operation corresponding to the data storage request when determining that the access interface is the first interface;
[0014] An attribute field determination module, used to obtain a target message corresponding to the data storage request when determining that the access interface is the second interface, and determine whether the target message contains a target attribute field, the target attribute field being a field used to determine whether to perform a full update operation on the target message;
[0015] The second update module is used to execute a full update operation corresponding to the data storage request when it is determined that the target message contains a target attribute field.
[0016] A third aspect of the embodiments of the present disclosure provides an electronic device, including:
[0017] processor;
[0018] A memory for storing executable instructions;
[0019] The processor is used to read executable instructions from the memory and execute the executable instructions to implement the data storage method provided in the first aspect above.
[0020] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the data storage method provided by the first aspect.
[0021] A fifth aspect of an embodiment of the present disclosure provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the data storage method of the first aspect is implemented.
[0022] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0023] The data storage method, apparatus, device, medium and program product provided by the embodiments of the present disclosure can respond to the user's data storage request, and when it is determined that the data storage request uses a custom data primary key, determine the access interface corresponding to the data storage request, and when it is determined that the access interface is the first interface, perform the full update operation corresponding to the data storage request, and when it is determined that the access interface is the second interface, obtain the target message corresponding to the data storage request, and determine whether the target message contains a target attribute field, the target attribute field is a field used to determine whether to perform a full update operation on the target message, and when it is determined that the target message contains the target attribute field, perform the full update operation corresponding to the data storage request. Therefore, it is possible to determine whether to perform a full update operation by determining the type of the access interface and / or the situation that the target message contains the target attribute field. When it is determined that the access interface is the first interface, or when the access interface is the second interface but the target message contains the target attribute field, perform the full update operation, thereby avoiding the resource consumption caused by performing a regular update operation each time a data storage request is executed, thereby improving system performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0026] Figure 1 is a flow chart of a data storage method provided by an embodiment of the present disclosure;
[0027] Figure 2 is a flow chart of a conventional update operation execution method provided by an embodiment of the present disclosure;
[0028] Figure 3 is a flow chart of another data storage method provided by an embodiment of the present disclosure;
[0029] Figure 4 is a structural schematic diagram of a data storage device provided by an embodiment of the present disclosure;
[0030] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0031] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0032] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0033] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0034] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0035] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0036] Normally, when an application uses a custom data primary key, each time a write or update operation is performed on the data corresponding to the custom data primary key, the elasticsearch cluster will automatically trigger regular update operations such as query and deletion for the write or update operation to ensure the uniqueness of the data. When the amount of data is large, it will cause a large IO resource occupation, affecting the performance of the system and causing resource consumption. To address this problem, the embodiments of the present disclosure provide a data storage method, which is introduced below in conjunction with specific embodiments.
[0037] Figure 1It is a flowchart of a data storage method provided in an embodiment of the present disclosure. The method can be executed by a data storage device. The data storage device can be implemented in software and / or hardware. The data storage device can be configured in an electronic device, such as a server or a terminal or a server cluster, wherein the terminal specifically includes a mobile phone, a computer or a tablet computer, etc.
[0038] like Figure 1 As shown, the data storage method provided by the embodiment of the present disclosure can be applied to a data search cluster. For example, it can be an elasticsearch cluster, hereinafter referred to as an ES cluster. The data storage method can specifically include the following steps.
[0039] S110 . In response to a data storage request from a user, when it is determined that the data storage request uses a user-defined data primary key, determine an access interface corresponding to the data storage request.
[0040] In the embodiment of the present disclosure, the data storage request may be a request for writing or updating data.
[0041] In the embodiments of the present disclosure, the data primary key is used to uniquely identify the data in the table. Taking the ES cluster as an example, each doc data written into the ES cluster will have an ID, which can be understood as the data primary key. The data primary key can be automatically generated by the ES cluster or customized by the user according to usage requirements.
[0042] The access interface can be understood as the interface for clients to access the data search cluster.
[0043] Specifically, the electronic device determines, in response to a data storage request from a user, whether the data storage request uses a custom data primary key, and if the custom data primary key is used, determines an access interface corresponding to the data storage request.
[0044] S120: When it is determined that the access interface is the first interface, execute a full update operation corresponding to the data storage request.
[0045] In an embodiment of the present disclosure, the first interface is an update interface that is pre-set and newly added in a data search cluster such as an ES cluster, and is used to receive a data storage request for directly performing a full update operation on the data.
[0046] The full update operation can be understood as writing all the data corresponding to the data storage request into the data search cluster or the preset database without performing operations such as querying or deleting the data.
[0047] Specifically, when determining that the access interface is the first interface, the electronic device parses the data storage request, obtains a target message corresponding to the data storage request, and performs a full update operation corresponding to the data storage request based on the target message.
[0048] S130. When it is determined that the access interface is the second interface, obtain a target message corresponding to the data storage request, and determine whether the target message contains a target attribute field, where the target attribute field is a field used to determine whether to perform a full update operation on the target message.
[0049] In the embodiment of the present disclosure, the second access interface is a native update interface in a data search cluster such as an ES cluster.
[0050] The target attribute field can be understood as a field added to the target message before the user transmits the target message through the client, and used to perform a full update operation on the target message. Exemplarily, the target attribute field can be updateall or the like.
[0051] Specifically, when the electronic device determines that the access interface is the second interface, it parses the data storage request, obtains the target message corresponding to the data storage request, and identifies and analyzes the target message to determine whether the target message contains a target attribute field.
[0052] S140. When it is determined that the target message contains a target attribute field, a full update operation corresponding to the data storage request is executed.
[0053] In the disclosed embodiment, when the electronic device determines that the target message contains a target attribute field and the value of this field is true, it determines to perform a full update operation on the data storage request, and then performs the full update operation corresponding to the data storage request.
[0054] Furthermore, after the complete update operation is executed, the execution result is returned to the client.
[0055] In an embodiment of the present disclosure, it is possible to respond to a user's data storage request, and when it is determined that the data storage request uses a custom data primary key, determine the access interface corresponding to the data storage request, and when it is determined that the access interface is the first interface, execute the full update operation corresponding to the data storage request, and when it is determined that the access interface is the second interface, obtain the target message corresponding to the data storage request, and determine whether the target message contains a target attribute field, the target attribute field is a field used to determine whether to perform a full update operation on the target message, and when it is determined that the target message contains the target attribute field, execute the full update operation corresponding to the data storage request. Thus, it is possible to determine whether to perform a full update operation by determining the type of the access interface and / or the situation in which the target message contains the target attribute field. When it is determined that the access interface is the first interface, or when the access interface is the second interface but the target message contains the target attribute field, execute the full update operation, thereby avoiding resource consumption caused by performing a regular update operation each time a data storage request is executed, thereby improving system performance.
[0056] On the basis of the above-mentioned embodiments of the present disclosure, after obtaining the target message corresponding to the data storage request and determining whether the target message contains the target attribute field, the data storage method may further include: when it is determined that the target message does not contain the target attribute field, obtaining the field information to be stored corresponding to the target message; comparing the field information to be stored with the preset index mapping relationship; and determining whether to perform a full update operation based on the comparison result.
[0057] The preset index mapping relationship is a mapping relationship pre-set for the index before creating the index, wherein the index mapping relationship specifies the attribute fields and field types corresponding to each piece of data in the index, that is, which fields are in each piece of data and the field types of these fields.
[0058] The attribute fields in an indexed relation determine how it is indexed and which attributes of the index should be stored.
[0059] The field information to be stored includes the number of fields to be stored (ie, the number of fields corresponding to the target data to be stored) and the type of fields to be stored (ie, the type of fields corresponding to the target data to be stored).
[0060] Specifically, after obtaining the to-be-stored field information corresponding to the target message, determine whether there is an index mapping relationship corresponding to the target message. If not, it means that the index mapping relationship has not been configured, and the full update operation is directly performed at this time; if it exists, obtain the index mapping relationship, compare the to-be-stored field information with the preset index mapping relationship, obtain the comparison result, and determine whether to perform the full update operation based on the comparison result.
[0061] In the embodiments of the present disclosure, when it is determined that the access interface is the second interface and the target message does not contain the target attribute field, the field information to be stored corresponding to the target message is compared with the preset index mapping relationship to determine which fields are involved in the write or update corresponding to the data storage request and what the updated value is, so as to determine whether it is the writing of new data or the complete update of the original data. If so, a full update operation is performed. In this way, the flexibility and diversity of determining whether to perform a full update operation are improved, thereby further avoiding the resource consumption caused by directly executing conventional update operations, improving system performance, and reducing resource consumption.
[0062] Furthermore, the field information to be stored is compared with the preset index mapping relationship, which may specifically include: comparing the number of fields to be stored corresponding to the field information to be stored with the number of fields in the preset index mapping relationship, and at the same time comparing the type of field to be stored corresponding to the field information to be stored with the field type in the preset index mapping relationship to obtain a comparison result.
[0063] In an embodiment of the present disclosure, determining whether to perform a full update operation is based on the comparison result, which may specifically include: when the comparison result is that the number of fields to be stored is consistent with the number of fields in the preset index mapping relationship, and the type of the field to be stored is consistent with the field type in the preset index mapping relationship, performing a full update operation; when the comparison result is that the number of fields to be stored is inconsistent with the number of fields in the preset index mapping relationship, and / or the type of the field to be stored is inconsistent with the field type in the preset index mapping relationship, performing a regular update operation.
[0064] Exemplarily, the index mapping relationship includes four attribute fields, namely, email, address, name, and age, and the field type of each attribute field. If the number of fields to be stored corresponding to the target message is 4, and the type of the field to be stored is consistent with the field type of each attribute field in the index mapping relationship, it means that the request to be stored is to update all attribute fields in the index, and then perform a full update operation; if the number of fields to be stored corresponding to the target message is less than 4, such as 3, it means that the request to be stored is to update some attribute fields in the index, and then perform a regular update operation.
[0065] In the disclosed embodiment, it is possible to create an index mapping relationship for the index in advance, and determine whether to perform a full update operation by comparing the to-be-stored field information corresponding to the target message with the index mapping relationship, thereby improving the flexibility, diversity and accuracy of determining whether to perform a full update operation.
[0066] In some embodiments of the present disclosure, after acquiring the index mapping relationship, the electronic device may also dynamically update the index mapping relationship based on a preset interface in response to a user's target operation, including operations such as creating, modifying, and deleting the index mapping relationship.
[0067] In the disclosed embodiment, the index mapping relationship can be dynamically updated to ensure that the index mapping relationship meets the actual usage requirements of the user, thereby improving the user experience.
[0068] In the embodiment of the present disclosure, the data storage method may further include: when it is determined that the target message does not contain the target attribute field, performing a regular update operation.
[0069] Figure 2 is a flow chart of a conventional update operation execution method provided by an embodiment of the present disclosure, such as Figure 2 As shown, performing a regular update operation specifically includes the following steps:
[0070] S210: Determine whether the first data of the user-defined data primary key corresponding to the data storage request already exists.
[0071] In the disclosed embodiment, when the electronic device determines to perform a regular update operation, it searches the cluster for the first data of the custom data primary key by means of data query. If not, it executes step S220; if so, it executes steps S230-S260.
[0072] S220. When it is determined that the first data does not exist, write the first data into the cluster.
[0073] In the embodiment of the present disclosure, when the electronic device determines that the first data does not exist, the electronic device writes the first data into the cluster, and returns the update result after the writing is completed.
[0074] S230: When it is determined that the first data already exists, read the first data from the cluster, and obtain target data in a target message corresponding to the data storage request.
[0075] S240: Determine whether data is stored repeatedly based on the first data and the target data.
[0076] In the embodiment of the present disclosure, when the electronic device determines that the first data already exists, it reads the first data and obtains the target data in the target message, compares the first data with the target data, and determines whether the first data and the target data are consistent. If they are consistent, it is determined that the data is stored repeatedly; if they are inconsistent, it is determined that the data is not stored repeatedly but an update of part of the data.
[0077] S250: When it is determined that the data is not stored repeatedly, the first data and the target data are merged, the merged second data is stored, and the first data is deleted.
[0078] In the disclosed embodiment, the electronic device merges the first data and the target data, and the specific implementation method of storing the merged second data is similar to the existing implementation method of updating the data in a preset database such as a relational database, and will not be elaborated here.
[0079] S260: When it is determined that the data is stored repeatedly, an update result is returned.
[0080] In the disclosed embodiment, when it is determined not to perform a full update operation, a regular update operation can be continued to be performed, which avoids resource consumption caused by directly performing the regular update operation and does not affect the client accessing the cluster, thereby improving the user experience.
[0081] In the embodiment of the present disclosure, performing a full update operation may specifically include: writing target data in a target message corresponding to a data storage request into a cluster.
[0082] Specifically, when the electronic device determines to perform a full update operation, it determines whether there is a custom primary key for the request to be stored. If so, the target data is directly written into the cluster; if there is no custom primary key, the full update operation is automatically downgraded to a normal update process.
[0083] In the disclosed embodiment, when it is determined that the data storage request is for a full write or re-write of the target data, the process of verifying whether all data exist can be skipped, and the target data can be directly written into the cluster, and the data results in the cluster can be overwritten with the new data, i.e., the target data. This can reduce the data verification link and avoid the need to add an additional data reading process to check whether the first data of the custom primary key already exists in the cluster, thereby effectively reducing the IO load, thereby reducing resource consumption and improving system performance.
[0084] Figure 3 is a flowchart of another data storage method provided by an embodiment of the present disclosure, such as Figure 3 As shown, the data storage method may specifically include the following steps:
[0085] S310: In response to a data storage request from a user, when it is determined that the data storage request uses a user-defined data primary key, determine an access interface corresponding to the data storage request.
[0086] In the embodiment of the present disclosure, when it is determined that the access interface is the first interface, step S320 is executed; when it is determined that the access interface is the second interface, step S330 is executed.
[0087] S320: When it is determined that the access interface is the first interface, execute a full update operation corresponding to the data storage request.
[0088] S330. When it is determined that the access interface is the second interface, obtain a target message corresponding to the data storage request, and determine whether the target message contains a target attribute field, where the target attribute field is a field used to determine whether to perform a full update operation on the target message.
[0089] In the embodiment of the present disclosure, when it is determined that the target message does not contain the target attribute field, steps S340-S370 are executed; when it is determined that the target message contains the target attribute field, step S380 is executed.
[0090] S340: When it is determined that the target message does not contain the target attribute field, obtain the to-be-stored field information corresponding to the target message.
[0091] S350: Compare the field information to be stored with the preset index mapping relationship to obtain a comparison result.
[0092] S360: When the comparison result shows that the number of fields to be stored is consistent with the number of fields in the preset index mapping relationship, and the type of the field to be stored is consistent with the type of the field in the preset index mapping relationship, a full update operation is performed.
[0093] S370: When the comparison result shows that the number of fields to be stored is inconsistent with the number of fields in the preset index mapping relationship, and / or the type of the field to be stored is inconsistent with the type of the field in the preset index mapping relationship, a regular update operation is performed.
[0094] S380. When it is determined that the target message contains a target attribute field, a full update operation corresponding to the data storage request is executed.
[0095] It should be noted that the specific implementation of steps S310-S380 is similar to the implementation of the relevant steps in the above-mentioned embodiment of the present disclosure, and will not be repeated here.
[0096] In the embodiments of the present disclosure, it is possible to first determine whether to execute a full update operation through an access interface; when it is impossible to determine whether to execute a full update operation based on the access interface, determine whether to execute a full update operation based on the target attribute field in the target message corresponding to the data storage request; if the target message does not contain the target attribute field, determine whether it is a full update operation based on the to-be-stored field information corresponding to the target message and the preset index mapping relationship, and provide a variety of different ways to determine whether to execute a full update operation based on different data processing stages, thereby ensuring that a full update operation is executed in any case where the full update is satisfied, thereby minimizing the resource consumption caused by directly executing conventional update operations and improving system performance.
[0097] Figure 4 It is a structural schematic diagram of a data storage device provided by an embodiment of the present disclosure.
[0098] In the embodiments of the present disclosure, the data storage device may be provided in an electronic device and may be understood as a part of the functional modules in the electronic device. Specifically, the electronic device may be a server or a terminal, wherein the terminal may include a mobile phone, a computer or a tablet computer, etc., or may be any device capable of processing the data storage method, which is not limited here.
[0099] like Figure 4 As shown, the data storage device 400 may include an interface determination module 410 , a first update module 420 , an attribute field determination module 430 , and a second update module 440 .
[0100] The interface determination module 410 may be used to respond to a data storage request from a user, and when it is determined that the data storage request uses a user-defined data primary key, determine an access interface corresponding to the data storage request.
[0101] The first update module 420 may be configured to execute a full update operation corresponding to the data storage request when it is determined that the access interface is the first interface.
[0102] The attribute field determination module 430 can be used to obtain the target message corresponding to the data storage request when determining that the access interface is the second interface, and determine whether the target message contains a target attribute field, which is a field used to determine whether to perform a full update operation on the target message.
[0103] The second update module 440 may be configured to execute a full update operation corresponding to the data storage request when it is determined that the target message contains a target attribute field.
[0104] In an embodiment of the present disclosure, it is possible to respond to a user's data storage request, and when it is determined that the data storage request uses a custom data primary key, determine the access interface corresponding to the data storage request, and when it is determined that the access interface is the first interface, execute the full update operation corresponding to the data storage request, and when it is determined that the access interface is the second interface, obtain the target message corresponding to the data storage request, and determine whether the target message contains a target attribute field, the target attribute field is a field used to determine whether to perform a full update operation on the target message, and when it is determined that the target message contains the target attribute field, execute the full update operation corresponding to the data storage request. Thus, it is possible to determine whether to perform a full update operation by determining the type of the access interface and / or the situation in which the target message contains the target attribute field. When it is determined that the access interface is the first interface, or when the access interface is the second interface but the target message contains the target attribute field, execute the full update operation, thereby avoiding resource consumption caused by performing a regular update operation each time a data storage request is executed, thereby improving system performance.
[0105] In some embodiments of the present disclosure, the data storage device 400 may further include an information comparison module.
[0106] The information comparison module can be used to obtain the target message corresponding to the data storage request and determine whether the target message contains the target attribute field. When it is determined that the target message does not contain the target attribute field, obtain the field information to be stored corresponding to the target message; compare the field information to be stored with the preset index mapping relationship; and determine whether to perform a full update operation based on the comparison result.
[0107] In some embodiments of the present disclosure, the information comparison module can be specifically used to compare the number of fields to be stored corresponding to the field information to be stored with the number of fields in a preset index mapping relationship, and at the same time compare the type of fields to be stored corresponding to the field information to be stored with the field type in the preset index mapping relationship to obtain a comparison result.
[0108] In some embodiments of the present disclosure, the information comparison module can also be specifically used to perform a full update operation when the comparison result is that the number of fields to be stored is consistent with the number of fields in a preset index mapping relationship, and the type of fields to be stored is consistent with the field type in the preset index mapping relationship.
[0109] In some embodiments of the present disclosure, the data storage device 400 may further include a third update module.
[0110] The third update module can be used to compare the number of fields to be stored corresponding to the field information to be stored with the number of fields in the preset index mapping relationship, and at the same time compare the type of fields to be stored corresponding to the field information to be stored with the field type in the preset index mapping relationship. When the comparison result is that the number of fields to be stored is inconsistent with the number of fields in the preset index mapping relationship, and / or the type of field to be stored is inconsistent with the field type in the preset index mapping relationship, perform a regular update operation.
[0111] In some embodiments of the present disclosure, the data storage device 400 may further include a fourth update module.
[0112] The fourth updating module may be configured to perform a regular updating operation when it is determined that the target message does not contain the target attribute field.
[0113] The fourth update module can be specifically used to determine whether the first data of the custom data primary key corresponding to the data storage request already exists; when it is determined that the first data already exists, read the first data from the cluster and obtain the target data in the target message corresponding to the data storage request; determine whether the data is stored repeatedly based on the first data and the target data; when it is determined that the data is not stored repeatedly, merge the first data and the target data, store the merged second data, and delete the first data; when it is determined that the first data does not exist, write the first data into the cluster.
[0114] In some embodiments of the present disclosure, the first update module 420 or the second update module 440 may be specifically configured to write target data in a target message corresponding to a data storage request into a cluster.
[0115] It should be noted that Figure 4 The data storage device 400 shown can execute each step in the above method embodiment and realize each process and effect in the above method embodiment, which will not be described in detail here.
[0116] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure.
[0117] In the disclosed embodiment, Figure 5The electronic device shown may be a server or a terminal, wherein the terminal specifically includes a mobile phone, a computer or a tablet computer, etc. It may also be any device that can process the data storage method, and is not limited here.
[0118] like Figure 5 As shown, the electronic device may include a processor 510 and a memory 520 storing computer program instructions.
[0119] Specifically, the processor 510 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.
[0120] The memory 520 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 520 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 520 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 520 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 520 is a non-volatile solid-state memory. In a particular embodiment, the memory 520 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.
[0121] The processor 510 reads and executes the computer program instructions stored in the memory 520 to perform the steps of the data storage method provided in the embodiment of the present disclosure.
[0122] In one example, the electronic device may further include a transceiver 530 and a bus 540. Figure 5 As shown, the processor 510, the memory 520 and the transceiver 530 are connected via a bus 540 and communicate with each other.
[0123] The bus 540 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 540 may include one or more buses.
[0124] The embodiments of the present disclosure further provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the data storage method provided by the embodiments of the present disclosure.
[0125] The above-mentioned storage medium may, for example, include a memory 520 of computer program instructions, and the above-mentioned instructions may be executed by the processor 510 of the electronic device to complete the data storage method provided by the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.
[0126] The embodiments of the present disclosure also provide a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the data storage method provided by the embodiments of the present disclosure is implemented, and the various processes and effects in the above-mentioned embodiments of the present disclosure can be implemented, which will not be elaborated here.
[0127] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data storage method, characterized in that: include: In response to a data storage request from a user, when it is determined that the data storage request uses a user-defined data primary key, determining an access interface corresponding to the data storage request; When it is determined that the access interface is the first interface, executing a full update operation corresponding to the data storage request; When it is determined that the access interface is the second interface, obtaining a target message corresponding to the data storage request, and determining whether the target message includes a target attribute field, the target attribute field being a field used to determine whether to perform a full update operation on the target message; When it is determined that the target message contains a target attribute field, a full update operation corresponding to the data storage request is performed.
2. The method according to claim 1, characterized in that After obtaining the target message corresponding to the data storage request and determining whether the target message includes a target attribute field, the method further includes: When it is determined that the target message does not contain the target attribute field, obtaining the to-be-stored field information corresponding to the target message; Comparing the field information to be stored with the preset index mapping relationship; Based on the comparison result, it is determined whether to execute the full update operation.
3. The method according to claim 2, characterized in that The comparing the field information to be stored with a preset index mapping relationship includes: Compare the number of fields to be stored corresponding to the field information to be stored with the number of fields in the preset index mapping relationship, and compare the type of fields to be stored corresponding to the field information to be stored with the type of fields in the preset index mapping relationship to obtain the comparison result; The determining whether to perform the full update operation based on the comparison result includes: When the comparison result is that the number of fields to be stored is consistent with the number of fields in the preset index mapping relationship, and the type of fields to be stored is consistent with the type of fields in the preset index mapping relationship, the full update operation is performed.
4. The method according to claim 3, characterized in that After comparing the number of fields to be stored corresponding to the field information to be stored with the number of fields in the preset index mapping relationship, and comparing the type of fields to be stored corresponding to the field information to be stored with the type of fields in the preset index mapping relationship, the method further includes: When the comparison result is that the number of fields to be stored is inconsistent with the number of fields in the preset index mapping relationship, and / or the type of fields to be stored is inconsistent with the type of fields in the preset index mapping relationship, a regular update operation is performed.
5. The method according to claim 1, characterized in that The method further comprises: When it is determined that the target message does not contain the target attribute field, performing a conventional update operation; The performing of the conventional update operation comprises: Determine whether the first data of the user-defined data primary key corresponding to the data storage request already exists; When it is determined that the first data already exists, read the first data from the cluster, and obtain the target data in the target message corresponding to the data storage request; Determining whether data is stored repeatedly based on the first data and the target data; When it is determined that the data is not stored repeatedly, merging the first data and the target data, storing the merged second data, and deleting the first data; When it is determined that the first data does not exist, the first data is written into the cluster.
6. The method according to claim 1 or 3, characterized in that: Perform a full update operation, including: The target data in the target message corresponding to the data storage request is written into the cluster.
7. A data storage device, characterized in that: include: An interface determination module, for responding to a data storage request from a user, and when determining that the data storage request uses a user-defined data primary key, determining an access interface corresponding to the data storage request; A first update module, configured to execute a full update operation corresponding to the data storage request when determining that the access interface is the first interface; an attribute field determination module, for obtaining a target message corresponding to the data storage request when determining that the access interface is the second interface, and determining whether the target message contains a target attribute field, wherein the target attribute field is a field for determining whether to perform a full update operation on the target message; The second update module is used to execute a full update operation corresponding to the data storage request when it is determined that the target message contains a target attribute field.
8. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the data storage method described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the data storage method described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or instruction is executed by a processor, the data storage method according to any one of claims 1 to 6 is implemented.