Data processing method, device and system

Abstract

The invention provides a data processing method, device and system, and relates to the field of computers. The method is applied to a server and a client of the semi-structured key value database. The data operation request sent by the client to the server comprises historical data content, and the historical data content is a value corresponding to the target primary key before the server executes the data operation corresponding to the data operation request. And the server executes a data operation corresponding to the data operation request on the data content, determines the change content of a value corresponding to the target primary key according to the historical data content, updates the secondary index according to the change content, and returns a data operation response to the client. Therefore, the server side does not need to read the data content of the key value data from the semi-structured key value database, and one time of data content reading operation is reduced, so that the response time delay of the data operation request is reduced, and the data processing efficiency of the semi-structured key value database is improved.

Description

[0001] This application claims priority to Chinese patent application No. 202310876351.8 filed on July 17, 2023, with invention name “A method, device and other equipment for data processing”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data processing method, device and system. Background Art

[0003] Semi-structured key-value databases are usually used to manage semi-structured key-value data, which includes a primary key index and may also include a secondary index (si). In a semi-structured key-value database, a primary key index is usually used to access the data content contained in the primary key index, and a secondary index may also be used to quickly access the data content.

[0004] When operating on the semi-structured key-value data of a semi-structured key-value database, the data content of the primary key index is changed, and the secondary index points to the primary key index based on part of the data content contained in the primary key index. Changes in the data content may cause changes in the secondary index. Therefore, after operating on the semi-structured key-value data, the primary key index and secondary index corresponding to the semi-structured key-value data need to be updated. However, the semi-structured key-value database has a long processing delay in updating the primary key index and secondary index, which reduces the efficiency of the semi-structured key-value database in data processing. Summary of the invention

[0005] The present application provides a data processing method, device and system, thereby improving the efficiency of data processing in a semi-structured key-value database.

[0006] In a first aspect, the present application provides a data processing method, which is applied to a semi-structured key-value database. The semi-structured key-value database includes a client and a server. The key-value data stored in the semi-structured key-value database includes a primary key index and a secondary index, the primary key index includes a primary key and a value corresponding to the primary key, the value corresponding to the primary key is the data content contained in the semi-structured key-value data, and the secondary index is used to point to the primary key index based on part of the data content. In the process of the data processing method, first, the client sends a data operation request to the server. The data operation request includes target data content and historical data content, and the historical data content is the value corresponding to the target primary key before the server performs the data operation corresponding to the data operation request according to the target data content. Then, the server performs the data operation corresponding to the data operation request according to the target data content, wherein the data operation corresponding to the data operation request may include overwriting (put) and updating (update). After completing the data operation, the server determines the data content corresponding to the target primary key after the operation relative to the change content before the operation according to the historical data content, including deleted content and newly added content. Finally, the server updates the secondary index according to the changed content, for example, deletes the secondary index corresponding to the deleted content, and adds the secondary index corresponding to the newly added content.

[0007] The data content of the semi-structured key-value data stored in the semi-structured key-value database may be referred to as a document (doc).

[0008] In most data operation scenarios, before the client requests to overwrite or update key-value data, the client usually has read the value corresponding to the target primary key from the server, that is, the data content of the key-value data, to generate a data operation request based on the data content of the key-value data. For example, before the client sends a data operation request for an update operation, the client queries the data content of the key-value data from the server to generate a data operation request for an update operation based on the data content of the key-value data. However, when the client sends a data operation request to the server, it usually does not carry the data content of the key-value data that has been read. The server needs to re-read the data content of the key-value data from the semi-structured key-value database to perform subsequent operations on the data operation request.

[0009] Compared with the above data operation process, in the data processing method of the present application, the client carries the target data content indicating the data operation in the data operation request sent to the server, and also carries the data content of the key-value data that has been read, that is, the historical data content. When the server executes the data operation corresponding to the data operation request, it can update the index of the semi-structured key-value data based on the historical data content. In this way, compared with the server reading the data content of the semi-structured key-value data in the semi-structured key-value database after receiving the data operation request, and then updating the index based on the read data content, the data content reading operation is reduced once, thereby reducing the response delay of the data operation request and improving the data processing efficiency of the semi-structured key-value database.

[0010] As a possible implementation method, when the data content sent by the client is the data content currently stored in the semi-structured key-value database, that is, when the historical data content is consistent with the data content contained in the semi-structured key-value data in the semi-structured key-value database, the server updates the secondary index according to the changed content. In this way, when the server confirms that the historical data content carried by the data operation request is the latest record stored in the semi-structured key-value database, it updates the secondary index based on the historical data content, thereby avoiding the situation where the data content of the semi-structured key-value database on the server has been modified and the secondary index is incorrectly updated based on the wrong version of the historical data content.

[0011] Optionally, the server can cache the version information of the data content after each modification when the data content of the semi-structured key-value data in the semi-structured key-value database is modified. On the basis of caching the version information of each data content, when the server receives the historical data content, it can determine whether the historical data content is the latest record stored in the semi-structured key-value database based on the comparison between the version information of the historical data content and the version information of the cached data content. For example, when the version information of the historical data content is consistent with the version information of the data content of the semi-structured key-value data cached by the server, the server determines that the historical data content is consistent with the data content contained in the semi-structured key-value data in the semi-structured key-value database. In this way, the server can quickly confirm whether the historical data content is the latest record based on the version information of the cached data content, thereby improving the overall efficiency of data operations.

[0012] The version information of the data content may include table schema version information and data version information. For example, the data version information is document version information. The table schema version information is used to indicate the version of the table schema. The table schema is used to define the primary key index and secondary index of semi-structured key-value data. Users can specify the primary key index and secondary index through the table schema to access the document. The data version information is used to indicate the version of the data content.

[0013] As a possible implementation manner, before the client sends a data operation request to the server so that the server performs a corresponding data operation on the semi-structured key-value data, the client has read the data content of the semi-structured key-value data from the server.

[0014] Optionally, the client sends a data read request to the server before sending a data operation request to the server. The server sends the historical data content and the version information of the historical data content of the semi-structured key-value data to the client according to the data read request, and the client receives and stores the historical data content and the version information of the historical data content. In this way, a data basis is provided for the client to carry the historical data content and / or the version information of the historical data content when sending a data operation request subsequently.

[0015] As a possible implementation method, after receiving the historical data content carried in the data operation request, the server compares the historical data content with the data content of the semi-structured key-value data after the data operation is performed to obtain the changed content of the data.

[0016] In a second aspect, the present application provides a data processing method, which is applied to a semi-structured key-value database. The semi-structured key-value database includes a client and a server. The key-value data stored in the semi-structured key-value database includes a primary key index and a secondary index, the primary key index includes a primary key and a value corresponding to the primary key, the value corresponding to the primary key is the data content contained in the semi-structured key-value data, and the secondary index is used to point to the primary key index based on part of the data content. In the process of the data processing method, first, the client sends a data operation request to the server. The data operation request includes historical data content, and the historical data content is the value corresponding to the target primary key before the server performs the data operation corresponding to the data operation request. Then, the server performs the data operation corresponding to the data operation request, wherein the data operation corresponding to the data operation request may include deletion. After completing the data operation, the server determines the change content of the value corresponding to the target primary key after the operation relative to the change content before the operation according to the historical data content, including the deletion content. Finally, the server updates the secondary index according to the changed content, for example, deletes the secondary index corresponding to the deletion content.

[0017] The second aspect differs from the first aspect in that the data operation type corresponding to the data operation request is different, and the steps for the server to update the secondary index are different, for example, in the second aspect, the server does not add the secondary index corresponding to the newly added content. The second aspect can be combined with any possible implementation of the first aspect, which will not be described in detail here.

[0018] In a third aspect, the present application provides a data processing system, including a client and a server. The client is used to send a data operation request to the server, the data operation request includes target data content and / or historical data content, and the historical data content is the value corresponding to the target primary key before the server responds to the data operation request. The server is used to perform the data operation corresponding to the data operation request according to the target data content, and determine the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deleted content and / or newly added content. Finally, the server is also used to delete the secondary index corresponding to the deleted content, and / or add the secondary index corresponding to the newly added content.

[0019] In a fourth aspect, the present application provides a data processing device, including a transceiver module, an operation module and an index module. The transceiver module is used to receive a data operation request sent by a client, the data operation request includes target data content and historical data content, and the historical data content is the value corresponding to the target primary key before the server responds to the data operation request. The operation module is used to perform data operations corresponding to the data operation request according to the target data content, the data operations include updating or overwriting, and determine the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deleted content and newly added content. The index module is used to delete the secondary index corresponding to the deleted content, and add the secondary index corresponding to the newly added content.

[0020] As a possible implementation manner, the data processing device may further include other modules for executing the operation steps of the data processing method described in the first aspect.

[0021] In a fifth aspect, the present application provides a data processing device, including a transceiver module, an operation module and an index module. The transceiver module is used to receive a data operation request sent by a client, and the data operation request includes historical data content, and the historical data content is the value corresponding to the target primary key before the server responds to the data operation request. The operation module is used to perform data operations corresponding to the data operation request, and the data operations include deletion, and determine the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes the deletion content. The index module is used to delete the secondary index corresponding to the deletion content.

[0022] As a possible implementation manner, the data processing device may further include other modules for executing the operation steps of the data processing method described in the second aspect.

[0023] Regarding the technical principles and beneficial effects of the second, third, fourth and fifth aspects, reference may be made to the relevant description of the first aspect above and will not be repeated here.

[0024] In a sixth aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory. The processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the data processing method described in any possible implementation of the first aspect.

[0025] In a seventh aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory. The processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the data processing method described in any possible implementation of the second aspect above.

[0026] In an eighth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are executed on a computer, the computer executes the data processing method described in any possible implementation of the first aspect.

[0027] In a ninth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer executes the data processing method described in any possible implementation of the second aspect.

[0028] In a tenth aspect, a computer-readable storage medium is provided. The computer-readable storage medium includes: a computer program or an instruction; when the computer program or the instruction is executed on a computer, the computer executes the data processing method described in any possible implementation of the first aspect.

[0029] In an eleventh aspect, a computer-readable storage medium is provided. The computer-readable storage medium includes: a computer program or instruction; when the computer program or instruction is executed on a computer, the computer executes the data processing method described in any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the architecture of a computer system provided for this application;

[0031] Figure 2a A flowchart of a data processing method provided in this application;

[0032] Figure 2b A flowchart of another data processing method provided by the present application;

[0033] Figure 3 A flowchart of another data processing method provided by the present application;

[0034] Figure 4 A schematic diagram of the structure of a data processing device provided by the present application;

[0035] Figure 5 A structural schematic diagram of another data processing device provided by the present application;

[0036] Figure 6 A schematic diagram of the structure of a data processing system provided for this application;

[0037] Figure 7 A schematic diagram of the structure of a computing device provided for this application;

[0038] Figure 8 A schematic diagram of the structure of a computing device cluster provided for this application;

[0039] Fig. 9 It is a structural diagram of a network connection between computing devices provided by the present application. DETAILED DESCRIPTION

[0040] The data processing method, device and system provided in the embodiments of the present application can be applied to the scenario of a semi-structured key-value database, and a brief introduction is given to the technologies that may be involved in the present application.

[0041] (1) Semi-structured key-value database

[0042] Compared with structured data (i.e. row data, which is stored in the database and can be logically expressed and realized using a two-dimensional table structure), data that is not convenient to be expressed using a two-dimensional database logical table is called unstructured data, including office documents of all formats, texts, pictures, various reports, images, and audio / video information, etc. Semi-structured data is data between fully structured data (such as data in relational databases and object-oriented databases) and completely unstructured data (such as sound and image files, etc.). The structure of semi-structured data is hidden in the data, and the structure and content of the data are mixed together. For example, semi-structured key-value data includes a primary key index and a secondary index. The primary key index includes the primary key and the value corresponding to the primary key. The value corresponding to the primary key is the stored data content, i.e., the document. The secondary index includes the secondary index key and the value corresponding to the secondary index key. The secondary index key is the value of the field in the document, and the value corresponding to the secondary index key is the primary key of the primary key index to which the document belongs.

[0043] Semi-structured key-value data is maintained by a semi-structured key-value database, which can be a non-relational database. Based on the non-structured key-value database, the semi-structured key-value database perceives the data type and structure of the value (such as list, set, and map, etc.), supports users to define the schema of the semi-structured key-value database, and implements the definition of secondary indexes and supporting query capabilities.

[0044] A schema is a logical concept used to organize objects in a database. Objects in a schema usually include tables, indexes, data types, sequences, views, stored procedures, primary keys, foreign keys, and so on.

[0045] (2) Non-relational database

[0046] Non-relational databases, also known as NoSQL (not only structured query language), use a data model that is different from the "row and column" organization of relational data and can support a variety of data types. For example, non-relational databases are used to store semi-structured key-value data. The structure of each semi-structured key-value data is not fixed. Each semi-structured key-value data can have different fields, and users can add or delete fields as needed. In non-relational databases, users obtain different information without the need to perform associated queries on multiple tables like in relational databases.

[0047] The following is a brief example of the database's operation on indexes when writing semi-structured key-value data to a non-relational database.

[0048] a. insert

[0049] insert(primary key index), insert(secondary index).

[0050] b. Delete

[0051] delete(primary key index), delete(secondary index).

[0052] c. Overwrite (put)

[0053] delete (old secondary index), insert (new primary key index), insert (new secondary index).

[0054] d. Update

[0055] delete (old secondary index), insert (new primary key index), insert (new secondary index).

[0056] Among them, insert, delete, etc. are database operation languages ​​corresponding to non-relational databases, which are instruction sets for running data access work on database objects. The insert statement is used to add data to a data table, and delete is used to delete data.

[0057] Taking the overwrite operation as an example, when the server for maintaining a semi-structured key-value database receives a data overwrite operation request, the server first reads the old data content contained in the primary key index according to the primary key carried in the data overwrite operation request. Then, the server compares the new data content overwritten by the data overwrite operation request with the read old data content, generates a deletion record of the secondary index based on the field changes in the new data content and the old data content, and then generates an insertion record of the primary key index and the secondary index based on the new data content. Finally, the server writes the deletion record and the insertion record into the index file.

[0058] The present application provides a data processing method, specifically a data processing method of "updating an index according to the historical data content carried by a data operation request". The client sends a data operation request to the server, and the data operation request includes historical data content, and the historical data content is the value corresponding to the target primary key when the server does not execute the data operation request. The server performs the data operation corresponding to the data operation request on the data content. The server updates the secondary index according to the changed content of the value corresponding to the target primary key after executing the data operation corresponding to the data operation request.

[0059] For data operations such as updating and overwriting, the data operation request also includes the target data content. The server performs the data operation corresponding to the data operation request based on the target data content. The step of updating the secondary index includes deleting the secondary index corresponding to the deleted content and adding the secondary index corresponding to the newly added content.

[0060] For data operations such as deletion, the step of updating the secondary index includes deleting the secondary index corresponding to the deleted content.

[0061] In this way, compared with the traditional non-relational database server that needs to read the old data content when receiving a data operation request, the server performs subsequent index updates based on the old data content carried in the data operation request sent by the client, thereby reducing the delay caused by reading the old data content, and then reducing the processing delay of the data operation request, thereby improving the overall efficiency of data operations.

[0062] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0063] Figure 1 This is a schematic diagram of the architecture of a computer system provided in this application. Figure 1As shown, the computer system 100 includes a computing server cluster 110, a storage server cluster 120, a management server cluster 130, a network device cluster 140 and a user terminal 150. The computing server cluster 110, the storage server cluster 120 and the management server cluster 130 communicate with the user terminal 150 through the network device cluster 140 respectively.

[0064] The computing server cluster 110 includes one or more computing servers ( Figure 1 Two computing servers, namely computing server 111 and computing server 112, are shown in FIG. 1 , but are not limited to two computing servers).

[0065] The computing server is a computing resource in the computer system 100, such as a server, a desktop computer, etc., which is used to generate and allocate computing resources according to user needs based on virtualization technology. At the hardware level, the computing server is equipped with a processor and a memory ( Figure 1 The computing function of the computing server is realized by the processor running the program in the memory. The computing server can also read / write data in each storage server in the storage server cluster 120 according to user needs.

[0066] The storage server cluster 120 includes one or more storage servers ( Figure 1 Two storage servers, storage server 121 and storage server 122, are shown in FIG. 1 , but are not limited to two storage servers).

[0067] The storage server is a storage resource in the computer system 100, such as a server, desktop computer or storage array controller, hard disk frame, etc., which is used to provide logical disk storage, semi-structured data storage, integrated backup and other services for the cloud virtual machine in the computer system 100. In terms of hardware, the storage server is provided with a network card, a processor and a memory. The processor in the storage server is used to process data from outside the storage server. The network card is used to control the access process of the memory, such as the control of address signals, data signals and various command signals, so that the storage server can provide the memory as a storage resource to the user. The memory is used to store data and may include memory and / or hard disk. Memory refers to an internal memory that directly exchanges data with the processor. The memory can read and write data quickly at any time and serve as a temporary data storage for the operating system or other running programs. Unlike memory, the hard disk reads and writes data slower than memory and is usually used to store data persistently.

[0068] For example, the storage server is used to store semi-structured key-value data contained in a semi-structured key-value database. The key-value data includes a primary key index and a secondary index.

[0069] The management server cluster 130 includes one or more management servers ( Figure 1 Two management servers, namely, management server 131 and management server 132, are shown in FIG. 1 , but are not limited to two management servers).

[0070] The management server is used to manage all computing services, shared storage, and networks of the entire computer system 100, and provide users or administrators with an application program interface (API) for managing the entire node. In the present application, the computer system 100 can provide the program product of the present application to users by providing an accessible application program interface. For example, the application program interface encapsulates the code of the data operation request provided by the present application.

[0071] The network device cluster 140 includes one or more switches and routers, such as Figure 1 As shown, in this embodiment, the network device cluster 140 includes a router 141, a switch 142, a switch 143, a switch 144, and a switch 145. Among them, the user terminal 150 is connected to the router 141 through the Internet, and the router 141 is connected to the switch 143 through the switch 142, and the switch 143 is connected to each computing server in the computing server cluster 110. The switch 144 is respectively connected to each computing server in the computing server cluster 110 and each storage server in the storage server cluster 120. The switch 145 is respectively connected to each computing server in the computing server cluster 110, each storage server in the storage server cluster 120, and each management server in the management server cluster 130.

[0072] Optionally, the number and type of switches included in the network device cluster 140 can be adjusted according to the needs of the cloud platform 100, and the switches 142, 143, 144, and 145 can be switches with different functions, for example, the switch 142 is a core switch, and the switches 143, 144, and 145 are switches for managing specific network segments. For example, the switch 142 can be a core switch, the switch 143 can be an internal and external switching network segment switch, the switch 144 can be a storage network segment switch, and the switch 145 can be a management network segment switch.

[0073] The user terminal 150 includes one or more user terminals ( Figure 1 Two user terminals, namely user terminal 151 and user terminal 152, are shown in FIG. 1 , but are not limited to two user terminals). The user terminal includes an interface and application required for accessing the computer system 100, such as a client of a semi-structured key-value database.

[0074] It is worth noting that Figure 1 This is only a schematic diagram and should not be construed as limiting the present application. The computer system 100 may also include other devices. Figure 1 Not drawn in.

[0075] In such Figure 1 Based on the computer system 100 shown, the computer system 100 can implement the functions of a semi-structured key-value database based on infrastructure such as computing servers and storage servers.

[0076] For example, the computing server is deployed with a server of a semi-structured key-value database, and the user terminal is deployed with a client of the semi-structured key-value database. The server is used to receive a data operation request sent by the client, and the data operation request includes target data content and / or historical data content, and the historical data content is the value corresponding to the target primary key before the server executes the data operation corresponding to the data operation request. Then, the server is used to execute the data operation corresponding to the data operation request, and update the secondary index according to the changed content of the value corresponding to the target primary key after the server executes the data operation corresponding to the data operation request.

[0077] As another example, the computing server caches version information of the data content of the key-value data. The server is also used to determine the validity of the historical data content carried by the data operation request based on the version information of the data content of the key-value data. When the historical data content is valid, the server updates the secondary index based on the changes in the data content relative to the historical data content after the server executes the data operation corresponding to the data operation request. When the historical data content is invalid, the server reads the old data content (i.e., the historical data content) contained in the primary key index based on the primary key, and then updates the secondary index based on the historical data content.

[0078] In a possible embodiment of the present application, the server and client of the semi-structured key-value database can also be regarded as hardware resources. For example, the computing server that processes data operation requests is called the server, and the user terminal that sends data operation requests is called the client.

[0079] In such Figure 1On the basis of the computer system 100 shown, the computer system 100 can also implement the function of the service node based on software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS), and provide services (e.g., computing services, storage services and network services) to users through the service node. The service node can be a cloud node (e.g., a management node, a computing node or a storage node, etc.) obtained by virtualizing the resources (e.g., computing resources and storage resources) of the computer system 100.

[0080] For example, a server of a semi-structured key-value database is deployed on the service node, and a client of the semi-structured key-value database is deployed on the user terminal. The service node implements the functions of the server deployed on the computing server.

[0081] The steps of the data processing method provided in this application are performed by the above-mentioned server and client. Figure 2a The data processing method of this application is described. Figure 2a As shown, the data processing method of the present application includes the following steps 201 to 206.

[0082] Step 201: The client sends a data read request to the server.

[0083] As a possible example, before the client sends a data operation request to the server according to a user operation, such as a data operation request such as overwrite, update, and delete, the client sends a data read request to the server. The data read request is used to obtain the data content of the semi-structured key-value data from the server.

[0084] Optionally, the data read request includes a get request, the data content is a document included in the primary key index of the semi-structured key-value data, and the get request includes the primary key of the semi-structured key-value data to be accessed.

[0085] Step 202: The server returns data content to the client according to the data read request.

[0086] As a possible example, the server responds to the data read request, reads the value contained in the primary key index of the key value data corresponding to the target primary key carried in the data read request, that is, the data content contained in the primary key index, and returns the data content to the client.

[0087] Correspondingly, the client receives the data content returned by the server, and stores the received data content as historical data content.

[0088] Step 203: The client sends a data operation request to the server.

[0089] As a possible example, after obtaining the historical data content, the client sends a data operation request to the server, wherein the data operation request includes the target data content and the historical data content.

[0090] As a possible implementation manner, the data operation request may further include a target primary key of the semi-structured key-value data to be operated, and the historical data content is the historical data content of the key-value data corresponding to the target primary key.

[0091] Optionally, the data operation indicated by the data operation request may be overwriting, updating, etc. The target data content is new data content corresponding to the overwriting, updating, etc. operation.

[0092] like Figure 2a In the example shown, the primary key of the semi-structured key-value data included in the data operation request is A-1, and the value of the semi-structured key-value data corresponding to the primary key A-1 is {A:1, B:2, C:3}. The semi-structured key-value data also includes secondary index 1 and secondary index 2. The secondary index key of secondary index 1 is B-2, and the value of secondary index 1 is A-1. The secondary index key of secondary index 2 is C-3, and the value of secondary index 2 is A-1. Among them, A:1 indicates that the value of field A is 1, B:2 indicates that the value of field B is 2, and C:3 indicates that the value of field C is 3. A-1 indicates that the value of field A is 1, B-2 indicates that the value of field B is 2, and C-3 indicates that the value of field C is 3.

[0093] For example, the data operation indicated by the data operation request is overwrite, and the data operation request also includes new data content for overwriting historical data content, such as {A:1, B:4, C:6}. The new data content can be referred to as the data content contained in the semi-structured key-value data after the server performs the overwrite data operation.

[0094] For another example, the data operation indicated by the data operation request is update, and the data operation request also includes a field for replacing part or all of the historical data content, such as {B:4}.

[0095] Step 204: The server performs the data operation corresponding to the data operation request according to the target data content.

[0096] As a possible example, the server performs corresponding data operations on the key-value data according to the data operation request and the target data content.

[0097] As a possible implementation method, the server determines the key-value data corresponding to the primary key in the semi-structured key-value database according to the primary key carried in the data operation request, performs corresponding data operations on the key-value data according to the target data content, and obtains the data content after the operation.

[0098] For example, if the data operation indicated by the data operation request is overwrite, the server uses the target data content carried in the data operation request to replace the data content of the key-value data to obtain the new data content of the key-value data. Continuing with the example that the target data content carried in the data operation request for overwriting the historical data content is {A:1,B:4,C:6}, the new data content of the key-value data is {A:1,B:4,C:6}.

[0099] For another example, if the data operation indicated by the data operation request is update, the server uses some or all of the fields carried in the data operation request to replace the corresponding fields of the data content of the key-value data to obtain the new data content of the key-value data. Continuing with the example that the target data content carried by the data operation request for replacing the historical data content is {B:4}, the new data content of the key-value data is {A:1, B:4, C:3}.

[0100] Step 205: The server determines the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deleted content and added content.

[0101] As a possible example, the server compares the historical data content with the value corresponding to the target primary key after the data operation is performed, that is, the data content, to obtain the changed content of the data content.

[0102] As a possible implementation method, the server compares the historical data content with the new data content of the key value data corresponding to the target primary key to obtain the changed content of the data.

[0103] Taking the new data content {A:1, B:4, C:6} of the key value data corresponding to the above covered data operation as an example, the server compares the historical data content {A:1, B:2, C:3} with the new data content {A:1, B:4, C:6}, and finds that the changed content is to modify field B:2 to field B:4, and field C:3 to field C:6. Among them, field B:2 and field C:3 are deleted content, and field B:4 and field C:6 are newly added content.

[0104] Taking the new data content {A:1, B:4, C:3} of the key value data corresponding to the above updated data operation as an example, the server compares the historical data content {A:1, B:2, C:3} with the new data content {A:1, B:4, C:3}, and finds that the changed content is to modify field B:2 to field B:4. Among them, field B:2 is deleted content, and field B:4 is added content.

[0105] Step 206: The server deletes the secondary index corresponding to the deleted content, and adds the secondary index corresponding to the newly added content.

[0106] As one possible example, the server updates the primary key index and secondary indexes based on the changes.

[0107] Optionally, the server generates a deletion record for the secondary index corresponding to the deleted content, and an insertion record for the primary key index and secondary index corresponding to the new content, and then writes the deletion record and the insertion record into the index file to complete the deletion of the secondary index corresponding to the deleted content and the addition of the secondary index corresponding to the new content.

[0108] For example, the newly added content corresponding to the above-covered data operations includes field B:4 and field C:6, and the deleted content includes field B:2 and field C:3. The server generates deletion records for field B:2 and field C:3, that is, deletion records for two secondary indexes with secondary index key B-2 and value A-1 and secondary index key C-3 and value A-1. The server generates insertion records for field B:4 and field C:6, that is, insertion records for two secondary indexes with secondary index key B-4 and value A-1 and secondary index key C-6 and value A-1.

[0109] For another example, the newly added content corresponding to the above updated data operation includes field B:4, and the deleted content includes field B:2. The server generates a deletion record for field B:2, that is, a deletion record for the secondary index with a secondary index key of B-2 and a value of A-1. The server generates an insertion record for field B:4, that is, an insertion record for the secondary index with a secondary index key of C-6 and a value of A-1.

[0110] exist Figure 2a In the example shown, only the secondary index corresponding to the covered data operation is shown. The secondary index corresponding to the update data operation is similar to the secondary index corresponding to the covered data operation. Figure 2a I will not go into details here.

[0111] Optionally, the index file includes an index file of a structure such as an ordered array, a binary tree, or a B+ tree.

[0112] In a possible embodiment of the present application, after step 206, the server returns a data operation response to the client.

[0113] As a possible example, the server returns a data operation response to the client to notify the client that the data operation request has been completed.

[0114] Based on the above steps 201 to 206, the client carries the data content of the key-value data that has been read, i.e., the historical data content, in the data operation request sent to the server. When the server executes the data operation corresponding to the data operation request, it can update the index of the semi-structured key-value data based on the historical data content. In this way, compared with the server reading the data content of the semi-structured key-value data in the semi-structured key-value database after receiving the data operation request, and then updating the index based on the read data content, the data content reading operation is reduced once, thereby reducing the response delay of the data operation request and improving the data processing efficiency of the semi-structured key-value database.

[0115] Combination of the above Figure 2a , taking the data operation corresponding to the data operation request as an overwrite or update operation as an example, the data processing method is explained. Figure 2b When the data operation corresponding to the data operation request is deletion, the data processing method is the same as above. Figure 2a The difference in data processing methods is shown in Figure 2. Figure 2b As shown, the data processing method of the present application includes the following steps 201' to 206'.

[0116] Step 201': the client sends a data read request to the server.

[0117] Step 202': the server returns data content to the client according to the data read request.

[0118] As a possible example, the specific implementation method of step 201'-step 202' is the same as the above-mentioned step 201-step 202, which will not be repeated here.

[0119] Step 203': the client sends a data operation request to the server.

[0120] As a possible example, step 203 ′ differs from step 203 in that the data operation request includes historical data content but does not include target data content.

[0121] As a possible implementation method, the data operation request does not carry the target data content alone, but carries the field to be deleted, such as field {C:3}, in the instruction of the deletion operation.

[0122] Step 204': the server executes the data operation corresponding to the data operation request.

[0123] As a possible example, the difference between step 204' and step 204 is that the server performs the data operation corresponding to the data operation request according to the instruction of the delete operation in the data operation request.

[0124] Step 205': the server determines the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deletion content.

[0125] As a possible example, step 205 ′ differs from step 205 in that changing the content includes deleting the content.

[0126] Taking the new data content {A:1,B:2} of the key value data corresponding to the above-mentioned deleted data operation as an example, the server compares the historical data content {A:1,B:2,C:3} with the new data content {A:1,B:2} and finds that the changed content is field C:3.

[0127] Step 206': the server deletes the secondary index corresponding to the deleted content.

[0128] As a possible example, the difference between step 206' and step 206 is that the server does not add a secondary index corresponding to the newly added content.

[0129] For example, the change content corresponding to the above-mentioned deletion operation includes field C: 3. The server generates a deletion record of field C: 3, that is, a deletion record of the secondary index with the secondary index key C-3 and the value A-1.

[0130] Based on the above steps 201-206 or steps 201'-206', the client carries the data content of the key-value data that has been read, i.e., the historical data content, in the data operation request sent to the server. When the server performs the data operation corresponding to the data operation request, it can update the index of the semi-structured key-value data based on the historical data content. In this way, compared with the server reading the data content of the semi-structured key-value data in the semi-structured key-value database after receiving the data operation request, and then updating the index based on the read data content, the data content reading operation is reduced once, thereby reducing the response delay of the data operation request and improving the data processing efficiency of the semi-structured key-value database.

[0131] The above describes the steps of interacting between the server and the client and executing the data processing method of the present application. A semi-structured key-value database maintained by a server usually needs to respond to data operation requests from multiple clients, and the historical data content carried in the data operation request sent by the client is not necessarily the latest data content of the semi-structured key-value data stored by the server. For example, after client 1 sends a data read request to the server to obtain historical data content, client 2 sends a data operation request to the server to modify the data content of the semi-structured key-value data. If client 1 initiates a data operation request to the server, the historical data content carried in the data operation request sent by client 1 is not the latest data content of the semi-structured key-value data stored by the server.

[0132] Therefore, this application is in the above Figure 2a or Figure 2b On the basis of the data processing method shown, a step of validating the historical data content can be added. When the historical data content carried in the data operation request is consistent with the data content contained in the key-value data in the semi-structured key-value database, the server can perform subsequent steps based on the historical data content. In this way, it is ensured that the historical data content carried in the data operation request sent by the client is the latest data content of the semi-structured key-value data stored by the server, so as to improve the accuracy of index update.

[0133] Next, combine Figure 3 A data processing method including validity verification of historical data content is described in detail. The data processing method includes the following steps 301 to 309.

[0134] Step 301: The client sends a data read request to the server.

[0135] As a possible example, the difference between step 301 and step 201 is that the data read request is used to obtain the data content of the key-value data and the version information of the data content from the server.

[0136] As a possible implementation manner, the version information of the data content includes data version information and table mode version information.

[0137] Optionally, the data read request includes a get request and a describle-table request. The get request is used to query the data content and data version information of the semi-structured key-value data, and the describle-table request is used to query the table mode and table mode version information. The above get request and describle-table request can be regarded as different operation requests in the same data read request, or as two requests sent independently by the client to the server.

[0138] Among them, for different key-value data in the semi-structured key-value database, different key-value data have different data content and data version information, and different key-value data have the same table schema and table schema version information. Optionally, when the client performs multiple data operations on one or more key-value data in the same semi-structured key-value database, it only needs to obtain the table schema and table schema version information once.

[0139] Step 302: The server returns data content to the client according to the data read request.

[0140] As a possible example, the server receives a data read request, reads the data content and version information of the data content contained in the primary key index of the key-value data corresponding to the target primary key carried in the data read request, and returns the data content and version information of the data content to the client.

[0141] Correspondingly, the client receives the data content and the version information of the data content returned by the server, and stores the received data content as the historical data content, and stores the version information of the received data content as the version information of the historical data content.

[0142] Step 303: The client sends a data operation request to the server.

[0143] As a possible example, after obtaining the historical data content, the client sends a data operation request to the server, wherein the data operation request includes the historical data content and the version information of the historical data content.

[0144] Compared with step 203, step 303 carries version information of historical data content in addition to the data operation request. The specific implementation method of step 303 is the same as that of step 203 and will not be repeated here.

[0145] Step 304 : The server determines, based on the version information of the historical data content, that the historical data content is consistent with the data content included in the key-value data in the semi-structured key-value database, and executes step 307 .

[0146] As a possible example, when the version information of the historical data content is consistent with the version information of the data content of the semi-structured key-value data cached by the server, the server determines that the historical data content is consistent with the data content contained in the semi-structured key-value data in the semi-structured key-value database.

[0147] As a possible implementation method, the version information of the historical data content includes data version information and table schema version information. The version information of the historical data content is consistent with the version information of the data content of the key-value data cached by the server, which means that the data version information and table schema version information corresponding to the key-value data carried by the data operation request are consistent with the data version information and table schema version information of the corresponding key-value data cached by the server.

[0148] Optionally, when the server returns version information of the data content to the client according to the data read request of the client, the server stores the version information of the data content in the cache.

[0149] Optionally, when the cache duration of the version information of the data content in the cache exceeds a preset duration, the server deletes the version information of the data content whose cache duration exceeds the preset duration.

[0150] As another possible example, the server compares the historical data content with the data content of the key-value data stored on the server, and when the two are the same, determines that the historical data content is consistent with the data content included in the key-value data in the semi-structured key-value database.

[0151] In the present application, the server's caching of version information of data content may be implemented by a semi-structured key-value database.

[0152] Step 305 : The server determines, based on the version information of the historical data content, that the historical data content is inconsistent with the data content included in the key-value data in the semi-structured key-value database, and executes step 306 .

[0153] For the method of determining whether the historical data content is consistent with the data content included in the key-value data in the semi-structured key-value database in the above step 305, please refer to step 304, which will not be repeated here.

[0154] Step 306: The server reads historical data content from the semi-structured key-value database.

[0155] As a possible implementation method, the server reads the data content contained in the key-value data from the semi-structured key-value database maintained by the server according to the target primary key carried in the data operation request, and uses the read data content as the historical data content.

[0156] The method in which the server reads data content from the semi-structured key-value database according to the primary key can be to query the semi-structured key-value data using an index file based on the target primary key, which will not be described in detail here.

[0157] Step 307: The server performs the data operation corresponding to the data operation request.

[0158] Step 308: The server determines the change content of the value corresponding to the target primary key according to the historical data content.

[0159] Step 309: The server updates the secondary index according to the changed content.

[0160] For the specific implementation of the above steps 307 to 309, please refer to Figure 2a Steps 204 to 206 are shown and will not be described in detail here.

[0161] In the above-mentioned embodiments of the present application, the server provides an application program interface of a semi-structured key-value database to the client, and the interaction between the client and the server is realized through the application program interface.

[0162] For example, the application program interface includes a table query interface, a data content query interface, a write request interface, etc.

[0163] The table query interface is used to query the table schema and table schema version information from the server through the describle-table request. The input parameters of the table query interface include the table schema name, and the return parameters include the table schema and table schema version information.

[0164] The data content query interface is used to query the data content of semi-structured key-value data from the server through a get request. The input parameters of the data content query interface include the target primary key, and the return parameters include the data content and data version information.

[0165] The write request interface includes data operation request interfaces of types such as overwrite, update, delete, etc. The input parameters of the write request interface include the target data content after overwrite, the historical data content, and the version information of the historical data content.

[0166] Based on the above steps 301 to 309, Figure 2a or Figure 2b The data processing method shown adds validity verification of historical data content. When the server confirms that the historical data content carried in the data operation request is the latest record stored in the semi-structured key-value database, it updates the secondary index based on the historical data content, thereby avoiding the situation where the data content of the semi-structured key-value database on the server has been modified and the secondary index is incorrectly updated based on the wrong version of the historical data content. In this way, the accuracy of data processing by the server in multi-client scenarios is improved.

[0167] In order to cooperate with the above Figure 2a or Figure 2b The present application also provides a data processing device 400, which can be used to implement the above data processing method. Figure 2a or Figure 2b The functions of the server in the data processing method shown in FIG. Figure 4 As shown, the data processing device 400 includes a transceiver module 410 , an operation module 420 , and an index module 430 .

[0168] The transceiver module 410 is used to receive a data operation request sent by a client. The data operation request includes target data content and historical data content. The historical data content is the value corresponding to the target primary key before the server responds to the data operation request.

[0169] The operation module 420 is used to perform the data operation corresponding to the data operation request, and the data operation includes updating or overwriting. Figure 2a Step 204 shown, or executing Figure 2b Step 204' is shown.

[0170] The operation module 420 is further used to determine the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deletion content and addition content. Figure 2a Step 205 shown, or executing Figure 2b Step 205' is shown.

[0171] Index module 430 is used to update the secondary index according to the changed content. For example, index module 430 is used to perform the following steps: Figure 2a Step 206 shown, or executing Figure 2b Step 206' is shown.

[0172] The transceiver module 410 is further configured to send a data operation response to the client.

[0173] The data processing device 400 may further include a cache module 440 and a storage module 450. The cache module 440 is used to cache version information of data content. The storage module 450 is used to store semi-structured key-value data.

[0174] Among them, the transceiver module 410, the operation module 420, the index module 430, the cache module 440 and the storage module 450 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the operation module 420 is introduced below by taking the operation module 420 as an example. Similarly, the implementation of the transceiver module 410, the index module 430, the cache module 440 and the storage module 450 can refer to the implementation of the operation module 420.

[0175] As an example of a software functional unit, the operation module 420 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the operation module 420 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0176] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0177] As an example of a hardware functional unit, the operation module 420 may include at least one computing device, such as a server, etc. Alternatively, the operation module 420 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0178] The multiple computing devices included in the operation module 420 can be distributed in the same region or in different regions. The multiple computing devices included in the operation module 420 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the operation module 420 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0179] It should be noted that, in other embodiments, any one of the transceiver module 410, the operation module 420, the index module 430, the cache module 440 and the storage module 450 can be used to execute any step in the data processing method, and the steps that the transceiver module 410, the operation module 420, the index module 430, the cache module 440 and the storage module 450 are responsible for implementing can be specified as needed. The full functions of the data processing device 400 are realized by respectively implementing different steps in the data processing method through the transceiver module 410, the operation module 420, the index module 430, the cache module 440 and the storage module 450.

[0180] In order to cooperate with the above Figure 2a or Figure 2b The present application also provides a data processing device 500, which can be used to implement the above data processing method. Figure 2a or Figure 2b The client functions in the data processing method shown. Figure 5 As shown, the data processing device 500 includes:

[0181] The transceiver module 510 is used to send a data operation request to the server. The data operation request includes target data content and historical data content. The historical data content is the value corresponding to the target primary key before the server responds to the data operation request. Figure 2a Step 203 shown, or executing Figure 2b Step 203' is shown.

[0182] The transceiver module 510 is also used to receive a data operation response sent by the server.

[0183] The transceiver module 510 may also be implemented as a software functional unit or a hardware functional unit. The specific functional implementation of the transceiver module 510 may refer to the example description of the operation module 420, which will not be described in detail here.

[0184] This application cooperates Figure 2a or Figure 2b The server in the embodiment provides the above-mentioned data processing device 400, in cooperation with Figure 2a or Figure 2b The client in the data processing device 500 is provided. The modules with the same functions in the data processing device 400 and the data processing device 500 can be the same module in the system composed of the client and the server. For example, the transceiver module 410 and the transceiver module 510 can be combined into the same transceiver module.

[0185] Since the data processing method provided by this application is Figure 2a or Figure 2b In the steps shown, the server and the client need to communicate with each other to complete. Therefore, the present application also provides a data processing system 600, such as Figure 6 As shown, the data processing system 600 includes a client 610 and a server 620 .

[0186] The client 610 is used to send a data operation request to the server 620, where the data operation request includes historical data content, which is data content included in the semi-structured key-value data before the server 620 executes the data operation corresponding to the data operation request.

[0187] The server 620 is used to perform a data operation corresponding to the data operation request on the data content.

[0188] The server 620 is further configured to determine, based on the historical data content, changes to the data content after the server 620 executes the data operation corresponding to the data operation request.

[0189] The server 620 is also used to update the secondary index according to the changed content.

[0190] The server 620 is further configured to send a data operation response to the client 610 .

[0191] Both the client 610 and the server 620 can be implemented by software or hardware. As an example, the implementation of the server 620 is described below. Similarly, the implementation of the client 610 can refer to the implementation of the server 620.

[0192] As an example of a software functional unit, the server 620 may include code running on a computing instance. The computing instance may be at least one of a physical host (computing device), a virtual machine, a container, and other computing devices. Furthermore, the computing device may be one or more. For example, the server 620 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Typically, a region may include multiple AZs.

[0193] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway must be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0194] In the embodiment of the present application, the server 620 may be used to execute Figure 2a or Figure 2b The software module of the steps executed by the server in the data processing method shown in FIG. Figure 4 The data processing device 400 shown. The client 610 is used to execute Figure 2a or Figure 2b The software module executed by the client in the data processing method shown in the figure may also be as follows Figure 5 The data processing device 500 is shown.

[0195] As an example of a hardware functional unit, the server 620 may include at least one computing device, such as a server, etc. Alternatively, the server 620 may also be a device implemented using ASIC or PLD, etc. The PLD may be implemented using CPLD, FPGA, GAL or any combination thereof.

[0196] The multiple computing devices included in the server 620 can be distributed in the same region or in different regions. The multiple computing devices included in the YY device can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the YY device can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0197] In an embodiment of the present application, the server 620 may be a computing server in the computer system 100 , and the client 610 may be a storage server in the computer system 100 .

[0198] The present application also provides a computing device 700. Figure 7 As shown, computing device 700 includes: bus 702, processor 704, memory 706 and communication interface 708. Processor 704, memory 706 and communication interface 708 communicate through bus 702. Computing device 700 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 700.

[0199] The bus 702 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 702 may include a path for transmitting information between various components of the computing device 700 (eg, the memory 706, the processor 704, and the communication interface 708).

[0200] The processor 704 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0201] The memory 706 may include a volatile memory, such as a random access memory (RAM). The processor 704 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0202] The memory 706 stores executable program codes, and the processor 704 executes the executable program codes to respectively implement the functions of the modules included in the aforementioned data processing device 400 or the data processing device 500, thereby implementing the data processing method. That is, the memory 706 stores instructions for executing the data processing method.

[0203] Alternatively, the memory 706 stores executable codes, and the processor 704 executes the executable codes to respectively implement the functions of the client 610 or the server 620, thereby implementing the data processing method. That is, the memory 706 stores instructions for executing the data processing method.

[0204] The communication interface 708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 700 and other devices or communication networks.

[0205] Considering that the data processing method provided in the present application is applied to the computer system 100, the respective infrastructures of the computing server cluster 110 and the storage server cluster 120 of the computer system 100 usually include multiple computing devices. Therefore, the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop, a laptop, or a smart phone.

[0206] like Figure 8As shown, the computing device cluster includes at least one computing device 700. The memory 706 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the data processing method.

[0207] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the data processing method. In other words, the combination of one or more computing devices 700 may jointly execute instructions for executing the data processing method.

[0208] It should be noted that the memory 706 in different computing devices 700 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the data processing device 400 or the data processing device 500. That is, the instructions stored in the memory 706 in different computing devices 700 may implement the functions of one or more modules included in the data processing device 400 or the data processing device 500.

[0209] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig. 9 A possible implementation is shown. Fig. 9 As shown, two computing devices 700A and 700B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 706 in the computing device 700A stores instructions for executing the functions of one or more modules in the transceiver module 410, the operation module 420, the index module 430, the cache module 440, and the storage module 450. Fig. 9 In the example, the memory 706 in the computing device 700A stores instructions for executing the functions of the transceiver module 410. Meanwhile, the memory 706 in the computing device 700B stores instructions for executing the functions of one or more modules among the transceiver module 410, the operation module 420, the index module 430, the cache module 440 and the storage module 450. Fig. 9 In the example, the memory 706 in the computing device 700B stores instructions for executing the functions of the operation module 420 and the index module 430.

[0210] It should be understood that Fig. 9 The functions of the computing device 700A shown in FIG. 7 may also be completed by multiple computing devices 700. Similarly, the functions of the computing device 700B may also be completed by multiple computing devices 700.

[0211] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the following steps: Figure 2a or Figure 2b The data processing method shown, or Figure 2a or Figure 2b The steps performed by the client or server in the data processing method shown.

[0212] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to store data such as Figure 2a or Figure 2b The data processing method shown, or Figure 2a or Figure 2b The steps performed by the client or server in the data processing method shown.

[0213] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0214] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0215] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0216] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0217] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0218] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0219] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0220] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b and c can be single or multiple.

[0221] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method, characterized in that: The method is applied to a semi-structured key-value database, the semi-structured key-value database includes a client and a server, the key-value data in the semi-structured key-value database includes a primary key index and a secondary index, the primary key index includes a primary key and a value corresponding to the primary key, the value corresponding to the primary key is the data content contained in the key-value data, the secondary index is used to point to the primary key index based on partial data content, and the method includes: The client sends a data operation request to the server, wherein the data operation request includes target data content and historical data content, wherein the historical data content is a value corresponding to the target primary key before the server responds to the data operation request; The server performs the data operation corresponding to the data operation request according to the target data content, and the data operation includes updating or overwriting; The server determines the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deleted content and added content; The server deletes the secondary index corresponding to the deleted content; The server adds a secondary index corresponding to the newly added content.

2. The method according to claim 1, characterized in that Before the server determines the change content of the value corresponding to the target primary key according to the historical data content, the method further includes: The server determines that the historical data content is consistent with the data content included in the key-value data in the semi-structured key-value database.

3. The method according to claim 2, characterized in that The data operation request also includes version information of the historical data content; The server determines that the historical data content is consistent with the data content included in the key-value data in the semi-structured key-value database, including: the server determines that the version information of the historical data content is consistent with the version information of the data content included in the key-value data in the semi-structured key-value database.

4. The method according to claim 3, characterized in that Before the client sends a data operation request to the server, the method further includes: The client sends a data reading request to the server; The server sends the historical data content of the key-value data and version information of the historical data content to the client according to the data read request; The client receives the historical data content and version information of the historical data content.

5. The method according to claim 3 or 4, characterized in that: The version information includes table mode version information and data version information, and the data version information is used to indicate the version of the data content.

6. The method according to any one of claims 1 to 5, characterized in that The server determines the change content of the value corresponding to the target primary key according to the historical data content, including: The server compares the historical data content with the data content after the data operation is performed to obtain the changed content of the value corresponding to the target primary key.

7. A data processing method, characterized in that: The method is applied to a semi-structured key-value database, the semi-structured key-value database includes a client and a server, the key-value data in the semi-structured key-value database includes a primary key index and a secondary index, the primary key index includes a primary key and a value corresponding to the primary key, the value corresponding to the primary key is the data content contained in the key-value data, the secondary index is used to point to the primary key index based on partial data content, and the method includes: The client sends a data operation request to the server, wherein the data operation request includes historical data content, and the historical data content is a value corresponding to a target primary key before the server responds to the data operation request; The server performs a data operation corresponding to the data operation request, wherein the data operation includes deletion; The server determines the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deletion content; The server deletes the secondary index corresponding to the deleted content.

8. The method according to claim 7, characterized in that The data operation request also includes version information of the historical data content. Before the server determines the change content of the value corresponding to the target primary key according to the historical data content, the method also includes: The server determines that the version information of the historical data content is consistent with the version information of the data content included in the key-value data in the semi-structured key-value database.

9. The method according to claim 8, characterized in that Before the client sends a data operation request to the server, the method further includes: The client sends a data reading request to the server; The server sends the historical data content of the key-value data and version information of the historical data content to the client according to the data read request; The client receives the historical data content and version information of the historical data content.

10. A data processing system, characterized in that: The system includes a client and a server; The client is used to send a data operation request to the server, wherein the data operation request includes target data content and / or historical data content, and the historical data content is a value corresponding to a target primary key before the server responds to the data operation request; The server is used to perform the data operation corresponding to the data operation request according to the target data content; The server is further configured to determine the change content of the value corresponding to the target primary key according to the historical data content, wherein the change content includes deletion content and / or addition content; The server is further configured to delete the secondary index corresponding to the deleted content and / or add the secondary index corresponding to the newly added content.

11. A data processing device, characterized in that: The device comprises: A transceiver module, configured to receive a data operation request sent by the client, wherein the data operation request includes target data content and historical data content, wherein the historical data content is a value corresponding to a target primary key before the server responds to the data operation request; An operation module, configured to perform a data operation corresponding to the data operation request according to the target data content, wherein the data operation includes updating or overwriting; The operation module is further used to determine the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deleted content and added content; The index module is used to delete the secondary index corresponding to the deleted content and add the secondary index corresponding to the newly added content.

12. A data processing device, characterized in that: The device comprises: A transceiver module, configured to receive a data operation request sent by the client, wherein the data operation request includes historical data content, and the historical data content is a value corresponding to a target primary key before the server responds to the data operation request; An operation module, used to execute a data operation corresponding to the data operation request, wherein the data operation includes deletion; The operation module is further used to determine the change content of the value corresponding to the target primary key according to the historical data content, and the change content includes deletion content; The index module is used to delete the secondary index corresponding to the deleted content.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 7 to 9.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

16. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 7 to 9.

17. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

18. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 7 to 9.

Images